From afa at codespeak.net Fri Oct 1 00:18:01 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 1 Oct 2010 00:18:01 +0200 (CEST)
Subject: [pypy-svn] r77512 -
pypy/branch/fast-forward/lib-python/modified-2.7.0/test
Message-ID: <20100930221801.84632282BDB@codespeak.net>
Author: afa
Date: Fri Oct 1 00:17:59 2010
New Revision: 77512
Modified:
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py
Log:
Make test_support.findfile search files in 2.7.0, not
in modified-2.7.0.
Exact copy of change r58845.
Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py
==============================================================================
--- pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py Fri Oct 1 00:17:59 2010
@@ -430,16 +430,20 @@
rmtree(name)
-def findfile(file, here=__file__, subdir=None):
+def findfile(file, here=None, subdir=None):
"""Try to find a file on sys.path and the working directory. If it is not
found the argument passed to the function is returned (this does not
necessarily signal failure; could still be the legitimate path)."""
+ import test
if os.path.isabs(file):
return file
if subdir is not None:
file = os.path.join(subdir, file)
path = sys.path
- path = [os.path.dirname(here)] + path
+ if here is None:
+ path = test.__path__ + path
+ else:
+ path = [os.path.dirname(here)] + path
for dn in path:
fn = os.path.join(dn, file)
if os.path.exists(fn): return fn
From afa at codespeak.net Fri Oct 1 00:30:45 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 1 Oct 2010 00:30:45 +0200 (CEST)
Subject: [pypy-svn] r77513 - pypy/branch/fast-forward/lib-python
Message-ID: <20100930223045.56FD5282BDB@codespeak.net>
Author: afa
Date: Fri Oct 1 00:30:43 2010
New Revision: 77513
Modified:
pypy/branch/fast-forward/lib-python/conftest.py
Log:
Reduce timeout to 100 seconds per test script.
Today 7 of them reached the 1000s limit,
and the test lasted more than 3 hours.
Modified: pypy/branch/fast-forward/lib-python/conftest.py
==============================================================================
--- pypy/branch/fast-forward/lib-python/conftest.py (original)
+++ pypy/branch/fast-forward/lib-python/conftest.py Fri Oct 1 00:30:43 2010
@@ -30,7 +30,7 @@
def pytest_addoption(parser):
group = parser.getgroup("complicance testing options")
group.addoption('-T', '--timeout', action="store", type="string",
- default="1000", dest="timeout",
+ default="100", dest="timeout",
help="fail a test module after the given timeout. "
"specify in seconds or 'NUMmp' aka Mega-Pystones")
group.addoption('--pypy', action="store", type="string",
From agaynor at codespeak.net Fri Oct 1 00:41:43 2010
From: agaynor at codespeak.net (agaynor at codespeak.net)
Date: Fri, 1 Oct 2010 00:41:43 +0200 (CEST)
Subject: [pypy-svn] r77514 - in pypy/branch/fast-forward/pypy/objspace/std:
. test
Message-ID: <20100930224143.4FAD5282BDB@codespeak.net>
Author: agaynor
Date: Fri Oct 1 00:41:41 2010
New Revision: 77514
Modified:
pypy/branch/fast-forward/pypy/objspace/std/inttype.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_intobject.py
Log:
Added int.bit_length.
Modified: pypy/branch/fast-forward/pypy/objspace/std/inttype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/inttype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/inttype.py Fri Oct 1 00:41:41 2010
@@ -15,6 +15,18 @@
def int_conjugate__ANY(space, w_int):
return space.pos(w_int)
+int_bit_length = SMM("bit_length", 1, doc="int.bit_length() -> int\n\nNumber of bits necessary to represent self in binary.\n>>> bin(37)\n'0b100101'\n>>> (37).bit_length()\n6")
+
+def int_bit_length__ANY(space, w_int):
+ val = space.int_w(w_int)
+ if val < 0:
+ val = -val
+ bits = 0
+ while val:
+ bits += 1
+ val >>= 1
+ return space.wrap(bits)
+
register_all(vars(), globals())
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_intobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_intobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_intobject.py Fri Oct 1 00:41:41 2010
@@ -437,6 +437,18 @@
# __eq__ & the others.
assert 1 .__cmp__
assert int .__cmp__
+
+ def test_bit_length(self):
+ for val, bits in [
+ (0, 0),
+ (1, 1),
+ (10, 4),
+ (150, 8),
+ (-1, 1),
+ (-10, 4),
+ (-150, 8),
+ ]:
+ assert val.bit_length() == bits
class AppTestIntOptimizedAdd(AppTestInt):
From afa at codespeak.net Fri Oct 1 00:48:30 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 1 Oct 2010 00:48:30 +0200 (CEST)
Subject: [pypy-svn] r77515 - in pypy/trunk/pypy/interpreter: . test
Message-ID: <20100930224830.270BC282BDB@codespeak.net>
Author: afa
Date: Fri Oct 1 00:48:28 2010
New Revision: 77515
Modified:
pypy/trunk/pypy/interpreter/function.py
pypy/trunk/pypy/interpreter/test/test_function.py
Log:
issue560 resolved: new.instancemethod should complain
when both self and class are None.
Merge of r75709 from branch/fast-forwad
Modified: pypy/trunk/pypy/interpreter/function.py
==============================================================================
--- pypy/trunk/pypy/interpreter/function.py (original)
+++ pypy/trunk/pypy/interpreter/function.py Fri Oct 1 00:48:28 2010
@@ -430,8 +430,11 @@
self.w_class = w_class # possibly space.w_None
def descr_method__new__(space, w_subtype, w_function, w_instance, w_class=None):
- if space.is_w( w_instance, space.w_None ):
+ if space.is_w(w_instance, space.w_None):
w_instance = None
+ if w_instance is None and space.is_w(w_class, space.w_None):
+ raise OperationError(space.w_TypeError,
+ space.wrap("unbound methods must have class"))
method = space.allocate_instance(Method, w_subtype)
Method.__init__(method, space, w_function, w_instance, w_class)
return space.wrap(method)
Modified: pypy/trunk/pypy/interpreter/test/test_function.py
==============================================================================
--- pypy/trunk/pypy/interpreter/test/test_function.py (original)
+++ pypy/trunk/pypy/interpreter/test/test_function.py Fri Oct 1 00:48:28 2010
@@ -482,6 +482,11 @@
raises(TypeError, m, MyInst(None))
raises(TypeError, m, MyInst(42))
+ def test_invalid_creation(self):
+ import new
+ def f(): pass
+ raises(TypeError, new.instancemethod, f, None)
+
class TestMethod:
def setup_method(self, method):
From agaynor at codespeak.net Fri Oct 1 03:45:23 2010
From: agaynor at codespeak.net (agaynor at codespeak.net)
Date: Fri, 1 Oct 2010 03:45:23 +0200 (CEST)
Subject: [pypy-svn] r77516 - pypy/branch/fast-forward/pypy/objspace/std
Message-ID: <20101001014523.7CCBC282BDB@codespeak.net>
Author: agaynor
Date: Fri Oct 1 03:45:20 2010
New Revision: 77516
Modified:
pypy/branch/fast-forward/pypy/objspace/std/complexobject.py
Log:
Don't let an RPython exception escape from abs(complex).
Modified: pypy/branch/fast-forward/pypy/objspace/std/complexobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/complexobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/complexobject.py Fri Oct 1 03:45:20 2010
@@ -216,7 +216,10 @@
return W_ComplexObject(w_complex.realval, w_complex.imagval)
def abs__Complex(space, w_complex):
- return space.newfloat(math.hypot(w_complex.realval, w_complex.imagval))
+ try:
+ return space.newfloat(math.hypot(w_complex.realval, w_complex.imagval))
+ except OverflowError, e:
+ raise OperationError(space.w_OverflowError, space.wrap(str(e)))
def eq__Complex_Complex(space, w_complex1, w_complex2):
return space.newbool((w_complex1.realval == w_complex2.realval) and
From arigo at codespeak.net Fri Oct 1 11:31:30 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 1 Oct 2010 11:31:30 +0200 (CEST)
Subject: [pypy-svn] r77517 - pypy/extradoc/sprintinfo/ddorf2010
Message-ID: <20101001093130.A5413282BDB@codespeak.net>
Author: arigo
Date: Fri Oct 1 11:31:28 2010
New Revision: 77517
Added:
pypy/extradoc/sprintinfo/ddorf2010/
- copied from r77448, pypy/extradoc/sprintinfo/ddorf2009/
Removed:
pypy/extradoc/sprintinfo/ddorf2010/planning.txt
Modified:
pypy/extradoc/sprintinfo/ddorf2010/announce.txt
pypy/extradoc/sprintinfo/ddorf2010/people.txt
Log:
The next sprint!
Modified: pypy/extradoc/sprintinfo/ddorf2010/announce.txt
==============================================================================
--- pypy/extradoc/sprintinfo/ddorf2009/announce.txt (original)
+++ pypy/extradoc/sprintinfo/ddorf2010/announce.txt Fri Oct 1 11:31:28 2010
@@ -1,39 +1,15 @@
-D?sseldorf PyPy sprint November 6 - November 13 2009
+D?sseldorf PyPy sprint October 25th-31st 2010
=====================================================
The next PyPy sprint will be held in the Computer Science department of
-Heinrich-Heine Universit?t D?sseldorf from the 6th to the 13th of
-November 2009. This is a fully public sprint, everyone is welcome to
+Heinrich-Heine Universit?t D?sseldorf from the 25th to the 31st of
+October 2010. This is a fully public sprint, everyone is welcome to
join us.
Topics and goals
----------------
-At the sprint we intend to work on the JIT generator in PyPy and on
-applying it to PyPy Python interpreter.
-
-The precise work that will be done is not fixed, as we don't know in
-which state the JIT will be in November. However, possible areas of
-work might include:
-
-- tweaking the interpreter/objspace to be more JIT-friendly, e.g.
- instance implementation code, call code
-- if there is interest starting non x86-32 JIT backends
-- trying out existing software to find features where the optimizations
- of the JIT could be improved
-- improving our benchmarking infrastructure
-
-We will give special priority to topics that "non-core" people find
-interesting (as long as they are somehow JIT-related).
-
-For an introduction of how our JIT-generation process works, please
-refer to our blog:
-
-http://morepypy.blogspot.com/2009/03/jit-bit-of-look-inside.html
-
-There is also a more dense academic paper about the subject:
-
-http://codespeak.net/svn/pypy/extradoc/talk/icooolps2009/bolz-tracing-jit-final.pdf
+Open. (to be defined more precisely)
Location
--------
@@ -54,4 +30,4 @@
you have codespeak commit rights).
.. _`pypy-sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint
-.. _`people`: http://codespeak.net/pypy/extradoc/sprintinfo/ddorf2009/people.txt
+.. _`people`: http://codespeak.net/pypy/extradoc/sprintinfo/ddorf2010/people.txt
Modified: pypy/extradoc/sprintinfo/ddorf2010/people.txt
==============================================================================
--- pypy/extradoc/sprintinfo/ddorf2009/people.txt (original)
+++ pypy/extradoc/sprintinfo/ddorf2010/people.txt Fri Oct 1 11:31:28 2010
@@ -1,4 +1,4 @@
-People coming to the Duesseldorf sprint November 2009
+People coming to the Duesseldorf sprint October 2010
====================================================
People who have a ``?`` in their arrive/depart or accomodation
@@ -9,11 +9,8 @@
Name Arrive/Depart Accomodation
==================== ============== =====================
Carl Friedrich Bolz always there private
-Antonio Cuni 6/11 - 13/11 Hotel Haus Hillesheim
-Armin Rigo 6/11 - 13/11 Hotel Haus Hillesheim
-Holger Krekel 9/11 - 13/11 ?
-Samuele Pedroni 5/11 - 13/11 Hotel Blaettler
-==================== ============== =====================
+Antonio Cuni leaves the 30 private
+Armin Rigo always there private
==================== ============== =====================
@@ -21,6 +18,8 @@
==================== ============== =====================
Name Arrive/Depart Accomodation
==================== ============== =====================
+Holger Krekel
+Samuele Pedroni
Armin Rigo
Michael Hudson
Anders Chrigstroem
@@ -44,9 +43,7 @@
Laura Creighton ? ?
Beatrice Duering ? ?
Johan Hahn ? ?
-Holger Krekel ? ?
Anders Chrigstroem ? ?
-Samuele Pedroni ? ?
Anders Lehmann ? ?
Niklaus Haldimann ? ?
==================== ============== =====================
From arigo at codespeak.net Fri Oct 1 11:41:31 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 1 Oct 2010 11:41:31 +0200 (CEST)
Subject: [pypy-svn] r77518 - in pypy/trunk/pypy: jit/backend
jit/backend/test jit/backend/x86 jit/metainterp
jit/metainterp/optimizeopt jit/tl translator translator/goal
Message-ID: <20101001094131.30A83282BDB@codespeak.net>
Author: arigo
Date: Fri Oct 1 11:41:29 2010
New Revision: 77518
Added:
pypy/trunk/pypy/jit/backend/conftest.py
- copied unchanged from r77517, pypy/branch/jit-str-fix/pypy/jit/backend/conftest.py
pypy/trunk/pypy/jit/tl/jittest.py
- copied unchanged from r77517, pypy/branch/jit-str-fix/pypy/jit/tl/jittest.py
Removed:
pypy/trunk/pypy/jit/backend/test/conftest.py
Modified:
pypy/trunk/pypy/jit/backend/test/test_ll_random.py
pypy/trunk/pypy/jit/backend/test/test_random.py
pypy/trunk/pypy/jit/backend/x86/regalloc.py
pypy/trunk/pypy/jit/backend/x86/runner.py
pypy/trunk/pypy/jit/metainterp/executor.py
pypy/trunk/pypy/jit/metainterp/optimizeopt/__init__.py
pypy/trunk/pypy/jit/metainterp/warmspot.py
pypy/trunk/pypy/translator/driver.py
pypy/trunk/pypy/translator/goal/translate.py
Log:
Merge branch/jit-str-fix, fixing the x86 backend's production
of COPYSTRCONTENT and adding the --jittest option to translate.py.
Modified: pypy/trunk/pypy/jit/backend/test/test_ll_random.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/test/test_ll_random.py (original)
+++ pypy/trunk/pypy/jit/backend/test/test_ll_random.py Fri Oct 1 11:41:29 2010
@@ -386,6 +386,20 @@
v_string = self.get_string(builder, r)
builder.do(self.opnum, [v_string])
+class AbstractCopyContentOperation(AbstractStringOperation):
+ def produce_into(self, builder, r):
+ v_srcstring = self.get_string(builder, r)
+ v_dststring = self.get_string(builder, r)
+ if v_srcstring.value == v_dststring.value: # because it's not a
+ raise test_random.CannotProduceOperation # memmove(), but memcpy()
+ srclen = len(v_srcstring.getref(self.ptr).chars)
+ dstlen = len(v_dststring.getref(self.ptr).chars)
+ v_length = builder.get_index(min(srclen, dstlen), r)
+ v_srcstart = builder.get_index(srclen - v_length.value + 1, r)
+ v_dststart = builder.get_index(dstlen - v_length.value + 1, r)
+ builder.do(self.opnum, [v_srcstring, v_dststring,
+ v_srcstart, v_dststart, v_length])
+
class StrGetItemOperation(AbstractGetItemOperation, _StrOperation):
pass
@@ -404,6 +418,13 @@
class UnicodeLenOperation(AbstractStringLenOperation, _UnicodeOperation):
pass
+class CopyStrContentOperation(AbstractCopyContentOperation, _StrOperation):
+ pass
+
+class CopyUnicodeContentOperation(AbstractCopyContentOperation,
+ _UnicodeOperation):
+ pass
+
# there are five options in total:
# 1. non raising call and guard_no_exception
@@ -577,6 +598,8 @@
OPERATIONS.append(UnicodeSetItemOperation(rop.UNICODESETITEM))
OPERATIONS.append(StrLenOperation(rop.STRLEN))
OPERATIONS.append(UnicodeLenOperation(rop.UNICODELEN))
+ OPERATIONS.append(CopyStrContentOperation(rop.COPYSTRCONTENT))
+ #OPERATIONS.append(CopyUnicodeContentOperation(rop.COPYUNICODECONTENT))
for i in range(2):
OPERATIONS.append(GuardClassOperation(rop.GUARD_CLASS))
Modified: pypy/trunk/pypy/jit/backend/test/test_random.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/test/test_random.py (original)
+++ pypy/trunk/pypy/jit/backend/test/test_random.py Fri Oct 1 11:41:29 2010
@@ -1,7 +1,7 @@
import py, sys
from pypy.rlib.rarithmetic import intmask, LONG_BIT
from pypy.rpython.lltypesystem import llmemory
-from pypy.jit.backend.test import conftest as demo_conftest
+from pypy.jit.backend import conftest as demo_conftest
from pypy.jit.metainterp.history import BasicFailDescr, TreeLoop
from pypy.jit.metainterp.history import BoxInt, ConstInt, LoopToken
from pypy.jit.metainterp.history import BoxPtr, ConstPtr
@@ -102,7 +102,7 @@
elif isinstance(v, ConstFloat):
args.append('ConstFloat(%r)' % v.value)
elif isinstance(v, ConstInt):
- args.append('ConstInt(%d)' % v.value)
+ args.append('ConstInt(%s)' % v.value)
else:
raise NotImplementedError(v)
if op.getdescr() is None:
@@ -113,7 +113,7 @@
except AttributeError:
descrstr = ', descr=...'
print >>s, ' ResOperation(rop.%s, [%s], %s%s),' % (
- opname[op.opnum], ', '.join(args), names[op.result], descrstr)
+ opname[op.getopnum()], ', '.join(args), names[op.result], descrstr)
#if getattr(op, 'suboperations', None) is not None:
# subops.append(op)
@@ -189,7 +189,7 @@
v.value)
print >>s, ' op = cpu.execute_token(looptoken)'
if self.should_fail_by is None:
- fail_args = self.loop.operations[-1].args
+ fail_args = self.loop.operations[-1].getarglist()
else:
fail_args = self.should_fail_by.getfailargs()
for i, v in enumerate(fail_args):
Modified: pypy/trunk/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/regalloc.py Fri Oct 1 11:41:29 2010
@@ -959,18 +959,23 @@
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(args[0], args)
ofs_loc = self.rm.make_sure_var_in_reg(args[2], args)
+ assert args[0] is not args[1] # forbidden case of aliasing
self.rm.possibly_free_var(args[0])
- self.rm.possibly_free_var(args[2])
+ if args[3] is not args[2] is not args[4]: # MESS MESS MESS: don't free
+ self.rm.possibly_free_var(args[2]) # it if ==args[3] or args[4]
srcaddr_box = TempBox()
- srcaddr_loc = self.rm.force_allocate_reg(srcaddr_box)
+ forbidden_vars = [args[1], args[3], args[4], srcaddr_box]
+ srcaddr_loc = self.rm.force_allocate_reg(srcaddr_box, forbidden_vars)
self._gen_address_inside_string(base_loc, ofs_loc, srcaddr_loc)
# compute the destination address
- base_loc = self.rm.make_sure_var_in_reg(args[1], args)
- ofs_loc = self.rm.make_sure_var_in_reg(args[3], args)
+ base_loc = self.rm.make_sure_var_in_reg(args[1], forbidden_vars)
+ ofs_loc = self.rm.make_sure_var_in_reg(args[3], forbidden_vars)
self.rm.possibly_free_var(args[1])
- self.rm.possibly_free_var(args[3])
+ if args[3] is not args[4]: # more of the MESS described above
+ self.rm.possibly_free_var(args[3])
+ forbidden_vars = [args[4], srcaddr_box]
dstaddr_box = TempBox()
- dstaddr_loc = self.rm.force_allocate_reg(dstaddr_box)
+ dstaddr_loc = self.rm.force_allocate_reg(dstaddr_box, forbidden_vars)
self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc)
# call memcpy()
length_loc = self.loc(args[4])
Modified: pypy/trunk/pypy/jit/backend/x86/runner.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/runner.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/runner.py Fri Oct 1 11:41:29 2010
@@ -87,7 +87,9 @@
def execute_token(self, executable_token):
addr = executable_token._x86_bootstrap_code
+ #llop.debug_print(lltype.Void, ">>>> Entering", addr)
func = rffi.cast(lltype.Ptr(self.BOOTSTRAP_TP), addr)
+ #llop.debug_print(lltype.Void, "<<<< Back")
fail_index = self._execute_call(func)
return self.get_fail_descr_from_number(fail_index)
@@ -99,10 +101,7 @@
LLInterpreter.current_interpreter = self.debug_ll_interpreter
res = 0
try:
- #llop.debug_print(lltype.Void, ">>>> Entering",
- # rffi.cast(lltype.Signed, func))
res = func()
- #llop.debug_print(lltype.Void, "<<<< Back")
finally:
if not self.translate_support_code:
LLInterpreter.current_interpreter = prev_interpreter
Modified: pypy/trunk/pypy/jit/metainterp/executor.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/executor.py (original)
+++ pypy/trunk/pypy/jit/metainterp/executor.py Fri Oct 1 11:41:29 2010
@@ -205,8 +205,8 @@
def do_copystrcontent(cpu, _, srcbox, dstbox,
srcstartbox, dststartbox, lengthbox):
- src = srcbox.getptr(lltype.Ptr(rstr.STR))
- dst = dstbox.getptr(lltype.Ptr(rstr.STR))
+ src = srcbox.getref(lltype.Ptr(rstr.STR))
+ dst = dstbox.getref(lltype.Ptr(rstr.STR))
srcstart = srcstartbox.getint()
dststart = dststartbox.getint()
length = lengthbox.getint()
@@ -214,8 +214,8 @@
def do_copyunicodecontent(cpu, _, srcbox, dstbox,
srcstartbox, dststartbox, lengthbox):
- src = srcbox.getptr(lltype.Ptr(rstr.UNICODE))
- dst = dstbox.getptr(lltype.Ptr(rstr.UNICODE))
+ src = srcbox.getref(lltype.Ptr(rstr.UNICODE))
+ dst = dstbox.getref(lltype.Ptr(rstr.UNICODE))
srcstart = srcstartbox.getint()
dststart = dststartbox.getint()
length = lengthbox.getint()
@@ -428,6 +428,10 @@
if arity == 3:
func = get_execute_funclist(3, False)[opnum]
return func(cpu, metainterp, argboxes[0], argboxes[1], argboxes[2])
+ if arity == 5: # copystrcontent, copyunicodecontent
+ func = get_execute_funclist(5, False)[opnum]
+ return func(cpu, metainterp, argboxes[0], argboxes[1],
+ argboxes[2], argboxes[3], argboxes[4])
raise NotImplementedError
Modified: pypy/trunk/pypy/jit/metainterp/optimizeopt/__init__.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/optimizeopt/__init__.py (original)
+++ pypy/trunk/pypy/jit/metainterp/optimizeopt/__init__.py Fri Oct 1 11:41:29 2010
@@ -14,7 +14,7 @@
optimizations = [OptIntBounds(),
OptRewrite(),
OptVirtualize(),
-# OptString(),
+ OptString(),
OptHeap(),
]
optimizer = Optimizer(metainterp_sd, loop, optimizations, virtuals)
Modified: pypy/trunk/pypy/jit/metainterp/warmspot.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/warmspot.py (original)
+++ pypy/trunk/pypy/jit/metainterp/warmspot.py Fri Oct 1 11:41:29 2010
@@ -67,9 +67,16 @@
def jittify_and_run(interp, graph, args, repeat=1,
backendopt=False, trace_limit=sys.maxint,
debug_level=DEBUG_STEPS, inline=False, **kwds):
+ from pypy.config.config import ConfigError
translator = interp.typer.annotator.translator
- translator.config.translation.gc = "boehm"
- translator.config.translation.list_comprehension_operations = True
+ try:
+ translator.config.translation.gc = "boehm"
+ except ConfigError:
+ pass
+ try:
+ translator.config.translation.list_comprehension_operations = True
+ except ConfigError:
+ pass
warmrunnerdesc = WarmRunnerDesc(translator, backendopt=backendopt, **kwds)
for jd in warmrunnerdesc.jitdrivers_sd:
jd.warmstate.set_param_threshold(3) # for tests
Modified: pypy/trunk/pypy/translator/driver.py
==============================================================================
--- pypy/trunk/pypy/translator/driver.py (original)
+++ pypy/trunk/pypy/translator/driver.py Fri Oct 1 11:41:29 2010
@@ -426,6 +426,22 @@
[OOTYPE],
"JIT compiler generation")
+ def task_jittest_lltype(self):
+ """ Run with the JIT on top of the llgraph backend
+ """
+ # parent process loop: spawn a child, wait for the child to finish,
+ # print a message, and restart
+ from pypy.translator.goal import unixcheckpoint
+ unixcheckpoint.restartable_point(auto='run')
+ # load the module pypy/jit/tl/jittest.py, which you can hack at
+ # and restart without needing to restart the whole translation process
+ from pypy.jit.tl import jittest
+ jittest.jittest(self)
+ #
+ task_jittest_lltype = taskdef(task_jittest_lltype,
+ [RTYPE],
+ "test of the JIT on the llgraph backend")
+
def task_backendopt_lltype(self):
""" Run all backend optimizations - lltype version
"""
@@ -433,7 +449,8 @@
backend_optimizations(self.translator)
#
task_backendopt_lltype = taskdef(task_backendopt_lltype,
- [RTYPE, '??pyjitpl_lltype'],
+ [RTYPE, '??pyjitpl_lltype',
+ '??jittest_lltype'],
"lltype back-end optimisations")
BACKENDOPT = 'backendopt_lltype'
Modified: pypy/trunk/pypy/translator/goal/translate.py
==============================================================================
--- pypy/trunk/pypy/translator/goal/translate.py (original)
+++ pypy/trunk/pypy/translator/goal/translate.py Fri Oct 1 11:41:29 2010
@@ -27,6 +27,7 @@
("annotate", "do type inference", "-a --annotate", ""),
("rtype", "do rtyping", "-t --rtype", ""),
("pyjitpl", "JIT generation step", "--pyjitpl", ""),
+ ("jittest", "JIT test with llgraph backend", "--jittest", ""),
("backendopt", "do backend optimizations", "--backendopt", ""),
("source", "create source", "-s --source", ""),
("compile", "compile", "-c --compile", " (default goal)"),
From arigo at codespeak.net Fri Oct 1 11:41:44 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 1 Oct 2010 11:41:44 +0200 (CEST)
Subject: [pypy-svn] r77519 - pypy/branch/jit-str-fix
Message-ID: <20101001094144.21CF8282BDB@codespeak.net>
Author: arigo
Date: Fri Oct 1 11:41:42 2010
New Revision: 77519
Removed:
pypy/branch/jit-str-fix/
Log:
Remove merged branch.
From afa at codespeak.net Fri Oct 1 11:54:07 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 1 Oct 2010 11:54:07 +0200 (CEST)
Subject: [pypy-svn] r77520 - in pypy/branch/fast-forward/pypy/module/rctime:
. test
Message-ID: <20101001095407.1BDF7282BDB@codespeak.net>
Author: afa
Date: Fri Oct 1 11:54:05 2010
New Revision: 77520
Modified:
pypy/branch/fast-forward/pypy/module/rctime/app_time.py
pypy/branch/fast-forward/pypy/module/rctime/interp_time.py
pypy/branch/fast-forward/pypy/module/rctime/test/test_rctime.py
Log:
Fixes in time module:
- unsupported formats in strftime must not pass silently
- the python version of strptime is now called _strptime._strptime().
Modified: pypy/branch/fast-forward/pypy/module/rctime/app_time.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/rctime/app_time.py (original)
+++ pypy/branch/fast-forward/pypy/module/rctime/app_time.py Fri Oct 1 11:54:05 2010
@@ -5,6 +5,7 @@
class struct_time:
__metaclass__ = structseqtype
__module__ = 'time'
+ name = 'time.struct_time'
tm_year = structseqfield(0)
tm_mon = structseqfield(1)
@@ -24,7 +25,7 @@
(same as strftime())."""
import _strptime # from the CPython standard library
- return _strptime.strptime(string, format)
+ return _strptime._strptime(string, format)
__doc__ = """This module provides various functions to manipulate time values.
Modified: pypy/branch/fast-forward/pypy/module/rctime/interp_time.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/rctime/interp_time.py (original)
+++ pypy/branch/fast-forward/pypy/module/rctime/interp_time.py Fri Oct 1 11:54:05 2010
@@ -450,6 +450,21 @@
raise OperationError(space.w_ValueError,
space.wrap("daylight savings flag out of range"))
+ if _WIN:
+ # check that the format string contains only valid directives
+ length = len(format)
+ i = 0
+ while i < length:
+ if format[i] == '%':
+ i += 1
+ if i < length and format[i] == '#':
+ # not documented by python
+ i += 1
+ if i >= length or format[i] not in "aAbBcdfHIjmMpSUwWxXyYzZ%":
+ raise OperationError(space.w_ValueError,
+ space.wrap("invalid format string"))
+ i += 1
+
i = 1024
while True:
outbuf = lltype.malloc(rffi.CCHARP.TO, i, flavor='raw')
Modified: pypy/branch/fast-forward/pypy/module/rctime/test/test_rctime.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/rctime/test/test_rctime.py (original)
+++ pypy/branch/fast-forward/pypy/module/rctime/test/test_rctime.py Fri Oct 1 11:54:05 2010
@@ -135,7 +135,7 @@
raises(TypeError, rctime.struct_time, (1, 2, 3))
tup = (1, 2, 3, 4, 5, 6, 7, 8, 9)
st_time = rctime.struct_time(tup)
- assert str(st_time) == str(tup)
+ assert str(st_time).startswith('time.struct_time(tm_year=1, ')
assert len(st_time) == len(tup)
def test_tzset(self):
@@ -224,8 +224,11 @@
import time as rctime
tt = rctime.gmtime()
- result = rctime.strftime('%D', tt)
- if result != '': # else format not supported and we got ''
+ try:
+ result = rctime.strftime('%D', tt)
+ except ValueError:
+ pass
+ else:
assert result == rctime.strftime('%m/%d/%y', tt)
def test_strftime_bounds_checking(self):
From antocuni at codespeak.net Fri Oct 1 12:05:18 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri, 1 Oct 2010 12:05:18 +0200 (CEST)
Subject: [pypy-svn] r77521 - in pypy/branch/jitffi/pypy: config
jit/backend/test module/_rawffi module/_rawffi/test rlib
rlib/test translator/c/test
Message-ID: <20101001100518.45EA2282BDB@codespeak.net>
Author: antocuni
Date: Fri Oct 1 12:05:16 2010
New Revision: 77521
Added:
pypy/branch/jitffi/pypy/rlib/test/test_clibffi.py
- copied, changed from r77301, pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
Removed:
pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
Modified:
pypy/branch/jitffi/pypy/config/pypyoption.py
pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py
pypy/branch/jitffi/pypy/module/_rawffi/__init__.py
pypy/branch/jitffi/pypy/module/_rawffi/callback.py
pypy/branch/jitffi/pypy/module/_rawffi/interp_rawffi.py
pypy/branch/jitffi/pypy/module/_rawffi/structure.py
pypy/branch/jitffi/pypy/module/_rawffi/test/test__rawffi.py
pypy/branch/jitffi/pypy/rlib/libffi.py
pypy/branch/jitffi/pypy/rlib/test/test_rdynload.py
pypy/branch/jitffi/pypy/translator/c/test/test_newgc.py
Log:
remove the "import *" from clibffi at the top of libffi.py. Fix (hopefully
:-)) all the places that are using the old interface to import from clibffi
instead of libffi. Move clibffi specific tests into their own file.
Modified: pypy/branch/jitffi/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/jitffi/pypy/config/pypyoption.py (original)
+++ pypy/branch/jitffi/pypy/config/pypyoption.py Fri Oct 1 12:05:16 2010
@@ -73,9 +73,9 @@
}
module_import_dependencies = {
- # no _rawffi if importing pypy.rlib.libffi raises ImportError
+ # no _rawffi if importing pypy.rlib.clibffi raises ImportError
# or CompilationError
- "_rawffi" : ["pypy.rlib.libffi"],
+ "_rawffi" : ["pypy.rlib.clibffi"],
"zlib" : ["pypy.rlib.rzlib"],
"bz2" : ["pypy.module.bz2.interp_bz2"],
Modified: pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py Fri Oct 1 12:05:16 2010
@@ -517,7 +517,8 @@
assert res.value == func_ints(*args)
def test_call_to_c_function(self):
- from pypy.rlib.libffi import CDLL, ffi_type_uchar, ffi_type_sint
+ # XXX: fix this to use libffi instead of clibffi
+ from pypy.rlib.clibffi import CDLL, ffi_type_uchar, ffi_type_sint
libc = CDLL('libc.so.6')
c_tolower = libc.getpointer('tolower', [ffi_type_uchar], ffi_type_sint)
c_tolower.push_arg('A')
Modified: pypy/branch/jitffi/pypy/module/_rawffi/__init__.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/_rawffi/__init__.py (original)
+++ pypy/branch/jitffi/pypy/module/_rawffi/__init__.py Fri Oct 1 12:05:16 2010
@@ -39,11 +39,11 @@
if hasattr(interp_rawffi, 'check_HRESULT'):
Module.interpleveldefs['check_HRESULT'] = 'interp_rawffi.check_HRESULT'
- from pypy.rlib import libffi
+ from pypy.rlib import clibffi
for name in ['FUNCFLAG_STDCALL', 'FUNCFLAG_CDECL', 'FUNCFLAG_PYTHONAPI',
]:
- if hasattr(libffi, name):
- Module.interpleveldefs[name] = "space.wrap(%r)" % getattr(libffi, name)
+ if hasattr(clibffi, name):
+ Module.interpleveldefs[name] = "space.wrap(%r)" % getattr(clibffi, name)
super(Module, cls).buildloaders()
buildloaders = classmethod(buildloaders)
Modified: pypy/branch/jitffi/pypy/module/_rawffi/callback.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/_rawffi/callback.py (original)
+++ pypy/branch/jitffi/pypy/module/_rawffi/callback.py Fri Oct 1 12:05:16 2010
@@ -8,8 +8,8 @@
from pypy.module._rawffi.array import get_elem, push_elem
from pypy.module._rawffi.interp_rawffi import W_DataInstance, letter2tp, \
wrap_value, unwrap_value, unwrap_truncate_int
-from pypy.rlib.libffi import USERDATA_P, CallbackFuncPtr, FUNCFLAG_CDECL
-from pypy.rlib.libffi import ffi_type_void
+from pypy.rlib.clibffi import USERDATA_P, CallbackFuncPtr, FUNCFLAG_CDECL
+from pypy.rlib.clibffi import ffi_type_void
from pypy.module._rawffi.tracker import tracker
from pypy.interpreter.error import OperationError
from pypy.interpreter import gateway
Modified: pypy/branch/jitffi/pypy/module/_rawffi/interp_rawffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/_rawffi/interp_rawffi.py (original)
+++ pypy/branch/jitffi/pypy/module/_rawffi/interp_rawffi.py Fri Oct 1 12:05:16 2010
@@ -5,7 +5,7 @@
from pypy.interpreter.gateway import interp2app, NoneNotWrapped
from pypy.interpreter.typedef import TypeDef, GetSetProperty
-from pypy.rlib.libffi import *
+from pypy.rlib.clibffi import *
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.rlib.unroll import unrolling_iterable
Modified: pypy/branch/jitffi/pypy/module/_rawffi/structure.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/_rawffi/structure.py (original)
+++ pypy/branch/jitffi/pypy/module/_rawffi/structure.py Fri Oct 1 12:05:16 2010
@@ -15,7 +15,7 @@
from pypy.module._rawffi.interp_rawffi import wrap_value, unwrap_value
from pypy.module._rawffi.interp_rawffi import unpack_shape_with_length
from pypy.module._rawffi.interp_rawffi import size_alignment
-from pypy.rlib import libffi
+from pypy.rlib import clibffi
from pypy.rlib.rarithmetic import intmask, r_uint
def unpack_fields(space, w_fields):
@@ -104,14 +104,14 @@
descr_fieldoffset.unwrap_spec = ['self', ObjSpace, str]
# get the corresponding ffi_type
- ffi_struct = lltype.nullptr(libffi.FFI_STRUCT_P.TO)
+ ffi_struct = lltype.nullptr(clibffi.FFI_STRUCT_P.TO)
def get_basic_ffi_type(self):
if not self.ffi_struct:
# Repeated fields are delicate. Consider for example
# struct { int a[5]; }
# or struct { struct {int x;} a[5]; }
- # Seeing no corresponding doc in libffi, let's just repeat
+ # Seeing no corresponding doc in clibffi, let's just repeat
# the field 5 times...
fieldtypes = []
for name, tp in self.fields:
@@ -122,7 +122,7 @@
while count + basic_size <= total_size:
fieldtypes.append(basic_ffi_type)
count += basic_size
- self.ffi_struct = libffi.make_struct_ffitype_e(self.size,
+ self.ffi_struct = clibffi.make_struct_ffitype_e(self.size,
self.alignment,
fieldtypes)
return self.ffi_struct.ffistruct
Modified: pypy/branch/jitffi/pypy/module/_rawffi/test/test__rawffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/_rawffi/test/test__rawffi.py (original)
+++ pypy/branch/jitffi/pypy/module/_rawffi/test/test__rawffi.py Fri Oct 1 12:05:16 2010
@@ -191,7 +191,7 @@
prepare_c_example = staticmethod(prepare_c_example)
def setup_class(cls):
- from pypy.rlib.libffi import get_libc_name
+ from pypy.rlib.clibffi import get_libc_name
space = gettestobjspace(usemodules=('_rawffi', 'struct'))
cls.space = space
cls.w_lib_name = space.wrap(cls.prepare_c_example())
Modified: pypy/branch/jitffi/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/libffi.py Fri Oct 1 12:05:16 2010
@@ -1,7 +1,16 @@
-from pypy.rlib.clibffi import *
+from pypy.rlib import clibffi
+from pypy.rlib.clibffi import get_libc_name
from pypy.rlib.objectmodel import specialize
from pypy.rlib import jit
+def import_types():
+ g = globals()
+ for key, value in clibffi.__dict__.iteritems():
+ if key.startswith('ffi_type_'):
+ g[key] = value
+import_types()
+del import_types
+
class AbstractArg(object):
next = None
@@ -46,8 +55,8 @@
self.funcptr.push_arg(value)
# XXX this is bad, fix it somehow in the future, but specialize:argtype
# doesn't work correctly with mixing non-negative and normal integers
- _push_arg._annenforceargs_ = [None, int]
- #push_arg._annspecialcase_ = 'specialize:argtype(1)'
+ #_push_arg._annenforceargs_ = [None, int]
+ _push_arg._annspecialcase_ = 'specialize:argtype(1)'
_push_arg.oopspec = 'libffi_push_arg(self, value)'
def _do_call(self, funcsym, RESULT):
Copied: pypy/branch/jitffi/pypy/rlib/test/test_clibffi.py (from r77301, pypy/branch/jitffi/pypy/rlib/test/test_libffi.py)
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_clibffi.py Fri Oct 1 12:05:16 2010
@@ -3,7 +3,7 @@
"""
from pypy.translator.c.test.test_genc import compile
-from pypy.rlib.libffi import *
+from pypy.rlib.clibffi import *
from pypy.rlib.objectmodel import keepalive_until_here
from pypy.rpython.lltypesystem.ll2ctypes import ALLOCATED
from pypy.rpython.lltypesystem import rffi, lltype
@@ -36,7 +36,6 @@
class TestCLibffi(BaseFfiTest):
def setup_method(self, meth):
- py.test.skip("broken during the refactoring, FIXME")
ALLOCATED.clear()
def test_library_open(self):
@@ -423,19 +422,3 @@
print hex(handle)
assert handle != 0
assert handle % 0x1000 == 0
-
-
-class TestLibffi(BaseFfiTest):
- """
- Test the new JIT-friendly interface to libffi
- """
-
- def test_call_argchain(self):
- libm = self.get_libm()
- pow_ptr = libm.getpointer('pow', [ffi_type_double, ffi_type_double],
- ffi_type_double)
- pow = Func(pow_ptr)
- argchain = FloatArg(2.0)
- argchain.next = FloatArg(3.0)
- res = pow.call(argchain, rffi.DOUBLE)
- assert res == 8.0
Modified: pypy/branch/jitffi/pypy/rlib/test/test_rdynload.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_rdynload.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_rdynload.py Fri Oct 1 12:05:16 2010
@@ -1,5 +1,5 @@
from pypy.rlib.rdynload import *
-from pypy.rlib.libffi import get_libc_name
+from pypy.rlib.clibffi import get_libc_name
from pypy.rpython.lltypesystem import rffi, lltype
import py
Modified: pypy/branch/jitffi/pypy/translator/c/test/test_newgc.py
==============================================================================
--- pypy/branch/jitffi/pypy/translator/c/test/test_newgc.py (original)
+++ pypy/branch/jitffi/pypy/translator/c/test/test_newgc.py Fri Oct 1 12:05:16 2010
@@ -624,13 +624,13 @@
os.unlink(self.filename)
def define_callback_with_collect(cls):
- from pypy.rlib.libffi import ffi_type_pointer, cast_type_to_ffitype,\
+ from pypy.rlib.clibffi import ffi_type_pointer, cast_type_to_ffitype,\
CDLL, ffi_type_void, CallbackFuncPtr, ffi_type_sint
from pypy.rpython.lltypesystem import rffi, ll2ctypes
import gc
ffi_size_t = cast_type_to_ffitype(rffi.SIZE_T)
- from pypy.rlib.libffi import get_libc_name
+ from pypy.rlib.clibffi import get_libc_name
def callback(ll_args, ll_res, stuff):
gc.collect()
From afa at codespeak.net Fri Oct 1 14:31:43 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 1 Oct 2010 14:31:43 +0200 (CEST)
Subject: [pypy-svn] r77522 - pypy/branch/fast-forward/lib_pypy/pypy_test
Message-ID: <20101001123143.24DD7282BDB@codespeak.net>
Author: afa
Date: Fri Oct 1 14:31:41 2010
New Revision: 77522
Modified:
pypy/branch/fast-forward/lib_pypy/pypy_test/test_collections.py
pypy/branch/fast-forward/lib_pypy/pypy_test/test_defaultdict.py
pypy/branch/fast-forward/lib_pypy/pypy_test/test_deque_extra.py
Log:
Pypy's own implementation of collections was moved to _collections.py
Modified: pypy/branch/fast-forward/lib_pypy/pypy_test/test_collections.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/pypy_test/test_collections.py (original)
+++ pypy/branch/fast-forward/lib_pypy/pypy_test/test_collections.py Fri Oct 1 14:31:41 2010
@@ -1,5 +1,5 @@
from __future__ import absolute_import
-from .. import collections
+from .. import _collections as collections
import py
def test_deque_remove_empty():
Modified: pypy/branch/fast-forward/lib_pypy/pypy_test/test_defaultdict.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/pypy_test/test_defaultdict.py (original)
+++ pypy/branch/fast-forward/lib_pypy/pypy_test/test_defaultdict.py Fri Oct 1 14:31:41 2010
@@ -10,7 +10,7 @@
import copy
-from ..collections import defaultdict
+from .._collections import defaultdict
def foobar():
return list
Modified: pypy/branch/fast-forward/lib_pypy/pypy_test/test_deque_extra.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/pypy_test/test_deque_extra.py (original)
+++ pypy/branch/fast-forward/lib_pypy/pypy_test/test_deque_extra.py Fri Oct 1 14:31:41 2010
@@ -7,7 +7,7 @@
class Test_deque:
def setup_method(self,method):
- from ..collections import deque
+ from .._collections import deque
self.deque = deque
self.d = deque(range(n))
From afa at codespeak.net Fri Oct 1 15:03:10 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 1 Oct 2010 15:03:10 +0200 (CEST)
Subject: [pypy-svn] r77523 - in pypy/branch/fast-forward:
lib-python/modified-2.7.0/test lib_pypy/pypy_test
Message-ID: <20101001130310.8029A282BDC@codespeak.net>
Author: afa
Date: Fri Oct 1 15:03:09 2010
New Revision: 77523
Added:
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_functools.py
- copied, changed from r77516, pypy/branch/fast-forward/lib-python/2.7.0/test/test_functools.py
Removed:
pypy/branch/fast-forward/lib_pypy/pypy_test/test_functools.py
Log:
Don't duplicate test_functools.py in pypy_test,
use a slightly modified version of the official test suite.
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_functools.py (from r77516, pypy/branch/fast-forward/lib-python/2.7.0/test/test_functools.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/test/test_functools.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_functools.py Fri Oct 1 15:03:09 2010
@@ -136,6 +136,7 @@
p = proxy(f)
self.assertEqual(f.func, p.func)
f = None
+ test_support.gc_collect()
self.assertRaises(ReferenceError, getattr, p, 'func')
def test_with_bound_and_unbound_methods(self):
@@ -172,7 +173,7 @@
updated=functools.WRAPPER_UPDATES):
# Check attributes were assigned
for name in assigned:
- self.assertTrue(getattr(wrapper, name) is getattr(wrapped, name))
+ self.assertTrue(getattr(wrapper, name) == getattr(wrapped, name), name)
# Check attributes were updated
for name in updated:
wrapper_attr = getattr(wrapper, name)
From hakanardo at codespeak.net Fri Oct 1 15:46:27 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Fri, 1 Oct 2010 15:46:27 +0200 (CEST)
Subject: [pypy-svn] r77524 - pypy/extradoc/sprintinfo/ddorf2010
Message-ID: <20101001134627.DA6C6282BDB@codespeak.net>
Author: hakanardo
Date: Fri Oct 1 15:46:25 2010
New Revision: 77524
Modified:
pypy/extradoc/sprintinfo/ddorf2010/people.txt
Log:
registration
Modified: pypy/extradoc/sprintinfo/ddorf2010/people.txt
==============================================================================
--- pypy/extradoc/sprintinfo/ddorf2010/people.txt (original)
+++ pypy/extradoc/sprintinfo/ddorf2010/people.txt Fri Oct 1 15:46:25 2010
@@ -11,6 +11,7 @@
Carl Friedrich Bolz always there private
Antonio Cuni leaves the 30 private
Armin Rigo always there private
+Hakan Ardo 28/10 - 31/10 staying with antocuni
==================== ============== =====================
From arigo at codespeak.net Fri Oct 1 15:46:29 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 1 Oct 2010 15:46:29 +0200 (CEST)
Subject: [pypy-svn] r77525 -
pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests
Message-ID: <20101001134629.9676D282BDB@codespeak.net>
Author: arigo
Date: Fri Oct 1 15:46:28 2010
New Revision: 77525
Modified:
pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py
Log:
Add a failing test.
Modified: pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py
==============================================================================
--- pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py (original)
+++ pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py Fri Oct 1 15:46:28 2010
@@ -29,6 +29,7 @@
class RECT(Structure):
_fields_ = [("left", c_int), ("top", c_int),
("right", c_int), ("bottom", c_int)]
+
class TestFunctions(BaseCTypesTestChecker):
def test_mro(self):
@@ -392,6 +393,18 @@
result = f("abcd", ord("b"))
assert result == "bcd"
+ def test_caching_bug_1(self):
+ # the same test as test_call_some_args, with two extra lines
+ # in the middle that trigger caching in f._ptr, which then
+ # makes the last two lines fail
+ f = dll.my_strchr
+ f.argtypes = [c_char_p, c_int]
+ f.restype = c_char_p
+ result = f("abcd", ord("b"))
+ assert result == "bcd"
+ result = f("abcd", ord("b"), 42)
+ assert result == "bcd"
+
def test_sf1651235(self):
py.test.skip("we are less strict in checking callback parameters")
# see http://www.python.org/sf/1651235
From arigo at codespeak.net Fri Oct 1 16:18:53 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 1 Oct 2010 16:18:53 +0200 (CEST)
Subject: [pypy-svn] r77526 - pypy/trunk/lib_pypy/_ctypes
Message-ID: <20101001141853.0765B282BDB@codespeak.net>
Author: arigo
Date: Fri Oct 1 16:18:51 2010
New Revision: 77526
Modified:
pypy/trunk/lib_pypy/_ctypes/function.py
Log:
Minimal fix.
Modified: pypy/trunk/lib_pypy/_ctypes/function.py
==============================================================================
--- pypy/trunk/lib_pypy/_ctypes/function.py (original)
+++ pypy/trunk/lib_pypy/_ctypes/function.py Fri Oct 1 16:18:51 2010
@@ -171,7 +171,7 @@
return self._build_result(restype, resbuffer, argtypes, argsandobjs)
def _getfuncptr(self, argtypes, restype, thisarg=None):
- if self._ptr is not None:
+ if self._ptr is not None and argtypes is self._argtypes_:
return self._ptr
if restype is None or not isinstance(restype, _CDataMeta):
import ctypes
From arigo at codespeak.net Fri Oct 1 16:24:15 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 1 Oct 2010 16:24:15 +0200 (CEST)
Subject: [pypy-svn] r77527 - pypy/branch/larger-writebarrier
Message-ID: <20101001142415.40AC7282BDB@codespeak.net>
Author: arigo
Date: Fri Oct 1 16:24:13 2010
New Revision: 77527
Added:
pypy/branch/larger-writebarrier/
- copied from r77526, pypy/trunk/
Log:
It seems that "smaller-writebarrier" may have had a performance
impact on a few benchmarks. In this branch I am going to try
a mixed solution: using the same two-args write barrier for
regular setfields, and the smaller (2 args instead of 3) version
for setarrayitem.
From afa at codespeak.net Fri Oct 1 16:43:33 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 1 Oct 2010 16:43:33 +0200 (CEST)
Subject: [pypy-svn] r77529 - in pypy/trunk/pypy/module/imp: . test
Message-ID: <20101001144333.2EC47282BDC@codespeak.net>
Author: afa
Date: Fri Oct 1 16:43:31 2010
New Revision: 77529
Modified:
pypy/trunk/pypy/module/imp/importing.py
pypy/trunk/pypy/module/imp/test/test_import.py
Log:
Avoid a crash when "from .. import x" is used at toplevel.
Modified: pypy/trunk/pypy/module/imp/importing.py
==============================================================================
--- pypy/trunk/pypy/module/imp/importing.py (original)
+++ pypy/trunk/pypy/module/imp/importing.py Fri Oct 1 16:43:31 2010
@@ -122,7 +122,7 @@
n = len(ctxt_name_prefix_parts)-level+1
assert n>=0
ctxt_name_prefix_parts = ctxt_name_prefix_parts[:n]
- if ctxt_w_path is None: # plain module
+ if ctxt_name_prefix_parts and ctxt_w_path is None: # plain module
ctxt_name_prefix_parts.pop()
if ctxt_name_prefix_parts:
rel_modulename = '.'.join(ctxt_name_prefix_parts)
Modified: pypy/trunk/pypy/module/imp/test/test_import.py
==============================================================================
--- pypy/trunk/pypy/module/imp/test/test_import.py (original)
+++ pypy/trunk/pypy/module/imp/test/test_import.py Fri Oct 1 16:43:31 2010
@@ -360,6 +360,12 @@
""".rstrip()
raises(ValueError, imp)
+ def test_future_relative_import_error_when_in_non_package2(self):
+ exec """def imp():
+ from .. import inpackage
+ """.rstrip()
+ raises(ValueError, imp)
+
def test_relative_import_with___name__(self):
import sys
mydict = {'__name__': 'sys.foo'}
From arigo at codespeak.net Fri Oct 1 16:55:32 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 1 Oct 2010 16:55:32 +0200 (CEST)
Subject: [pypy-svn] r77530 - in pypy/branch/larger-writebarrier/pypy:
jit/backend/llgraph jit/backend/llsupport
jit/backend/llsupport/test jit/backend/test jit/backend/x86
jit/metainterp rpython/memory rpython/memory/gc
rpython/memory/gc/test rpython/memory/gctransform
Message-ID: <20101001145532.DC842282BDC@codespeak.net>
Author: arigo
Date: Fri Oct 1 16:55:30 2010
New Revision: 77530
Modified:
pypy/branch/larger-writebarrier/pypy/jit/backend/llgraph/llimpl.py
pypy/branch/larger-writebarrier/pypy/jit/backend/llsupport/gc.py
pypy/branch/larger-writebarrier/pypy/jit/backend/llsupport/test/test_gc.py
pypy/branch/larger-writebarrier/pypy/jit/backend/test/runner_test.py
pypy/branch/larger-writebarrier/pypy/jit/backend/x86/assembler.py
pypy/branch/larger-writebarrier/pypy/jit/backend/x86/regalloc.py
pypy/branch/larger-writebarrier/pypy/jit/metainterp/resoperation.py
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/base.py
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/generation.py
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/test/test_direct.py
pypy/branch/larger-writebarrier/pypy/rpython/memory/gctransform/framework.py
pypy/branch/larger-writebarrier/pypy/rpython/memory/gcwrapper.py
Log:
In-progress: partial revert of the merge of branch/smaller-writebarrier
(revision 77403).
Modified: pypy/branch/larger-writebarrier/pypy/jit/backend/llgraph/llimpl.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/jit/backend/llgraph/llimpl.py (original)
+++ pypy/branch/larger-writebarrier/pypy/jit/backend/llgraph/llimpl.py Fri Oct 1 16:55:30 2010
@@ -129,7 +129,7 @@
'arraylen_gc' : (('ref',), 'int'),
'call' : (('ref', 'varargs'), 'intorptr'),
'call_assembler' : (('varargs',), 'intorptr'),
- 'cond_call_gc_wb' : (('ptr',), None),
+ 'cond_call_gc_wb' : (('ptr', 'ptr'), None),
'oosend' : (('varargs',), 'intorptr'),
'oosend_pure' : (('varargs',), 'intorptr'),
'guard_true' : (('bool',), None),
@@ -810,7 +810,7 @@
FLOAT: 0.0}
return d[calldescr.typeinfo]
- def op_cond_call_gc_wb(self, descr, a):
+ def op_cond_call_gc_wb(self, descr, a, b):
py.test.skip("cond_call_gc_wb not supported")
def op_oosend(self, descr, obj, *args):
Modified: pypy/branch/larger-writebarrier/pypy/jit/backend/llsupport/gc.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/jit/backend/llsupport/gc.py (original)
+++ pypy/branch/larger-writebarrier/pypy/jit/backend/llsupport/gc.py Fri Oct 1 16:55:30 2010
@@ -404,7 +404,7 @@
self.GC_MALLOC_BASIC = lltype.Ptr(lltype.FuncType(
[lltype.Signed, lltype.Signed], llmemory.GCREF))
self.WB_FUNCPTR = lltype.Ptr(lltype.FuncType(
- [llmemory.Address], lltype.Void))
+ [llmemory.Address, llmemory.Address], lltype.Void))
self.write_barrier_descr = WriteBarrierDescr(self)
#
def malloc_array(itemsize, tid, num_elem):
@@ -550,7 +550,8 @@
# the GC, and call it immediately
llop1 = self.llop1
funcptr = llop1.get_write_barrier_failing_case(self.WB_FUNCPTR)
- funcptr(llmemory.cast_ptr_to_adr(gcref_struct))
+ funcptr(llmemory.cast_ptr_to_adr(gcref_struct),
+ llmemory.cast_ptr_to_adr(gcref_newptr))
def rewrite_assembler(self, cpu, operations):
# Perform two kinds of rewrites in parallel:
@@ -589,7 +590,7 @@
v = op.getarg(1)
if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and
bool(v.value)): # store a non-NULL
- self._gen_write_barrier(newops, op.getarg(0))
+ self._gen_write_barrier(newops, op.getarg(0), v)
op = op.copy_and_change(rop.SETFIELD_RAW)
# ---------- write barrier for SETARRAYITEM_GC ----------
if op.getopnum() == rop.SETARRAYITEM_GC:
@@ -598,15 +599,15 @@
bool(v.value)): # store a non-NULL
# XXX detect when we should produce a
# write_barrier_from_array
- self._gen_write_barrier(newops, op.getarg(0))
+ self._gen_write_barrier(newops, op.getarg(0), v)
op = op.copy_and_change(rop.SETARRAYITEM_RAW)
# ----------
newops.append(op)
del operations[:]
operations.extend(newops)
- def _gen_write_barrier(self, newops, v_base):
- args = [v_base]
+ def _gen_write_barrier(self, newops, v_base, v_value):
+ args = [v_base, v_value]
newops.append(ResOperation(rop.COND_CALL_GC_WB, args, None,
descr=self.write_barrier_descr))
Modified: pypy/branch/larger-writebarrier/pypy/jit/backend/llsupport/test/test_gc.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/jit/backend/llsupport/test/test_gc.py (original)
+++ pypy/branch/larger-writebarrier/pypy/jit/backend/llsupport/test/test_gc.py Fri Oct 1 16:55:30 2010
@@ -141,8 +141,8 @@
repr(offset_to_length), p))
return p
- def _write_barrier_failing_case(self, adr_struct):
- self.record.append(('barrier', adr_struct))
+ def _write_barrier_failing_case(self, adr_struct, adr_newptr):
+ self.record.append(('barrier', adr_struct, adr_newptr))
def get_write_barrier_failing_case(self, FPTRTYPE):
return llhelper(FPTRTYPE, self._write_barrier_failing_case)
@@ -239,6 +239,7 @@
s_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
r_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, r)
s_adr = llmemory.cast_ptr_to_adr(s)
+ r_adr = llmemory.cast_ptr_to_adr(r)
#
s_hdr.tid &= ~gc_ll_descr.GCClass.JIT_WB_IF_FLAG
gc_ll_descr.do_write_barrier(s_gcref, r_gcref)
@@ -246,7 +247,7 @@
#
s_hdr.tid |= gc_ll_descr.GCClass.JIT_WB_IF_FLAG
gc_ll_descr.do_write_barrier(s_gcref, r_gcref)
- assert self.llop1.record == [('barrier', s_adr)]
+ assert self.llop1.record == [('barrier', s_adr, r_adr)]
def test_gen_write_barrier(self):
gc_ll_descr = self.gc_ll_descr
@@ -254,11 +255,13 @@
#
newops = []
v_base = BoxPtr()
- gc_ll_descr._gen_write_barrier(newops, v_base)
+ v_value = BoxPtr()
+ gc_ll_descr._gen_write_barrier(newops, v_base, v_value)
assert llop1.record == []
assert len(newops) == 1
assert newops[0].getopnum() == rop.COND_CALL_GC_WB
assert newops[0].getarg(0) == v_base
+ assert newops[0].getarg(1) == v_value
assert newops[0].result is None
wbdescr = newops[0].getdescr()
assert isinstance(wbdescr.jit_wb_if_flag, int)
@@ -358,6 +361,7 @@
#
assert operations[0].getopnum() == rop.COND_CALL_GC_WB
assert operations[0].getarg(0) == v_base
+ assert operations[0].getarg(1) == v_value
assert operations[0].result is None
#
assert operations[1].getopnum() == rop.SETFIELD_RAW
@@ -381,6 +385,7 @@
#
assert operations[0].getopnum() == rop.COND_CALL_GC_WB
assert operations[0].getarg(0) == v_base
+ assert operations[0].getarg(1) == v_value
assert operations[0].result is None
#
assert operations[1].getopnum() == rop.SETARRAYITEM_RAW
Modified: pypy/branch/larger-writebarrier/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/branch/larger-writebarrier/pypy/jit/backend/test/runner_test.py Fri Oct 1 16:55:30 2010
@@ -1427,12 +1427,12 @@
assert not excvalue
def test_cond_call_gc_wb(self):
- def func_void(a):
- record.append(a)
+ def func_void(a, b):
+ record.append((a, b))
record = []
#
S = lltype.GcStruct('S', ('tid', lltype.Signed))
- FUNC = self.FuncType([lltype.Ptr(S)], lltype.Void)
+ FUNC = self.FuncType([lltype.Ptr(S), lltype.Signed], lltype.Void)
func_ptr = llhelper(lltype.Ptr(FUNC), func_void)
funcbox = self.get_funcbox(self.cpu, func_ptr)
class WriteBarrierDescr(AbstractDescr):
@@ -1453,10 +1453,10 @@
sgcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
del record[:]
self.execute_operation(rop.COND_CALL_GC_WB,
- [BoxPtr(sgcref)],
+ [BoxPtr(sgcref), ConstInt(-2121)],
'void', descr=WriteBarrierDescr())
if cond:
- assert record == [s]
+ assert record == [(s, -2121)]
else:
assert record == []
Modified: pypy/branch/larger-writebarrier/pypy/jit/backend/x86/assembler.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/jit/backend/x86/assembler.py (original)
+++ pypy/branch/larger-writebarrier/pypy/jit/backend/x86/assembler.py Fri Oct 1 16:55:30 2010
@@ -1780,11 +1780,12 @@
self.mc.PUSH_i32(loc.getint())
if IS_X86_64:
- # We clobber this register to pass the arguments, but that's
+ # We clobber these registers to pass the arguments, but that's
# okay, because consider_cond_call_gc_wb makes sure that any
# caller-save registers with values in them are present in arglocs,
# so they are saved on the stack above and restored below
self.mc.MOV_rs(edi.value, 0)
+ self.mc.MOV_rs(esi.value, 8)
# misaligned stack in the call, but it's ok because the write barrier
# is not going to call anything more. Also, this assumes that the
Modified: pypy/branch/larger-writebarrier/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/branch/larger-writebarrier/pypy/jit/backend/x86/regalloc.py Fri Oct 1 16:55:30 2010
@@ -696,9 +696,13 @@
def consider_cond_call_gc_wb(self, op):
assert op.result is None
args = op.getarglist()
+ loc_newvalue = self.rm.make_sure_var_in_reg(op.getarg(1), args)
+ # ^^^ we force loc_newvalue in a reg (unless it's a Const),
+ # because it will be needed anyway by the following setfield_gc.
+ # It avoids loading it twice from the memory.
loc_base = self.rm.make_sure_var_in_reg(op.getarg(0), args,
imm_fine=False)
- arglocs = [loc_base]
+ arglocs = [loc_base, loc_newvalue]
# add eax, ecx and edx as extra "arguments" to ensure they are
# saved and restored. Fish in self.rm to know which of these
# registers really need to be saved (a bit of a hack). Moreover,
Modified: pypy/branch/larger-writebarrier/pypy/jit/metainterp/resoperation.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/jit/metainterp/resoperation.py (original)
+++ pypy/branch/larger-writebarrier/pypy/jit/metainterp/resoperation.py Fri Oct 1 16:55:30 2010
@@ -455,7 +455,7 @@
'UNICODESETITEM/3',
'NEWUNICODE/1',
#'RUNTIMENEW/1', # ootype operation
- 'COND_CALL_GC_WB/1d', # [objptr] (for the write barrier)
+ 'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier)
'DEBUG_MERGE_POINT/1', # debugging only
'VIRTUAL_REF_FINISH/2', # removed before it's passed to the backend
'COPYSTRCONTENT/5', # src, dst, srcstart, dststart, length
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/base.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/base.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/base.py Fri Oct 1 16:55:30 2010
@@ -79,7 +79,7 @@
def set_root_walker(self, root_walker):
self.root_walker = root_walker
- def write_barrier(self, addr_struct):
+ def write_barrier(self, newvalue, addr_struct):
pass
def statistics(self, index):
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/generation.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/generation.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/generation.py Fri Oct 1 16:55:30 2010
@@ -157,6 +157,11 @@
"odd-valued (i.e. tagged) pointer unexpected here")
return self.nursery <= addr < self.nursery_top
+ def appears_to_be_in_nursery(self, addr):
+ # same as is_in_nursery(), but may return True accidentally if
+ # 'addr' is a tagged pointer with just the wrong value.
+ return self.nursery <= addr < self.nursery_top
+
def malloc_fixedsize_clear(self, typeid, size, can_collect,
has_finalizer=False, contains_weakptr=False):
if (has_finalizer or not can_collect or
@@ -326,7 +331,7 @@
addr = pointer.address[0]
newaddr = self.copy(addr)
pointer.address[0] = newaddr
- self.write_into_last_generation_obj(obj)
+ self.write_into_last_generation_obj(obj, newaddr)
# ____________________________________________________________
# Implementation of nursery-only collections
@@ -457,9 +462,9 @@
# "if addr_struct.int0 & JIT_WB_IF_FLAG: remember_young_pointer()")
JIT_WB_IF_FLAG = GCFLAG_NO_YOUNG_PTRS
- def write_barrier(self, addr_struct):
- if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS:
- self.remember_young_pointer(addr_struct)
+ def write_barrier(self, newvalue, addr_struct):
+ if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS:
+ self.remember_young_pointer(addr_struct, newvalue)
def _setup_wb(self):
DEBUG = self.DEBUG
@@ -470,23 +475,33 @@
# For x86, there is also an extra requirement: when the JIT calls
# remember_young_pointer(), it assumes that it will not touch the SSE
# registers, so it does not save and restore them (that's a *hack*!).
- def remember_young_pointer(addr_struct):
+ def remember_young_pointer(addr_struct, addr):
#llop.debug_print(lltype.Void, "\tremember_young_pointer",
# addr_struct, "<-", addr)
if DEBUG:
ll_assert(not self.is_in_nursery(addr_struct),
"nursery object with GCFLAG_NO_YOUNG_PTRS")
- self.old_objects_pointing_to_young.append(addr_struct)
- self.header(addr_struct).tid &= ~GCFLAG_NO_YOUNG_PTRS
- self.write_into_last_generation_obj(addr_struct)
+ #
+ # What is important in this function is that it *must*
+ # clear the flag GCFLAG_NO_YOUNG_PTRS from 'addr_struct'
+ # if 'addr' is in the nursery. It is ok if, accidentally,
+ # it also clears the flag in some more rare cases, like
+ # 'addr' being a tagged pointer whose value happens to be
+ # a large integer that fools is_in_nursery().
+ if self.appears_to_be_in_nursery(addr):
+ self.old_objects_pointing_to_young.append(addr_struct)
+ self.header(addr_struct).tid &= ~GCFLAG_NO_YOUNG_PTRS
+ self.write_into_last_generation_obj(addr_struct, addr)
remember_young_pointer._dont_inline_ = True
self.remember_young_pointer = remember_young_pointer
- def write_into_last_generation_obj(self, addr_struct):
+ def write_into_last_generation_obj(self, addr_struct, addr):
objhdr = self.header(addr_struct)
if objhdr.tid & GCFLAG_NO_HEAP_PTRS:
- objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
- self.last_generation_root_objects.append(addr_struct)
+ if (self.is_valid_gc_object(addr) and
+ not self.is_last_generation(addr)):
+ objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
+ self.last_generation_root_objects.append(addr_struct)
write_into_last_generation_obj._always_inline_ = True
def assume_young_pointers(self, addr_struct):
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py Fri Oct 1 16:55:30 2010
@@ -726,16 +726,16 @@
def JIT_max_size_of_young_obj(cls):
return cls.TRANSLATION_PARAMS['large_object']
- def write_barrier(self, addr_struct):
+ def write_barrier(self, newvalue, addr_struct):
if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS:
- self.remember_young_pointer(addr_struct)
+ self.remember_young_pointer(addr_struct, newvalue)
- def write_barrier_from_array(self, addr_array, index):
+ def write_barrier_from_array(self, newvalue, addr_array, index):
if self.header(addr_array).tid & GCFLAG_NO_YOUNG_PTRS:
if self.card_page_indices > 0: # <- constant-folded
self.remember_young_pointer_from_array(addr_array, index)
else:
- self.remember_young_pointer(addr_array)
+ self.remember_young_pointer(addr_array, newvalue)
def _init_writebarrier_logic(self):
DEBUG = self.DEBUG
@@ -746,7 +746,7 @@
# For x86, there is also an extra requirement: when the JIT calls
# remember_young_pointer(), it assumes that it will not touch the SSE
# registers, so it does not save and restore them (that's a *hack*!).
- def remember_young_pointer(addr_struct):
+ def remember_young_pointer(addr_struct, newvalue):
# 'addr_struct' is the address of the object in which we write.
if DEBUG:
ll_assert(not self.is_in_nursery(addr_struct),
@@ -754,7 +754,9 @@
#
# We assume that what we are writing is a pointer to the nursery
# (and don't care for the fact that this new pointer may not
- # actually point to the nursery, which seems ok). What we need is
+ # actually point to the nursery, which seems ok
+ # XXXXXX XXX wrong !! fix me
+ # ). What we need is
# to remove the flag GCFLAG_NO_YOUNG_PTRS and add the old object
# to the list 'old_objects_pointing_to_young'. We know that
# 'addr_struct' cannot be in the nursery, because nursery objects
@@ -784,12 +786,19 @@
# 'addr_array' is the address of the object in which we write,
# which must have an array part; 'index' is the index of the
# item that is (or contains) the pointer that we write.
+ if DEBUG:
+ ll_assert(not self.is_in_nursery(addr_array),
+ "nursery array with GCFLAG_NO_YOUNG_PTRS")
objhdr = self.header(addr_array)
if objhdr.tid & GCFLAG_HAS_CARDS == 0:
#
- # no cards, use default logic. The 'nocard_logic()' is just
- # 'remember_young_pointer()', but forced to be inlined here.
- nocard_logic(addr_array)
+ # no cards, use default logic. Mostly copied from above.
+ self.old_objects_pointing_to_young.append(addr_array)
+ objhdr = self.header(addr_array)
+ objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
+ if objhdr.tid & GCFLAG_NO_HEAP_PTRS:
+ objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
+ self.prebuilt_root_objects.append(addr_array)
return
#
# 'addr_array' is a raw_malloc'ed array with card markers
@@ -807,17 +816,15 @@
return
#
# We set the flag (even if the newly written address does not
- # actually point to the nursery -- like remember_young_pointer()).
+ # actually point to the nursery, which seems to be ok -- actually
+ # it seems more important that remember_young_pointer_from_array()
+ # does not take 3 arguments).
addr_byte.char[0] = chr(byte | bitmask)
#
if objhdr.tid & GCFLAG_CARDS_SET == 0:
self.old_objects_with_cards_set.append(addr_array)
objhdr.tid |= GCFLAG_CARDS_SET
- nocard_logic = func_with_new_name(self.remember_young_pointer,
- 'remember_young_pointer_nocard')
- del nocard_logic._dont_inline_
- nocard_logic._always_inline_ = True
remember_young_pointer_from_array._dont_inline_ = True
self.remember_young_pointer_from_array = (
remember_young_pointer_from_array)
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/test/test_direct.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/test/test_direct.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/test/test_direct.py Fri Oct 1 16:55:30 2010
@@ -86,17 +86,19 @@
def write(self, p, fieldname, newvalue):
if self.gc.needs_write_barrier:
+ newaddr = llmemory.cast_ptr_to_adr(newvalue)
addr_struct = llmemory.cast_ptr_to_adr(p)
- self.gc.write_barrier(addr_struct)
+ self.gc.write_barrier(newaddr, addr_struct)
setattr(p, fieldname, newvalue)
def writearray(self, p, index, newvalue):
if self.gc.needs_write_barrier:
+ newaddr = llmemory.cast_ptr_to_adr(newvalue)
addr_struct = llmemory.cast_ptr_to_adr(p)
if hasattr(self.gc, 'write_barrier_from_array'):
- self.gc.write_barrier_from_array(addr_struct, index)
+ self.gc.write_barrier_from_array(newvalue, addr_struct, index)
else:
- self.gc.write_barrier(addr_struct)
+ self.gc.write_barrier(newaddr, addr_struct)
p[index] = newvalue
def malloc(self, TYPE, n=None):
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gctransform/framework.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gctransform/framework.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gctransform/framework.py Fri Oct 1 16:55:30 2010
@@ -426,6 +426,7 @@
if GCClass.needs_write_barrier:
self.write_barrier_ptr = getfn(GCClass.write_barrier.im_func,
[s_gc,
+ annmodel.SomeAddress(),
annmodel.SomeAddress()],
annmodel.s_None,
inline=True)
@@ -434,7 +435,8 @@
# func should not be a bound method, but a real function
assert isinstance(func, types.FunctionType)
self.write_barrier_failing_case_ptr = getfn(func,
- [annmodel.SomeAddress()],
+ [annmodel.SomeAddress(),
+ annmodel.SomeAddress()],
annmodel.s_None)
func = getattr(GCClass, 'write_barrier_from_array', None)
if func is not None:
@@ -1034,8 +1036,11 @@
v_index])
else:
self.write_barrier_calls += 1
+ v_newvalue = hop.genop("cast_ptr_to_adr", [v_newvalue],
+ resulttype = llmemory.Address)
hop.genop("direct_call", [self.write_barrier_ptr,
self.c_const_gc,
+ v_newvalue,
v_structaddr])
hop.rename('bare_' + opname)
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gcwrapper.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gcwrapper.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gcwrapper.py Fri Oct 1 16:55:30 2010
@@ -94,6 +94,7 @@
assert (type(index) is int # <- fast path
or lltype.typeOf(index) == lltype.Signed)
self.gc.write_barrier_from_array(
+ llmemory.cast_ptr_to_adr(newvalue),
llmemory.cast_ptr_to_adr(toplevelcontainer),
index)
wb = False
@@ -101,6 +102,7 @@
#
if wb:
self.gc.write_barrier(
+ llmemory.cast_ptr_to_adr(newvalue),
llmemory.cast_ptr_to_adr(toplevelcontainer))
llheap.setinterior(toplevelcontainer, inneraddr, INNERTYPE, newvalue)
From arigo at codespeak.net Fri Oct 1 17:00:25 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 1 Oct 2010 17:00:25 +0200 (CEST)
Subject: [pypy-svn] r77531 -
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc
Message-ID: <20101001150025.6ED33282BDC@codespeak.net>
Author: arigo
Date: Fri Oct 1 17:00:23 2010
New Revision: 77531
Modified:
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py
Log:
Add appears_to_be_in_nursery() in minimark.py too.
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py Fri Oct 1 17:00:23 2010
@@ -636,6 +636,11 @@
"odd-valued (i.e. tagged) pointer unexpected here")
return self.nursery <= addr < self.nursery_top
+ def appears_to_be_in_nursery(self, addr):
+ # same as is_in_nursery(), but may return True accidentally if
+ # 'addr' is a tagged pointer with just the wrong value.
+ return self.nursery <= addr < self.nursery_top
+
def is_forwarded(self, obj):
"""Returns True if the nursery obj is marked as forwarded.
Implemented a bit obscurely by checking an unrelated flag
@@ -752,23 +757,21 @@
ll_assert(not self.is_in_nursery(addr_struct),
"nursery object with GCFLAG_NO_YOUNG_PTRS")
#
- # We assume that what we are writing is a pointer to the nursery
- # (and don't care for the fact that this new pointer may not
- # actually point to the nursery, which seems ok
- # XXXXXX XXX wrong !! fix me
- # ). What we need is
+ # If it seems that what we are writing is a pointer to the nursery
+ # (as checked with appears_to_be_in_nursery()), then we need
# to remove the flag GCFLAG_NO_YOUNG_PTRS and add the old object
# to the list 'old_objects_pointing_to_young'. We know that
# 'addr_struct' cannot be in the nursery, because nursery objects
# never have the flag GCFLAG_NO_YOUNG_PTRS to start with.
- self.old_objects_pointing_to_young.append(addr_struct)
- objhdr = self.header(addr_struct)
- objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
+ if self.appears_to_be_in_nursery(newvalue):
+ self.old_objects_pointing_to_young.append(addr_struct)
+ objhdr = self.header(addr_struct)
+ objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
#
# Second part: if 'addr_struct' is actually a prebuilt GC
# object and it's the first time we see a write to it, we
# add it to the list 'prebuilt_root_objects'. Note that we
- # do it even in the (rare?) case of 'addr' being another
+ # do it even in the (rare?) case of 'addr' being NULL or another
# prebuilt object, to simplify code.
if objhdr.tid & GCFLAG_NO_HEAP_PTRS:
objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
From arigo at codespeak.net Fri Oct 1 17:06:33 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 1 Oct 2010 17:06:33 +0200 (CEST)
Subject: [pypy-svn] r77532 - in
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc: . test
Message-ID: <20101001150633.60755282BDB@codespeak.net>
Author: arigo
Date: Fri Oct 1 17:06:32 2010
New Revision: 77532
Modified:
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/test/test_direct.py
Log:
Typos.
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py Fri Oct 1 17:06:32 2010
@@ -753,6 +753,7 @@
# registers, so it does not save and restore them (that's a *hack*!).
def remember_young_pointer(addr_struct, newvalue):
# 'addr_struct' is the address of the object in which we write.
+ # 'newvalue' is the address that we are going to write in there.
if DEBUG:
ll_assert(not self.is_in_nursery(addr_struct),
"nursery object with GCFLAG_NO_YOUNG_PTRS")
@@ -763,9 +764,9 @@
# to the list 'old_objects_pointing_to_young'. We know that
# 'addr_struct' cannot be in the nursery, because nursery objects
# never have the flag GCFLAG_NO_YOUNG_PTRS to start with.
+ objhdr = self.header(addr_struct)
if self.appears_to_be_in_nursery(newvalue):
self.old_objects_pointing_to_young.append(addr_struct)
- objhdr = self.header(addr_struct)
objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
#
# Second part: if 'addr_struct' is actually a prebuilt GC
@@ -785,6 +786,7 @@
def _init_writebarrier_with_card_marker(self):
+ DEBUG = self.DEBUG
def remember_young_pointer_from_array(addr_array, index):
# 'addr_array' is the address of the object in which we write,
# which must have an array part; 'index' is the index of the
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/test/test_direct.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/test/test_direct.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/test/test_direct.py Fri Oct 1 17:06:32 2010
@@ -96,7 +96,7 @@
newaddr = llmemory.cast_ptr_to_adr(newvalue)
addr_struct = llmemory.cast_ptr_to_adr(p)
if hasattr(self.gc, 'write_barrier_from_array'):
- self.gc.write_barrier_from_array(newvalue, addr_struct, index)
+ self.gc.write_barrier_from_array(newaddr, addr_struct, index)
else:
self.gc.write_barrier(newaddr, addr_struct)
p[index] = newvalue
From antocuni at codespeak.net Fri Oct 1 17:15:59 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri, 1 Oct 2010 17:15:59 +0200 (CEST)
Subject: [pypy-svn] r77533 - in pypy/branch/jitffi/pypy/rlib: . test
Message-ID: <20101001151559.65F0B282BDB@codespeak.net>
Author: antocuni
Date: Fri Oct 1 17:15:57 2010
New Revision: 77533
Modified:
pypy/branch/jitffi/pypy/rlib/libffi.py
pypy/branch/jitffi/pypy/rlib/test/test_clibffi.py
Log:
remove the dependency on clibffi.FuncPtr: now the new libffi.Func does the work by itself. This temporarly breaks some jit test, will fix them later
Modified: pypy/branch/jitffi/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/libffi.py Fri Oct 1 17:15:57 2010
@@ -1,7 +1,11 @@
-from pypy.rlib import clibffi
-from pypy.rlib.clibffi import get_libc_name
+from pypy.rpython.lltypesystem import rffi, lltype
from pypy.rlib.objectmodel import specialize
+from pypy.rlib.rarithmetic import intmask, r_uint
from pypy.rlib import jit
+from pypy.rlib import clibffi
+from pypy.rlib.clibffi import get_libc_name, FUNCFLAG_CDECL, AbstractFuncPtr, \
+ push_arg_as_ffiptr, c_ffi_call
+from pypy.rlib.rdynload import dlopen, dlclose, dlsym, dlsym_byordinal
def import_types():
g = globals()
@@ -12,6 +16,8 @@
del import_types
+# ----------------------------------------------------------------------
+
class AbstractArg(object):
next = None
@@ -22,8 +28,8 @@
def __init__(self, intval):
self.intval = intval
- def push(self, func):
- func._push_arg(self.intval)
+ def push(self, func, ll_args, i):
+ func._push_arg(self.intval, ll_args, i)
class FloatArg(AbstractArg):
""" An argument holding a float
@@ -32,45 +38,111 @@
def __init__(self, floatval):
self.floatval = floatval
- def push(self, func):
- func._push_arg(self.floatval)
+ def push(self, func, ll_args, i):
+ func._push_arg(self.floatval, ll_args, i)
+
-class Func(object):
+class Func(AbstractFuncPtr):
- _immutable_fields_ = ['funcptr', 'funcsym', 'argtypes', 'restype']
+ _immutable_fields_ = ['funcsym', 'argtypes', 'restype']
- def __init__(self, funcptr):
- # XXX: for now, this is just a wrapper around clibffi.FuncPtr, but in
- # the future it will replace it completely
- self.funcptr = funcptr
- self.funcsym = funcptr.funcsym
- self.argtypes = funcptr.argtypes
- self.restype = funcptr.restype
+ def __init__(self, name, argtypes, restype, funcsym, flags=FUNCFLAG_CDECL,
+ keepalive=None):
+ AbstractFuncPtr.__init__(self, name, argtypes, restype, flags)
+ self.keepalive = keepalive
+ self.funcsym = funcsym
def _prepare(self):
- pass
+ ll_args = lltype.malloc(rffi.VOIDPP.TO, len(self.argtypes), flavor='raw')
+ return ll_args
_prepare.oopspec = 'libffi_prepare_call(self)'
- def _push_arg(self, value):
- self.funcptr.push_arg(value)
+ def _push_arg(self, value, ll_args, i):
+ # XXX: check the type is not translated?
+ argtype = self.argtypes[i]
+ c_size = intmask(argtype.c_size)
+ ll_buf = lltype.malloc(rffi.CCHARP.TO, c_size, flavor='raw')
+ push_arg_as_ffiptr(argtype, value, ll_buf)
+ ll_args[i] = ll_buf
# XXX this is bad, fix it somehow in the future, but specialize:argtype
# doesn't work correctly with mixing non-negative and normal integers
#_push_arg._annenforceargs_ = [None, int]
_push_arg._annspecialcase_ = 'specialize:argtype(1)'
- _push_arg.oopspec = 'libffi_push_arg(self, value)'
+ _push_arg.oopspec = 'libffi_push_arg(self, value, ll_args, i)'
- def _do_call(self, funcsym, RESULT):
- return self.funcptr.call(RESULT)
+ def _do_call(self, funcsym, ll_args, RESULT):
+ # XXX: check len(args)?
+ ll_result = lltype.nullptr(rffi.CCHARP.TO)
+ if self.restype != ffi_type_void:
+ ll_result = lltype.malloc(rffi.CCHARP.TO,
+ intmask(self.restype.c_size),
+ flavor='raw')
+ ffires = c_ffi_call(self.ll_cif,
+ self.funcsym,
+ rffi.cast(rffi.VOIDP, ll_result),
+ rffi.cast(rffi.VOIDPP, ll_args))
+ if RESULT is not lltype.Void:
+ TP = lltype.Ptr(rffi.CArray(RESULT))
+ res = rffi.cast(TP, ll_result)[0]
+ else:
+ res = None
+ self._free_buffers(ll_result, ll_args)
+ #check_fficall_result(ffires, self.flags)
+ return res
_do_call._annspecialcase_ = 'specialize:arg(2)'
_do_call.oopspec = 'libffi_call(self, funcsym, RESULT)'
+ def _free_buffers(self, ll_result, ll_args):
+ lltype.free(ll_result, flavor='raw')
+ for i in range(len(self.argtypes)):
+ lltype.free(ll_args[i], flavor='raw')
+ lltype.free(ll_args, flavor='raw')
+
@jit.unroll_safe
@specialize.arg(2)
def call(self, argchain, RESULT):
- self._prepare()
+ # WARNING! This code is written carefully in a way that the JIT
+ # optimizer will see a sequence of calls like the following:
+ #
+ # libffi_prepare_call
+ # libffi_push_arg
+ # libffi_push_arg
+ # ...
+ # libffi_call
+ #
+ # It is important that there is no other operation in the middle, else
+ # the optimizer will fail to recognize the pattern and won't turn it
+ # into a fast CALL. Note that "arg = arg.next" is optimized away,
+ # assuming that archain is completely virtual.
+ ll_args = self._prepare()
+ i = 0
arg = argchain
while arg:
- arg.push(self)
+ arg.push(self, ll_args, i)
+ i += 1
arg = arg.next
- #return self.funcptr.call(RESULT)
- return self._do_call(self.funcsym, RESULT)
+ result = self._do_call(self.funcsym, ll_args, RESULT)
+ return result
+
+# ----------------------------------------------------------------------
+
+
+# XXX: it partially duplicate the code in clibffi.py
+class CDLL(object):
+ def __init__(self, libname):
+ """Load the library, or raises DLOpenError."""
+ self.lib = lltype.nullptr(rffi.CCHARP.TO)
+ ll_libname = rffi.str2charp(libname)
+ try:
+ self.lib = dlopen(ll_libname)
+ finally:
+ lltype.free(ll_libname, flavor='raw')
+
+ def __del__(self):
+ if self.lib:
+ dlclose(self.lib)
+ self.lib = lltype.nullptr(rffi.CCHARP.TO)
+
+ def getpointer(self, name, argtypes, restype, flags=FUNCFLAG_CDECL):
+ return Func(name, argtypes, restype, dlsym(self.lib, name),
+ flags=flags, keepalive=self)
Modified: pypy/branch/jitffi/pypy/rlib/test/test_clibffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_clibffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_clibffi.py Fri Oct 1 17:15:57 2010
@@ -11,13 +11,6 @@
import sys
import time
-def setup_module(mod):
- for name in type_names:
- # XXX force this to be seen by ll2ctypes
- # so that ALLOCATED.clear() clears it
- ffistruct = globals()[name]
- rffi.cast(rffi.VOIDP, ffistruct)
-
def get_libm_name(platform):
if platform == 'win32':
return 'msvcrt.dll'
@@ -27,17 +20,26 @@
return 'libm.so'
class BaseFfiTest(object):
+
+ def setup_class(cls):
+ for name in type_names:
+ # XXX force this to be seen by ll2ctypes
+ # so that ALLOCATED.clear() clears it
+ ffistruct = globals()[name]
+ rffi.cast(rffi.VOIDP, ffistruct)
+
+ def setup_method(self, meth):
+ ALLOCATED.clear()
+
+
+class TestCLibffi(BaseFfiTest):
+
def get_libc(self):
return CDLL(get_libc_name())
def get_libm(self):
return CDLL(get_libm_name(sys.platform))
-
-class TestCLibffi(BaseFfiTest):
- def setup_method(self, meth):
- ALLOCATED.clear()
-
def test_library_open(self):
lib = self.get_libc()
del lib
@@ -411,10 +413,11 @@
assert not ALLOCATED
-class TestWin32Handles:
+class TestWin32Handles(BaseFfiTest):
def setup_class(cls):
if sys.platform != 'win32':
py.test.skip("Handle to libc library, Win-only test")
+ BaseFfiTest.setup_class(cls)
def test_get_libc_handle(self):
handle = get_libc_handle()
From antocuni at codespeak.net Fri Oct 1 17:22:59 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri, 1 Oct 2010 17:22:59 +0200 (CEST)
Subject: [pypy-svn] r77534 - in pypy/branch/jitffi/pypy: jit/codewriter
jit/metainterp/test rlib
Message-ID: <20101001152259.37DCA282BDB@codespeak.net>
Author: antocuni
Date: Fri Oct 1 17:22:57 2010
New Revision: 77534
Modified:
pypy/branch/jitffi/pypy/jit/codewriter/support.py
pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py
pypy/branch/jitffi/pypy/rlib/libffi.py
Log:
fix the jit and the test, after the refactoring of libffi.py
Modified: pypy/branch/jitffi/pypy/jit/codewriter/support.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/codewriter/support.py (original)
+++ pypy/branch/jitffi/pypy/jit/codewriter/support.py Fri Oct 1 17:22:57 2010
@@ -222,18 +222,20 @@
# libffi support
# --------------
-def _ll_1_libffi_prepare_call(func):
- pass
+def _ll_1_libffi_prepare_call(llfunc):
+ from pypy.rlib.libffi import Func
+ func = cast_base_ptr_to_instance(Func, llfunc)
+ return func._prepare()
-def _ll_2_libffi_push_arg(llfunc, value):
+def _ll_4_libffi_push_arg(llfunc, value, ll_args, i):
from pypy.rlib.libffi import Func
func = cast_base_ptr_to_instance(Func, llfunc)
- return func._push_arg(value)
+ return func._push_arg(value, ll_args, i)
-def _ll_3_libffi_call(llfunc, funcsym, RESULT):
+def _ll_4_libffi_call(llfunc, funcsym, ll_args, RESULT):
from pypy.rlib.libffi import Func
func = cast_base_ptr_to_instance(Func, llfunc)
- return func._do_call(funcsym, lltype.Signed)
+ return func._do_call(funcsym, ll_args, lltype.Signed)
# XXX: should be RESULT, but it doesn't work
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py Fri Oct 1 17:22:57 2010
@@ -2,8 +2,7 @@
import py
from pypy.rlib.jit import JitDriver, hint
from pypy.jit.metainterp.test.test_basic import LLJitMixin
-from pypy.rlib.clibffi import FuncPtr, CDLL, ffi_type_sint
-from pypy.rlib.libffi import IntArg, Func
+from pypy.rlib.libffi import CDLL, ffi_type_sint, IntArg, Func
from pypy.tool.udir import udir
from pypy.translator.tool.cbuild import ExternalCompilationInfo
from pypy.translator.platform import platform
@@ -29,9 +28,8 @@
def f(n):
cdll = CDLL(self.lib_name)
- fn = cdll.getpointer('sum_xy', [ffi_type_sint, ffi_type_sint],
- ffi_type_sint)
- func = Func(fn)
+ func = cdll.getpointer('sum_xy', [ffi_type_sint, ffi_type_sint],
+ ffi_type_sint)
while n < 10:
driver.jit_merge_point(n=n, func=func)
driver.can_enter_jit(n=n, func=func)
Modified: pypy/branch/jitffi/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/libffi.py Fri Oct 1 17:22:57 2010
@@ -66,8 +66,8 @@
ll_args[i] = ll_buf
# XXX this is bad, fix it somehow in the future, but specialize:argtype
# doesn't work correctly with mixing non-negative and normal integers
- #_push_arg._annenforceargs_ = [None, int]
- _push_arg._annspecialcase_ = 'specialize:argtype(1)'
+ _push_arg._annenforceargs_ = [None, int, None, int]
+ #_push_arg._annspecialcase_ = 'specialize:argtype(1)'
_push_arg.oopspec = 'libffi_push_arg(self, value, ll_args, i)'
def _do_call(self, funcsym, ll_args, RESULT):
@@ -89,8 +89,8 @@
self._free_buffers(ll_result, ll_args)
#check_fficall_result(ffires, self.flags)
return res
- _do_call._annspecialcase_ = 'specialize:arg(2)'
- _do_call.oopspec = 'libffi_call(self, funcsym, RESULT)'
+ _do_call._annspecialcase_ = 'specialize:arg(3)'
+ _do_call.oopspec = 'libffi_call(self, funcsym, ll_args, RESULT)'
def _free_buffers(self, ll_result, ll_args):
lltype.free(ll_result, flavor='raw')
From arigo at codespeak.net Fri Oct 1 17:23:52 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 1 Oct 2010 17:23:52 +0200 (CEST)
Subject: [pypy-svn] r77535 -
pypy/branch/larger-writebarrier/pypy/rpython/memory/gctransform
Message-ID: <20101001152352.67787282BDB@codespeak.net>
Author: arigo
Date: Fri Oct 1 17:23:50 2010
New Revision: 77535
Modified:
pypy/branch/larger-writebarrier/pypy/rpython/memory/gctransform/framework.py
Log:
Fix: write_barrier_from_array() always takes 4 arguments now
(but is inlined and may call remember_young_pointer_from_array()
with less arguments).
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gctransform/framework.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gctransform/framework.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gctransform/framework.py Fri Oct 1 17:23:50 2010
@@ -443,6 +443,7 @@
self.write_barrier_from_array_ptr = getfn(func.im_func,
[s_gc,
annmodel.SomeAddress(),
+ annmodel.SomeAddress(),
annmodel.SomeInteger()],
annmodel.s_None,
inline=True)
@@ -1023,6 +1024,8 @@
and not isinstance(v_newvalue, Constant)
and v_struct.concretetype.TO._gckind == "gc"
and hop.spaceop not in self.clean_sets):
+ v_newvalue = hop.genop("cast_ptr_to_adr", [v_newvalue],
+ resulttype = llmemory.Address)
v_structaddr = hop.genop("cast_ptr_to_adr", [v_struct],
resulttype = llmemory.Address)
if (self.write_barrier_from_array_ptr is not None and
@@ -1032,12 +1035,11 @@
assert v_index.concretetype == lltype.Signed
hop.genop("direct_call", [self.write_barrier_from_array_ptr,
self.c_const_gc,
+ v_newvalue,
v_structaddr,
v_index])
else:
self.write_barrier_calls += 1
- v_newvalue = hop.genop("cast_ptr_to_adr", [v_newvalue],
- resulttype = llmemory.Address)
hop.genop("direct_call", [self.write_barrier_ptr,
self.c_const_gc,
v_newvalue,
From arigo at codespeak.net Fri Oct 1 17:54:06 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 1 Oct 2010 17:54:06 +0200 (CEST)
Subject: [pypy-svn] r77536 - in
pypy/branch/larger-writebarrier/pypy/rpython/memory: . gc
gc/test test
Message-ID: <20101001155406.A04FB282BDB@codespeak.net>
Author: arigo
Date: Fri Oct 1 17:54:04 2010
New Revision: 77536
Modified:
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/base.py
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/generation.py
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/markcompact.py
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/marksweep.py
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/semispace.py
pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/test/test_direct.py
pypy/branch/larger-writebarrier/pypy/rpython/memory/gcwrapper.py
pypy/branch/larger-writebarrier/pypy/rpython/memory/test/test_transformed_gc.py
Log:
Fix. This really requires doing something different if we are
translated to C or not.
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/base.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/base.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/base.py Fri Oct 1 17:54:04 2010
@@ -20,12 +20,15 @@
prebuilt_gc_objects_are_static_roots = True
object_minimal_size = 0
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE):
+ def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
+ translated_to_c=True):
self.gcheaderbuilder = GCHeaderBuilder(self.HDR)
self.AddressStack = get_address_stack(chunk_size)
self.AddressDeque = get_address_deque(chunk_size)
self.AddressDict = AddressDict
self.config = config
+ assert isinstance(translated_to_c, bool)
+ self.translated_to_c = translated_to_c
def setup(self):
# all runtime mutable values' setup should happen here
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/generation.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/generation.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/generation.py Fri Oct 1 17:54:04 2010
@@ -5,7 +5,6 @@
from pypy.rpython.memory.gc.base import read_from_env
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
from pypy.rpython.lltypesystem import lltype, llmemory, llarena
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rlib.objectmodel import free_non_gc_object
from pypy.rlib.debug import ll_assert
from pypy.rlib.debug import debug_print, debug_start, debug_stop
@@ -49,15 +48,17 @@
nursery_hash_base = -1
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
+ def __init__(self, config,
nursery_size=32*WORD,
min_nursery_size=32*WORD,
auto_nursery_size=False,
space_size=1024*WORD,
- max_space_size=sys.maxint//2+1):
- SemiSpaceGC.__init__(self, config, chunk_size = chunk_size,
+ max_space_size=sys.maxint//2+1,
+ **kwds):
+ SemiSpaceGC.__init__(self, config,
space_size = space_size,
- max_space_size = max_space_size)
+ max_space_size = max_space_size,
+ **kwds)
assert min_nursery_size <= nursery_size <= space_size // 2
self.initial_nursery_size = nursery_size
self.auto_nursery_size = auto_nursery_size
@@ -160,6 +161,9 @@
def appears_to_be_in_nursery(self, addr):
# same as is_in_nursery(), but may return True accidentally if
# 'addr' is a tagged pointer with just the wrong value.
+ if not self.translated_to_c:
+ if not self.is_valid_gc_object(addr):
+ return False
return self.nursery <= addr < self.nursery_top
def malloc_fixedsize_clear(self, typeid, size, can_collect,
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/markcompact.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/markcompact.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/markcompact.py Fri Oct 1 17:54:04 2010
@@ -2,7 +2,6 @@
from pypy.rpython.memory.gc.base import MovingGCBase, read_from_env
from pypy.rlib.debug import ll_assert, have_debug_prints
from pypy.rlib.debug import debug_print, debug_start, debug_stop
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rpython.memory.support import get_address_stack, get_address_deque
from pypy.rpython.memory.support import AddressDict
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
@@ -86,9 +85,9 @@
free = NULL
next_collect_after = -1
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, space_size=4096,
- min_next_collect_after=128):
- MovingGCBase.__init__(self, config, chunk_size)
+ def __init__(self, config, space_size=4096,
+ min_next_collect_after=128, **kwds):
+ MovingGCBase.__init__(self, config, **kwds)
self.space_size = space_size
self.min_next_collect_after = min_next_collect_after
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/marksweep.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/marksweep.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/marksweep.py Fri Oct 1 17:54:04 2010
@@ -1,7 +1,6 @@
from pypy.rpython.lltypesystem.llmemory import raw_malloc, raw_free
from pypy.rpython.lltypesystem.llmemory import raw_memcopy, raw_memclear
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rpython.memory.support import get_address_stack
from pypy.rpython.memory.gcheader import GCHeaderBuilder
from pypy.rpython.lltypesystem import lltype, llmemory, rffi, llgroup
@@ -48,9 +47,9 @@
# translating to a real backend.
TRANSLATION_PARAMS = {'start_heap_size': 8*1024*1024} # XXX adjust
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, start_heap_size=4096):
+ def __init__(self, config, start_heap_size=4096, **kwds):
self.param_start_heap_size = start_heap_size
- GCBase.__init__(self, config, chunk_size)
+ GCBase.__init__(self, config, **kwds)
def setup(self):
GCBase.setup(self)
@@ -714,8 +713,8 @@
_alloc_flavor_ = "raw"
COLLECT_EVERY = 2000
- def __init__(self, chunk_size=DEFAULT_CHUNK_SIZE, start_heap_size=4096):
- MarkSweepGC.__init__(self, chunk_size, start_heap_size)
+ def __init__(self, config, **kwds):
+ MarkSweepGC.__init__(self, config, **kwds)
self.count_mallocs = 0
def maybe_collect(self):
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/minimark.py Fri Oct 1 17:54:04 2010
@@ -4,7 +4,6 @@
from pypy.rpython.lltypesystem.llmemory import raw_malloc_usage
from pypy.rpython.memory.gc.base import GCBase, MovingGCBase
from pypy.rpython.memory.gc import minimarkpage, base, generation
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask, r_uint
from pypy.rlib.rarithmetic import LONG_BIT_SHIFT
from pypy.rlib.debug import ll_assert, debug_print, debug_start, debug_stop
@@ -140,7 +139,7 @@
"large_object_gcptrs": 8250*WORD,
}
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
+ def __init__(self, config,
read_from_env=False,
nursery_size=32*WORD,
page_size=16*WORD,
@@ -150,8 +149,9 @@
card_page_indices=0,
large_object=8*WORD,
large_object_gcptrs=10*WORD,
- ArenaCollectionClass=None):
- MovingGCBase.__init__(self, config, chunk_size)
+ ArenaCollectionClass=None,
+ **kwds):
+ MovingGCBase.__init__(self, config, **kwds)
assert small_request_threshold % WORD == 0
self.read_from_env = read_from_env
self.nursery_size = nursery_size
@@ -639,6 +639,9 @@
def appears_to_be_in_nursery(self, addr):
# same as is_in_nursery(), but may return True accidentally if
# 'addr' is a tagged pointer with just the wrong value.
+ if not self.translated_to_c:
+ if not self.is_valid_gc_object(addr):
+ return False
return self.nursery <= addr < self.nursery_top
def is_forwarded(self, obj):
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/semispace.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/semispace.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/semispace.py Fri Oct 1 17:54:04 2010
@@ -1,7 +1,6 @@
from pypy.rpython.lltypesystem.llmemory import raw_malloc, raw_free
from pypy.rpython.lltypesystem.llmemory import raw_memcopy, raw_memclear
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rpython.memory.support import get_address_stack, get_address_deque
from pypy.rpython.memory.support import AddressDict
from pypy.rpython.lltypesystem import lltype, llmemory, llarena, rffi, llgroup
@@ -59,11 +58,11 @@
# translating to a real backend.
TRANSLATION_PARAMS = {'space_size': 8*1024*1024} # XXX adjust
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, space_size=4096,
- max_space_size=sys.maxint//2+1):
+ def __init__(self, config, space_size=4096, max_space_size=sys.maxint//2+1,
+ **kwds):
self.param_space_size = space_size
self.param_max_space_size = max_space_size
- MovingGCBase.__init__(self, config, chunk_size)
+ MovingGCBase.__init__(self, config, **kwds)
def setup(self):
#self.total_collection_time = 0.0
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/test/test_direct.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/test/test_direct.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gc/test/test_direct.py Fri Oct 1 17:54:04 2010
@@ -70,6 +70,7 @@
GC_PARAMS = self.GC_PARAMS.copy()
if hasattr(meth, 'GC_PARAMS'):
GC_PARAMS.update(meth.GC_PARAMS)
+ GC_PARAMS['translated_to_c'] = False
self.gc = self.GCClass(config, **GC_PARAMS)
self.gc.DEBUG = True
self.rootwalker = DirectRootWalker(self)
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/gcwrapper.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/gcwrapper.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/gcwrapper.py Fri Oct 1 17:54:04 2010
@@ -9,7 +9,10 @@
def __init__(self, llinterp, flowgraphs, gc_class, GC_PARAMS={}):
translator = llinterp.typer.annotator.translator
config = translator.config.translation
- self.gc = gc_class(config, chunk_size = 10, **GC_PARAMS)
+ self.gc = gc_class(config,
+ chunk_size = 10,
+ translated_to_c = False,
+ **GC_PARAMS)
self.gc.set_root_walker(LLInterpRootWalker(self))
self.gc.DEBUG = True
self.llinterp = llinterp
Modified: pypy/branch/larger-writebarrier/pypy/rpython/memory/test/test_transformed_gc.py
==============================================================================
--- pypy/branch/larger-writebarrier/pypy/rpython/memory/test/test_transformed_gc.py (original)
+++ pypy/branch/larger-writebarrier/pypy/rpython/memory/test/test_transformed_gc.py Fri Oct 1 17:54:04 2010
@@ -906,7 +906,8 @@
gcname = "marksweep"
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
- GC_PARAMS = {'start_heap_size': 1024*WORD }
+ GC_PARAMS = {'start_heap_size': 1024*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
@@ -1144,7 +1145,8 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.marksweep import PrintingMarkSweepGC as GCClass
- GC_PARAMS = {'start_heap_size': 1024*WORD }
+ GC_PARAMS = {'start_heap_size': 1024*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestSemiSpaceGC(GenericMovingGCTests):
@@ -1154,7 +1156,8 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.semispace import SemiSpaceGC as GCClass
- GC_PARAMS = {'space_size': 512*WORD}
+ GC_PARAMS = {'space_size': 512*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestMarkCompactGC(GenericMovingGCTests):
@@ -1163,7 +1166,8 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.markcompact import MarkCompactGC as GCClass
- GC_PARAMS = {'space_size': 4096*WORD}
+ GC_PARAMS = {'space_size': 4096*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestGenerationGC(GenericMovingGCTests):
@@ -1175,7 +1179,8 @@
from pypy.rpython.memory.gc.generation import GenerationGC as \
GCClass
GC_PARAMS = {'space_size': 512*WORD,
- 'nursery_size': 32*WORD}
+ 'nursery_size': 32*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
def define_weakref_across_minor_collection(cls):
@@ -1372,7 +1377,8 @@
GenerationGC._teardown(self)
GC_PARAMS = {'space_size': 512*WORD,
- 'nursery_size': 128*WORD}
+ 'nursery_size': 128*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
def define_working_nursery(cls):
@@ -1404,7 +1410,8 @@
from pypy.rpython.memory.gc.hybrid import HybridGC as GCClass
GC_PARAMS = {'space_size': 512*WORD,
'nursery_size': 32*WORD,
- 'large_object': 8*WORD}
+ 'large_object': 8*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
def define_ref_from_rawmalloced_to_regular(cls):
@@ -1477,6 +1484,7 @@
'large_object': 8*WORD,
'large_object_gcptrs': 10*WORD,
'card_page_indices': 4,
+ 'translated_to_c': False,
}
root_stack_depth = 200
@@ -1585,7 +1593,8 @@
gcname = "marksweep"
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
- GC_PARAMS = {'start_heap_size': 1024*WORD }
+ GC_PARAMS = {'start_heap_size': 1024*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestHybridTaggedPointerGC(TaggedPointerGCTests):
@@ -1596,7 +1605,8 @@
from pypy.rpython.memory.gc.generation import GenerationGC as \
GCClass
GC_PARAMS = {'space_size': 512*WORD,
- 'nursery_size': 32*WORD}
+ 'nursery_size': 32*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestMarkCompactTaggedpointerGC(TaggedPointerGCTests):
@@ -1605,5 +1615,6 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.markcompact import MarkCompactGC as GCClass
- GC_PARAMS = {'space_size': 4096*WORD}
+ GC_PARAMS = {'space_size': 4096*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
From arigo at codespeak.net Fri Oct 1 17:56:04 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 1 Oct 2010 17:56:04 +0200 (CEST)
Subject: [pypy-svn] r77537 - in pypy/trunk/pypy: jit/backend/llgraph
jit/backend/llsupport jit/backend/llsupport/test
jit/backend/test jit/backend/x86 jit/metainterp
rpython/memory rpython/memory/gc rpython/memory/gc/test
rpython/memory/gctransform rpython/memory/test
Message-ID: <20101001155604.74639282BDB@codespeak.net>
Author: arigo
Date: Fri Oct 1 17:56:02 2010
New Revision: 77537
Modified:
pypy/trunk/pypy/jit/backend/llgraph/llimpl.py
pypy/trunk/pypy/jit/backend/llsupport/gc.py
pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py
pypy/trunk/pypy/jit/backend/test/runner_test.py
pypy/trunk/pypy/jit/backend/x86/assembler.py
pypy/trunk/pypy/jit/backend/x86/regalloc.py
pypy/trunk/pypy/jit/metainterp/resoperation.py
pypy/trunk/pypy/rpython/memory/gc/base.py
pypy/trunk/pypy/rpython/memory/gc/generation.py
pypy/trunk/pypy/rpython/memory/gc/markcompact.py
pypy/trunk/pypy/rpython/memory/gc/marksweep.py
pypy/trunk/pypy/rpython/memory/gc/minimark.py
pypy/trunk/pypy/rpython/memory/gc/semispace.py
pypy/trunk/pypy/rpython/memory/gc/test/test_direct.py
pypy/trunk/pypy/rpython/memory/gctransform/framework.py
pypy/trunk/pypy/rpython/memory/gcwrapper.py
pypy/trunk/pypy/rpython/memory/test/test_transformed_gc.py
Log:
Merge branch/larger-writebarrier:
It seems that "smaller-writebarrier" may have had a performance impact on a few
benchmarks. This is a mixed solution: using the old two-args write barrier
for regular setfields, and the smaller (2 args instead of 3) version for
setarrayitem.
Modified: pypy/trunk/pypy/jit/backend/llgraph/llimpl.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llgraph/llimpl.py (original)
+++ pypy/trunk/pypy/jit/backend/llgraph/llimpl.py Fri Oct 1 17:56:02 2010
@@ -129,7 +129,7 @@
'arraylen_gc' : (('ref',), 'int'),
'call' : (('ref', 'varargs'), 'intorptr'),
'call_assembler' : (('varargs',), 'intorptr'),
- 'cond_call_gc_wb' : (('ptr',), None),
+ 'cond_call_gc_wb' : (('ptr', 'ptr'), None),
'oosend' : (('varargs',), 'intorptr'),
'oosend_pure' : (('varargs',), 'intorptr'),
'guard_true' : (('bool',), None),
@@ -810,7 +810,7 @@
FLOAT: 0.0}
return d[calldescr.typeinfo]
- def op_cond_call_gc_wb(self, descr, a):
+ def op_cond_call_gc_wb(self, descr, a, b):
py.test.skip("cond_call_gc_wb not supported")
def op_oosend(self, descr, obj, *args):
Modified: pypy/trunk/pypy/jit/backend/llsupport/gc.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llsupport/gc.py (original)
+++ pypy/trunk/pypy/jit/backend/llsupport/gc.py Fri Oct 1 17:56:02 2010
@@ -404,7 +404,7 @@
self.GC_MALLOC_BASIC = lltype.Ptr(lltype.FuncType(
[lltype.Signed, lltype.Signed], llmemory.GCREF))
self.WB_FUNCPTR = lltype.Ptr(lltype.FuncType(
- [llmemory.Address], lltype.Void))
+ [llmemory.Address, llmemory.Address], lltype.Void))
self.write_barrier_descr = WriteBarrierDescr(self)
#
def malloc_array(itemsize, tid, num_elem):
@@ -550,7 +550,8 @@
# the GC, and call it immediately
llop1 = self.llop1
funcptr = llop1.get_write_barrier_failing_case(self.WB_FUNCPTR)
- funcptr(llmemory.cast_ptr_to_adr(gcref_struct))
+ funcptr(llmemory.cast_ptr_to_adr(gcref_struct),
+ llmemory.cast_ptr_to_adr(gcref_newptr))
def rewrite_assembler(self, cpu, operations):
# Perform two kinds of rewrites in parallel:
@@ -589,7 +590,7 @@
v = op.getarg(1)
if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and
bool(v.value)): # store a non-NULL
- self._gen_write_barrier(newops, op.getarg(0))
+ self._gen_write_barrier(newops, op.getarg(0), v)
op = op.copy_and_change(rop.SETFIELD_RAW)
# ---------- write barrier for SETARRAYITEM_GC ----------
if op.getopnum() == rop.SETARRAYITEM_GC:
@@ -598,15 +599,15 @@
bool(v.value)): # store a non-NULL
# XXX detect when we should produce a
# write_barrier_from_array
- self._gen_write_barrier(newops, op.getarg(0))
+ self._gen_write_barrier(newops, op.getarg(0), v)
op = op.copy_and_change(rop.SETARRAYITEM_RAW)
# ----------
newops.append(op)
del operations[:]
operations.extend(newops)
- def _gen_write_barrier(self, newops, v_base):
- args = [v_base]
+ def _gen_write_barrier(self, newops, v_base, v_value):
+ args = [v_base, v_value]
newops.append(ResOperation(rop.COND_CALL_GC_WB, args, None,
descr=self.write_barrier_descr))
Modified: pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py (original)
+++ pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py Fri Oct 1 17:56:02 2010
@@ -141,8 +141,8 @@
repr(offset_to_length), p))
return p
- def _write_barrier_failing_case(self, adr_struct):
- self.record.append(('barrier', adr_struct))
+ def _write_barrier_failing_case(self, adr_struct, adr_newptr):
+ self.record.append(('barrier', adr_struct, adr_newptr))
def get_write_barrier_failing_case(self, FPTRTYPE):
return llhelper(FPTRTYPE, self._write_barrier_failing_case)
@@ -239,6 +239,7 @@
s_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
r_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, r)
s_adr = llmemory.cast_ptr_to_adr(s)
+ r_adr = llmemory.cast_ptr_to_adr(r)
#
s_hdr.tid &= ~gc_ll_descr.GCClass.JIT_WB_IF_FLAG
gc_ll_descr.do_write_barrier(s_gcref, r_gcref)
@@ -246,7 +247,7 @@
#
s_hdr.tid |= gc_ll_descr.GCClass.JIT_WB_IF_FLAG
gc_ll_descr.do_write_barrier(s_gcref, r_gcref)
- assert self.llop1.record == [('barrier', s_adr)]
+ assert self.llop1.record == [('barrier', s_adr, r_adr)]
def test_gen_write_barrier(self):
gc_ll_descr = self.gc_ll_descr
@@ -254,11 +255,13 @@
#
newops = []
v_base = BoxPtr()
- gc_ll_descr._gen_write_barrier(newops, v_base)
+ v_value = BoxPtr()
+ gc_ll_descr._gen_write_barrier(newops, v_base, v_value)
assert llop1.record == []
assert len(newops) == 1
assert newops[0].getopnum() == rop.COND_CALL_GC_WB
assert newops[0].getarg(0) == v_base
+ assert newops[0].getarg(1) == v_value
assert newops[0].result is None
wbdescr = newops[0].getdescr()
assert isinstance(wbdescr.jit_wb_if_flag, int)
@@ -358,6 +361,7 @@
#
assert operations[0].getopnum() == rop.COND_CALL_GC_WB
assert operations[0].getarg(0) == v_base
+ assert operations[0].getarg(1) == v_value
assert operations[0].result is None
#
assert operations[1].getopnum() == rop.SETFIELD_RAW
@@ -381,6 +385,7 @@
#
assert operations[0].getopnum() == rop.COND_CALL_GC_WB
assert operations[0].getarg(0) == v_base
+ assert operations[0].getarg(1) == v_value
assert operations[0].result is None
#
assert operations[1].getopnum() == rop.SETARRAYITEM_RAW
Modified: pypy/trunk/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/trunk/pypy/jit/backend/test/runner_test.py Fri Oct 1 17:56:02 2010
@@ -1427,12 +1427,12 @@
assert not excvalue
def test_cond_call_gc_wb(self):
- def func_void(a):
- record.append(a)
+ def func_void(a, b):
+ record.append((a, b))
record = []
#
S = lltype.GcStruct('S', ('tid', lltype.Signed))
- FUNC = self.FuncType([lltype.Ptr(S)], lltype.Void)
+ FUNC = self.FuncType([lltype.Ptr(S), lltype.Signed], lltype.Void)
func_ptr = llhelper(lltype.Ptr(FUNC), func_void)
funcbox = self.get_funcbox(self.cpu, func_ptr)
class WriteBarrierDescr(AbstractDescr):
@@ -1453,10 +1453,10 @@
sgcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
del record[:]
self.execute_operation(rop.COND_CALL_GC_WB,
- [BoxPtr(sgcref)],
+ [BoxPtr(sgcref), ConstInt(-2121)],
'void', descr=WriteBarrierDescr())
if cond:
- assert record == [s]
+ assert record == [(s, -2121)]
else:
assert record == []
Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/assembler.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/assembler.py Fri Oct 1 17:56:02 2010
@@ -1780,11 +1780,12 @@
self.mc.PUSH_i32(loc.getint())
if IS_X86_64:
- # We clobber this register to pass the arguments, but that's
+ # We clobber these registers to pass the arguments, but that's
# okay, because consider_cond_call_gc_wb makes sure that any
# caller-save registers with values in them are present in arglocs,
# so they are saved on the stack above and restored below
self.mc.MOV_rs(edi.value, 0)
+ self.mc.MOV_rs(esi.value, 8)
# misaligned stack in the call, but it's ok because the write barrier
# is not going to call anything more. Also, this assumes that the
Modified: pypy/trunk/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/regalloc.py Fri Oct 1 17:56:02 2010
@@ -696,9 +696,13 @@
def consider_cond_call_gc_wb(self, op):
assert op.result is None
args = op.getarglist()
+ loc_newvalue = self.rm.make_sure_var_in_reg(op.getarg(1), args)
+ # ^^^ we force loc_newvalue in a reg (unless it's a Const),
+ # because it will be needed anyway by the following setfield_gc.
+ # It avoids loading it twice from the memory.
loc_base = self.rm.make_sure_var_in_reg(op.getarg(0), args,
imm_fine=False)
- arglocs = [loc_base]
+ arglocs = [loc_base, loc_newvalue]
# add eax, ecx and edx as extra "arguments" to ensure they are
# saved and restored. Fish in self.rm to know which of these
# registers really need to be saved (a bit of a hack). Moreover,
Modified: pypy/trunk/pypy/jit/metainterp/resoperation.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/resoperation.py (original)
+++ pypy/trunk/pypy/jit/metainterp/resoperation.py Fri Oct 1 17:56:02 2010
@@ -455,7 +455,7 @@
'UNICODESETITEM/3',
'NEWUNICODE/1',
#'RUNTIMENEW/1', # ootype operation
- 'COND_CALL_GC_WB/1d', # [objptr] (for the write barrier)
+ 'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier)
'DEBUG_MERGE_POINT/1', # debugging only
'VIRTUAL_REF_FINISH/2', # removed before it's passed to the backend
'COPYSTRCONTENT/5', # src, dst, srcstart, dststart, length
Modified: pypy/trunk/pypy/rpython/memory/gc/base.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/gc/base.py (original)
+++ pypy/trunk/pypy/rpython/memory/gc/base.py Fri Oct 1 17:56:02 2010
@@ -20,12 +20,15 @@
prebuilt_gc_objects_are_static_roots = True
object_minimal_size = 0
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE):
+ def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
+ translated_to_c=True):
self.gcheaderbuilder = GCHeaderBuilder(self.HDR)
self.AddressStack = get_address_stack(chunk_size)
self.AddressDeque = get_address_deque(chunk_size)
self.AddressDict = AddressDict
self.config = config
+ assert isinstance(translated_to_c, bool)
+ self.translated_to_c = translated_to_c
def setup(self):
# all runtime mutable values' setup should happen here
@@ -79,7 +82,7 @@
def set_root_walker(self, root_walker):
self.root_walker = root_walker
- def write_barrier(self, addr_struct):
+ def write_barrier(self, newvalue, addr_struct):
pass
def statistics(self, index):
Modified: pypy/trunk/pypy/rpython/memory/gc/generation.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/gc/generation.py (original)
+++ pypy/trunk/pypy/rpython/memory/gc/generation.py Fri Oct 1 17:56:02 2010
@@ -5,7 +5,6 @@
from pypy.rpython.memory.gc.base import read_from_env
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
from pypy.rpython.lltypesystem import lltype, llmemory, llarena
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rlib.objectmodel import free_non_gc_object
from pypy.rlib.debug import ll_assert
from pypy.rlib.debug import debug_print, debug_start, debug_stop
@@ -49,15 +48,17 @@
nursery_hash_base = -1
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
+ def __init__(self, config,
nursery_size=32*WORD,
min_nursery_size=32*WORD,
auto_nursery_size=False,
space_size=1024*WORD,
- max_space_size=sys.maxint//2+1):
- SemiSpaceGC.__init__(self, config, chunk_size = chunk_size,
+ max_space_size=sys.maxint//2+1,
+ **kwds):
+ SemiSpaceGC.__init__(self, config,
space_size = space_size,
- max_space_size = max_space_size)
+ max_space_size = max_space_size,
+ **kwds)
assert min_nursery_size <= nursery_size <= space_size // 2
self.initial_nursery_size = nursery_size
self.auto_nursery_size = auto_nursery_size
@@ -157,6 +158,14 @@
"odd-valued (i.e. tagged) pointer unexpected here")
return self.nursery <= addr < self.nursery_top
+ def appears_to_be_in_nursery(self, addr):
+ # same as is_in_nursery(), but may return True accidentally if
+ # 'addr' is a tagged pointer with just the wrong value.
+ if not self.translated_to_c:
+ if not self.is_valid_gc_object(addr):
+ return False
+ return self.nursery <= addr < self.nursery_top
+
def malloc_fixedsize_clear(self, typeid, size, can_collect,
has_finalizer=False, contains_weakptr=False):
if (has_finalizer or not can_collect or
@@ -326,7 +335,7 @@
addr = pointer.address[0]
newaddr = self.copy(addr)
pointer.address[0] = newaddr
- self.write_into_last_generation_obj(obj)
+ self.write_into_last_generation_obj(obj, newaddr)
# ____________________________________________________________
# Implementation of nursery-only collections
@@ -457,9 +466,9 @@
# "if addr_struct.int0 & JIT_WB_IF_FLAG: remember_young_pointer()")
JIT_WB_IF_FLAG = GCFLAG_NO_YOUNG_PTRS
- def write_barrier(self, addr_struct):
- if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS:
- self.remember_young_pointer(addr_struct)
+ def write_barrier(self, newvalue, addr_struct):
+ if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS:
+ self.remember_young_pointer(addr_struct, newvalue)
def _setup_wb(self):
DEBUG = self.DEBUG
@@ -470,23 +479,33 @@
# For x86, there is also an extra requirement: when the JIT calls
# remember_young_pointer(), it assumes that it will not touch the SSE
# registers, so it does not save and restore them (that's a *hack*!).
- def remember_young_pointer(addr_struct):
+ def remember_young_pointer(addr_struct, addr):
#llop.debug_print(lltype.Void, "\tremember_young_pointer",
# addr_struct, "<-", addr)
if DEBUG:
ll_assert(not self.is_in_nursery(addr_struct),
"nursery object with GCFLAG_NO_YOUNG_PTRS")
- self.old_objects_pointing_to_young.append(addr_struct)
- self.header(addr_struct).tid &= ~GCFLAG_NO_YOUNG_PTRS
- self.write_into_last_generation_obj(addr_struct)
+ #
+ # What is important in this function is that it *must*
+ # clear the flag GCFLAG_NO_YOUNG_PTRS from 'addr_struct'
+ # if 'addr' is in the nursery. It is ok if, accidentally,
+ # it also clears the flag in some more rare cases, like
+ # 'addr' being a tagged pointer whose value happens to be
+ # a large integer that fools is_in_nursery().
+ if self.appears_to_be_in_nursery(addr):
+ self.old_objects_pointing_to_young.append(addr_struct)
+ self.header(addr_struct).tid &= ~GCFLAG_NO_YOUNG_PTRS
+ self.write_into_last_generation_obj(addr_struct, addr)
remember_young_pointer._dont_inline_ = True
self.remember_young_pointer = remember_young_pointer
- def write_into_last_generation_obj(self, addr_struct):
+ def write_into_last_generation_obj(self, addr_struct, addr):
objhdr = self.header(addr_struct)
if objhdr.tid & GCFLAG_NO_HEAP_PTRS:
- objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
- self.last_generation_root_objects.append(addr_struct)
+ if (self.is_valid_gc_object(addr) and
+ not self.is_last_generation(addr)):
+ objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
+ self.last_generation_root_objects.append(addr_struct)
write_into_last_generation_obj._always_inline_ = True
def assume_young_pointers(self, addr_struct):
Modified: pypy/trunk/pypy/rpython/memory/gc/markcompact.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/gc/markcompact.py (original)
+++ pypy/trunk/pypy/rpython/memory/gc/markcompact.py Fri Oct 1 17:56:02 2010
@@ -2,7 +2,6 @@
from pypy.rpython.memory.gc.base import MovingGCBase, read_from_env
from pypy.rlib.debug import ll_assert, have_debug_prints
from pypy.rlib.debug import debug_print, debug_start, debug_stop
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rpython.memory.support import get_address_stack, get_address_deque
from pypy.rpython.memory.support import AddressDict
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
@@ -86,9 +85,9 @@
free = NULL
next_collect_after = -1
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, space_size=4096,
- min_next_collect_after=128):
- MovingGCBase.__init__(self, config, chunk_size)
+ def __init__(self, config, space_size=4096,
+ min_next_collect_after=128, **kwds):
+ MovingGCBase.__init__(self, config, **kwds)
self.space_size = space_size
self.min_next_collect_after = min_next_collect_after
Modified: pypy/trunk/pypy/rpython/memory/gc/marksweep.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/gc/marksweep.py (original)
+++ pypy/trunk/pypy/rpython/memory/gc/marksweep.py Fri Oct 1 17:56:02 2010
@@ -1,7 +1,6 @@
from pypy.rpython.lltypesystem.llmemory import raw_malloc, raw_free
from pypy.rpython.lltypesystem.llmemory import raw_memcopy, raw_memclear
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rpython.memory.support import get_address_stack
from pypy.rpython.memory.gcheader import GCHeaderBuilder
from pypy.rpython.lltypesystem import lltype, llmemory, rffi, llgroup
@@ -48,9 +47,9 @@
# translating to a real backend.
TRANSLATION_PARAMS = {'start_heap_size': 8*1024*1024} # XXX adjust
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, start_heap_size=4096):
+ def __init__(self, config, start_heap_size=4096, **kwds):
self.param_start_heap_size = start_heap_size
- GCBase.__init__(self, config, chunk_size)
+ GCBase.__init__(self, config, **kwds)
def setup(self):
GCBase.setup(self)
@@ -714,8 +713,8 @@
_alloc_flavor_ = "raw"
COLLECT_EVERY = 2000
- def __init__(self, chunk_size=DEFAULT_CHUNK_SIZE, start_heap_size=4096):
- MarkSweepGC.__init__(self, chunk_size, start_heap_size)
+ def __init__(self, config, **kwds):
+ MarkSweepGC.__init__(self, config, **kwds)
self.count_mallocs = 0
def maybe_collect(self):
Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/gc/minimark.py (original)
+++ pypy/trunk/pypy/rpython/memory/gc/minimark.py Fri Oct 1 17:56:02 2010
@@ -4,7 +4,6 @@
from pypy.rpython.lltypesystem.llmemory import raw_malloc_usage
from pypy.rpython.memory.gc.base import GCBase, MovingGCBase
from pypy.rpython.memory.gc import minimarkpage, base, generation
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask, r_uint
from pypy.rlib.rarithmetic import LONG_BIT_SHIFT
from pypy.rlib.debug import ll_assert, debug_print, debug_start, debug_stop
@@ -140,7 +139,7 @@
"large_object_gcptrs": 8250*WORD,
}
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
+ def __init__(self, config,
read_from_env=False,
nursery_size=32*WORD,
page_size=16*WORD,
@@ -150,8 +149,9 @@
card_page_indices=0,
large_object=8*WORD,
large_object_gcptrs=10*WORD,
- ArenaCollectionClass=None):
- MovingGCBase.__init__(self, config, chunk_size)
+ ArenaCollectionClass=None,
+ **kwds):
+ MovingGCBase.__init__(self, config, **kwds)
assert small_request_threshold % WORD == 0
self.read_from_env = read_from_env
self.nursery_size = nursery_size
@@ -636,6 +636,14 @@
"odd-valued (i.e. tagged) pointer unexpected here")
return self.nursery <= addr < self.nursery_top
+ def appears_to_be_in_nursery(self, addr):
+ # same as is_in_nursery(), but may return True accidentally if
+ # 'addr' is a tagged pointer with just the wrong value.
+ if not self.translated_to_c:
+ if not self.is_valid_gc_object(addr):
+ return False
+ return self.nursery <= addr < self.nursery_top
+
def is_forwarded(self, obj):
"""Returns True if the nursery obj is marked as forwarded.
Implemented a bit obscurely by checking an unrelated flag
@@ -726,16 +734,16 @@
def JIT_max_size_of_young_obj(cls):
return cls.TRANSLATION_PARAMS['large_object']
- def write_barrier(self, addr_struct):
+ def write_barrier(self, newvalue, addr_struct):
if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS:
- self.remember_young_pointer(addr_struct)
+ self.remember_young_pointer(addr_struct, newvalue)
- def write_barrier_from_array(self, addr_array, index):
+ def write_barrier_from_array(self, newvalue, addr_array, index):
if self.header(addr_array).tid & GCFLAG_NO_YOUNG_PTRS:
if self.card_page_indices > 0: # <- constant-folded
self.remember_young_pointer_from_array(addr_array, index)
else:
- self.remember_young_pointer(addr_array)
+ self.remember_young_pointer(addr_array, newvalue)
def _init_writebarrier_logic(self):
DEBUG = self.DEBUG
@@ -746,27 +754,28 @@
# For x86, there is also an extra requirement: when the JIT calls
# remember_young_pointer(), it assumes that it will not touch the SSE
# registers, so it does not save and restore them (that's a *hack*!).
- def remember_young_pointer(addr_struct):
+ def remember_young_pointer(addr_struct, newvalue):
# 'addr_struct' is the address of the object in which we write.
+ # 'newvalue' is the address that we are going to write in there.
if DEBUG:
ll_assert(not self.is_in_nursery(addr_struct),
"nursery object with GCFLAG_NO_YOUNG_PTRS")
#
- # We assume that what we are writing is a pointer to the nursery
- # (and don't care for the fact that this new pointer may not
- # actually point to the nursery, which seems ok). What we need is
+ # If it seems that what we are writing is a pointer to the nursery
+ # (as checked with appears_to_be_in_nursery()), then we need
# to remove the flag GCFLAG_NO_YOUNG_PTRS and add the old object
# to the list 'old_objects_pointing_to_young'. We know that
# 'addr_struct' cannot be in the nursery, because nursery objects
# never have the flag GCFLAG_NO_YOUNG_PTRS to start with.
- self.old_objects_pointing_to_young.append(addr_struct)
objhdr = self.header(addr_struct)
- objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
+ if self.appears_to_be_in_nursery(newvalue):
+ self.old_objects_pointing_to_young.append(addr_struct)
+ objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
#
# Second part: if 'addr_struct' is actually a prebuilt GC
# object and it's the first time we see a write to it, we
# add it to the list 'prebuilt_root_objects'. Note that we
- # do it even in the (rare?) case of 'addr' being another
+ # do it even in the (rare?) case of 'addr' being NULL or another
# prebuilt object, to simplify code.
if objhdr.tid & GCFLAG_NO_HEAP_PTRS:
objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
@@ -780,16 +789,24 @@
def _init_writebarrier_with_card_marker(self):
+ DEBUG = self.DEBUG
def remember_young_pointer_from_array(addr_array, index):
# 'addr_array' is the address of the object in which we write,
# which must have an array part; 'index' is the index of the
# item that is (or contains) the pointer that we write.
+ if DEBUG:
+ ll_assert(not self.is_in_nursery(addr_array),
+ "nursery array with GCFLAG_NO_YOUNG_PTRS")
objhdr = self.header(addr_array)
if objhdr.tid & GCFLAG_HAS_CARDS == 0:
#
- # no cards, use default logic. The 'nocard_logic()' is just
- # 'remember_young_pointer()', but forced to be inlined here.
- nocard_logic(addr_array)
+ # no cards, use default logic. Mostly copied from above.
+ self.old_objects_pointing_to_young.append(addr_array)
+ objhdr = self.header(addr_array)
+ objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
+ if objhdr.tid & GCFLAG_NO_HEAP_PTRS:
+ objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
+ self.prebuilt_root_objects.append(addr_array)
return
#
# 'addr_array' is a raw_malloc'ed array with card markers
@@ -807,17 +824,15 @@
return
#
# We set the flag (even if the newly written address does not
- # actually point to the nursery -- like remember_young_pointer()).
+ # actually point to the nursery, which seems to be ok -- actually
+ # it seems more important that remember_young_pointer_from_array()
+ # does not take 3 arguments).
addr_byte.char[0] = chr(byte | bitmask)
#
if objhdr.tid & GCFLAG_CARDS_SET == 0:
self.old_objects_with_cards_set.append(addr_array)
objhdr.tid |= GCFLAG_CARDS_SET
- nocard_logic = func_with_new_name(self.remember_young_pointer,
- 'remember_young_pointer_nocard')
- del nocard_logic._dont_inline_
- nocard_logic._always_inline_ = True
remember_young_pointer_from_array._dont_inline_ = True
self.remember_young_pointer_from_array = (
remember_young_pointer_from_array)
Modified: pypy/trunk/pypy/rpython/memory/gc/semispace.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/gc/semispace.py (original)
+++ pypy/trunk/pypy/rpython/memory/gc/semispace.py Fri Oct 1 17:56:02 2010
@@ -1,7 +1,6 @@
from pypy.rpython.lltypesystem.llmemory import raw_malloc, raw_free
from pypy.rpython.lltypesystem.llmemory import raw_memcopy, raw_memclear
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rpython.memory.support import get_address_stack, get_address_deque
from pypy.rpython.memory.support import AddressDict
from pypy.rpython.lltypesystem import lltype, llmemory, llarena, rffi, llgroup
@@ -59,11 +58,11 @@
# translating to a real backend.
TRANSLATION_PARAMS = {'space_size': 8*1024*1024} # XXX adjust
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, space_size=4096,
- max_space_size=sys.maxint//2+1):
+ def __init__(self, config, space_size=4096, max_space_size=sys.maxint//2+1,
+ **kwds):
self.param_space_size = space_size
self.param_max_space_size = max_space_size
- MovingGCBase.__init__(self, config, chunk_size)
+ MovingGCBase.__init__(self, config, **kwds)
def setup(self):
#self.total_collection_time = 0.0
Modified: pypy/trunk/pypy/rpython/memory/gc/test/test_direct.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/gc/test/test_direct.py (original)
+++ pypy/trunk/pypy/rpython/memory/gc/test/test_direct.py Fri Oct 1 17:56:02 2010
@@ -70,6 +70,7 @@
GC_PARAMS = self.GC_PARAMS.copy()
if hasattr(meth, 'GC_PARAMS'):
GC_PARAMS.update(meth.GC_PARAMS)
+ GC_PARAMS['translated_to_c'] = False
self.gc = self.GCClass(config, **GC_PARAMS)
self.gc.DEBUG = True
self.rootwalker = DirectRootWalker(self)
@@ -86,17 +87,19 @@
def write(self, p, fieldname, newvalue):
if self.gc.needs_write_barrier:
+ newaddr = llmemory.cast_ptr_to_adr(newvalue)
addr_struct = llmemory.cast_ptr_to_adr(p)
- self.gc.write_barrier(addr_struct)
+ self.gc.write_barrier(newaddr, addr_struct)
setattr(p, fieldname, newvalue)
def writearray(self, p, index, newvalue):
if self.gc.needs_write_barrier:
+ newaddr = llmemory.cast_ptr_to_adr(newvalue)
addr_struct = llmemory.cast_ptr_to_adr(p)
if hasattr(self.gc, 'write_barrier_from_array'):
- self.gc.write_barrier_from_array(addr_struct, index)
+ self.gc.write_barrier_from_array(newaddr, addr_struct, index)
else:
- self.gc.write_barrier(addr_struct)
+ self.gc.write_barrier(newaddr, addr_struct)
p[index] = newvalue
def malloc(self, TYPE, n=None):
Modified: pypy/trunk/pypy/rpython/memory/gctransform/framework.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/gctransform/framework.py (original)
+++ pypy/trunk/pypy/rpython/memory/gctransform/framework.py Fri Oct 1 17:56:02 2010
@@ -426,6 +426,7 @@
if GCClass.needs_write_barrier:
self.write_barrier_ptr = getfn(GCClass.write_barrier.im_func,
[s_gc,
+ annmodel.SomeAddress(),
annmodel.SomeAddress()],
annmodel.s_None,
inline=True)
@@ -434,13 +435,15 @@
# func should not be a bound method, but a real function
assert isinstance(func, types.FunctionType)
self.write_barrier_failing_case_ptr = getfn(func,
- [annmodel.SomeAddress()],
+ [annmodel.SomeAddress(),
+ annmodel.SomeAddress()],
annmodel.s_None)
func = getattr(GCClass, 'write_barrier_from_array', None)
if func is not None:
self.write_barrier_from_array_ptr = getfn(func.im_func,
[s_gc,
annmodel.SomeAddress(),
+ annmodel.SomeAddress(),
annmodel.SomeInteger()],
annmodel.s_None,
inline=True)
@@ -1021,6 +1024,8 @@
and not isinstance(v_newvalue, Constant)
and v_struct.concretetype.TO._gckind == "gc"
and hop.spaceop not in self.clean_sets):
+ v_newvalue = hop.genop("cast_ptr_to_adr", [v_newvalue],
+ resulttype = llmemory.Address)
v_structaddr = hop.genop("cast_ptr_to_adr", [v_struct],
resulttype = llmemory.Address)
if (self.write_barrier_from_array_ptr is not None and
@@ -1030,12 +1035,14 @@
assert v_index.concretetype == lltype.Signed
hop.genop("direct_call", [self.write_barrier_from_array_ptr,
self.c_const_gc,
+ v_newvalue,
v_structaddr,
v_index])
else:
self.write_barrier_calls += 1
hop.genop("direct_call", [self.write_barrier_ptr,
self.c_const_gc,
+ v_newvalue,
v_structaddr])
hop.rename('bare_' + opname)
Modified: pypy/trunk/pypy/rpython/memory/gcwrapper.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/gcwrapper.py (original)
+++ pypy/trunk/pypy/rpython/memory/gcwrapper.py Fri Oct 1 17:56:02 2010
@@ -9,7 +9,10 @@
def __init__(self, llinterp, flowgraphs, gc_class, GC_PARAMS={}):
translator = llinterp.typer.annotator.translator
config = translator.config.translation
- self.gc = gc_class(config, chunk_size = 10, **GC_PARAMS)
+ self.gc = gc_class(config,
+ chunk_size = 10,
+ translated_to_c = False,
+ **GC_PARAMS)
self.gc.set_root_walker(LLInterpRootWalker(self))
self.gc.DEBUG = True
self.llinterp = llinterp
@@ -94,6 +97,7 @@
assert (type(index) is int # <- fast path
or lltype.typeOf(index) == lltype.Signed)
self.gc.write_barrier_from_array(
+ llmemory.cast_ptr_to_adr(newvalue),
llmemory.cast_ptr_to_adr(toplevelcontainer),
index)
wb = False
@@ -101,6 +105,7 @@
#
if wb:
self.gc.write_barrier(
+ llmemory.cast_ptr_to_adr(newvalue),
llmemory.cast_ptr_to_adr(toplevelcontainer))
llheap.setinterior(toplevelcontainer, inneraddr, INNERTYPE, newvalue)
Modified: pypy/trunk/pypy/rpython/memory/test/test_transformed_gc.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/test/test_transformed_gc.py (original)
+++ pypy/trunk/pypy/rpython/memory/test/test_transformed_gc.py Fri Oct 1 17:56:02 2010
@@ -906,7 +906,8 @@
gcname = "marksweep"
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
- GC_PARAMS = {'start_heap_size': 1024*WORD }
+ GC_PARAMS = {'start_heap_size': 1024*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
@@ -1144,7 +1145,8 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.marksweep import PrintingMarkSweepGC as GCClass
- GC_PARAMS = {'start_heap_size': 1024*WORD }
+ GC_PARAMS = {'start_heap_size': 1024*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestSemiSpaceGC(GenericMovingGCTests):
@@ -1154,7 +1156,8 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.semispace import SemiSpaceGC as GCClass
- GC_PARAMS = {'space_size': 512*WORD}
+ GC_PARAMS = {'space_size': 512*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestMarkCompactGC(GenericMovingGCTests):
@@ -1163,7 +1166,8 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.markcompact import MarkCompactGC as GCClass
- GC_PARAMS = {'space_size': 4096*WORD}
+ GC_PARAMS = {'space_size': 4096*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestGenerationGC(GenericMovingGCTests):
@@ -1175,7 +1179,8 @@
from pypy.rpython.memory.gc.generation import GenerationGC as \
GCClass
GC_PARAMS = {'space_size': 512*WORD,
- 'nursery_size': 32*WORD}
+ 'nursery_size': 32*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
def define_weakref_across_minor_collection(cls):
@@ -1372,7 +1377,8 @@
GenerationGC._teardown(self)
GC_PARAMS = {'space_size': 512*WORD,
- 'nursery_size': 128*WORD}
+ 'nursery_size': 128*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
def define_working_nursery(cls):
@@ -1404,7 +1410,8 @@
from pypy.rpython.memory.gc.hybrid import HybridGC as GCClass
GC_PARAMS = {'space_size': 512*WORD,
'nursery_size': 32*WORD,
- 'large_object': 8*WORD}
+ 'large_object': 8*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
def define_ref_from_rawmalloced_to_regular(cls):
@@ -1477,6 +1484,7 @@
'large_object': 8*WORD,
'large_object_gcptrs': 10*WORD,
'card_page_indices': 4,
+ 'translated_to_c': False,
}
root_stack_depth = 200
@@ -1585,7 +1593,8 @@
gcname = "marksweep"
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
- GC_PARAMS = {'start_heap_size': 1024*WORD }
+ GC_PARAMS = {'start_heap_size': 1024*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestHybridTaggedPointerGC(TaggedPointerGCTests):
@@ -1596,7 +1605,8 @@
from pypy.rpython.memory.gc.generation import GenerationGC as \
GCClass
GC_PARAMS = {'space_size': 512*WORD,
- 'nursery_size': 32*WORD}
+ 'nursery_size': 32*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestMarkCompactTaggedpointerGC(TaggedPointerGCTests):
@@ -1605,5 +1615,6 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.markcompact import MarkCompactGC as GCClass
- GC_PARAMS = {'space_size': 4096*WORD}
+ GC_PARAMS = {'space_size': 4096*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
From afa at codespeak.net Fri Oct 1 18:26:13 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 1 Oct 2010 18:26:13 +0200 (CEST)
Subject: [pypy-svn] r77538 - in pypy/branch/fast-forward/pypy: interpreter
interpreter/test objspace objspace/std
Message-ID: <20101001162613.809B6282BDB@codespeak.net>
Author: afa
Date: Fri Oct 1 18:26:11 2010
New Revision: 77538
Modified:
pypy/branch/fast-forward/pypy/interpreter/test/test_typedef.py
pypy/branch/fast-forward/pypy/interpreter/typedef.py
pypy/branch/fast-forward/pypy/objspace/descroperation.py
pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py
pypy/branch/fast-forward/pypy/objspace/std/dicttype.py
pypy/branch/fast-forward/pypy/objspace/std/listtype.py
pypy/branch/fast-forward/pypy/objspace/std/objecttype.py
pypy/branch/fast-forward/pypy/objspace/std/settype.py
pypy/branch/fast-forward/pypy/objspace/std/slicetype.py
pypy/branch/fast-forward/pypy/objspace/std/stdtypedef.py
Log:
Set list.__hash__ is now None, instead of a function that raises TypeError.
no_hash_descr() is gone!
This fixes
isinstance([], _abcoll.Hashable)
Modified: pypy/branch/fast-forward/pypy/interpreter/test/test_typedef.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/test/test_typedef.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/test/test_typedef.py Fri Oct 1 18:26:11 2010
@@ -145,6 +145,19 @@
w_obj = self.space.wrap(W_SomeType())
assert self.space.getattr(w_obj, self.space.wrap('x')) is self.space.w_None
+ def test_unhashable(self):
+ class W_SomeType(Wrappable):
+ pass
+ W_SomeType.typedef = typedef.TypeDef(
+ 'some_type',
+ __hash__ = None)
+ w_obj = self.space.wrap(W_SomeType())
+ self.space.appexec([w_obj], """(obj):
+ assert type(obj).__hash__ is None
+ err = raises(TypeError, hash, obj)
+ assert err.value.message == "'some_type' objects are unhashable"
+ """)
+
class AppTestTypeDef:
Modified: pypy/branch/fast-forward/pypy/interpreter/typedef.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/typedef.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/typedef.py Fri Oct 1 18:26:11 2010
@@ -51,13 +51,6 @@
def default_identity_hash(space, w_obj):
return space.wrap(compute_identity_hash(w_obj))
-def descr__hash__unhashable(space, w_obj):
- typename = space.type(w_obj).getname(space, '?')
- raise operationerrfmt(space.w_TypeError,
- "'%s' objects are unhashable", typename)
-
-no_hash_descr = interp2app(descr__hash__unhashable)
-
# ____________________________________________________________
#
# For each built-in app-level type Xxx that can be subclassed at
@@ -885,7 +878,7 @@
__eq__ = interp2app(Cell.descr__eq__,
unwrap_spec=['self', ObjSpace, W_Root]),
__ne__ = descr_generic_ne,
- __hash__ = no_hash_descr,
+ __hash__ = None,
__reduce__ = interp2app(Cell.descr__reduce__,
unwrap_spec=['self', ObjSpace]),
__setstate__ = interp2app(Cell.descr__setstate__,
Modified: pypy/branch/fast-forward/pypy/objspace/descroperation.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/descroperation.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/descroperation.py Fri Oct 1 18:26:11 2010
@@ -392,6 +392,10 @@
# default __hash__. This path should only be taken under very
# obscure circumstances.
return default_identity_hash(space, w_obj)
+ if space.is_w(w_hash, space.w_None):
+ typename = space.type(w_obj).getname(space, '?')
+ raise operationerrfmt(space.w_TypeError,
+ "'%s' objects are unhashable", typename)
# XXX CPython has a special case for types with "__hash__ = None"
# to produce a nicer error message, namely "unhashable type: 'X'".
w_result = space.get_and_call_function(w_hash, w_obj)
Modified: pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py Fri Oct 1 18:26:11 2010
@@ -2,7 +2,7 @@
from pypy.interpreter import gateway
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
from pypy.objspace.std.register_all import register_all
-from pypy.objspace.std.stdtypedef import StdTypeDef, SMM, no_hash_descr
+from pypy.objspace.std.stdtypedef import StdTypeDef, SMM
from pypy.objspace.std.stringtype import (
str_count, str_index, str_rindex, str_find, str_rfind, str_replace,
@@ -31,6 +31,6 @@
If the argument is a bytearray, the return value is the same object.''',
__new__ = gateway.interp2app(descr__new__),
- __hash__ = no_hash_descr,
+ __hash__ = None,
)
bytearray_typedef.registermethods(globals())
Modified: pypy/branch/fast-forward/pypy/objspace/std/dicttype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/dicttype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/dicttype.py Fri Oct 1 18:26:11 2010
@@ -2,7 +2,7 @@
from pypy.interpreter.error import OperationError
from pypy.interpreter.mixedmodule import MixedModule
from pypy.interpreter import gateway
-from pypy.objspace.std.stdtypedef import StdTypeDef, SMM, no_hash_descr
+from pypy.objspace.std.stdtypedef import StdTypeDef, SMM
from pypy.objspace.std.register_all import register_all
dict_copy = SMM('copy', 1,
@@ -179,7 +179,7 @@
unwrap_spec=
[gateway.ObjSpace,
gateway.W_Root,gateway.Arguments]),
- __hash__ = no_hash_descr,
+ __hash__ = None,
fromkeys = gateway.interp2app(descr_fromkeys, as_classmethod=True),
)
dict_typedef.registermethods(globals())
Modified: pypy/branch/fast-forward/pypy/objspace/std/listtype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/listtype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/listtype.py Fri Oct 1 18:26:11 2010
@@ -1,6 +1,6 @@
from pypy.interpreter import gateway
from pypy.interpreter.error import OperationError
-from pypy.objspace.std.stdtypedef import StdTypeDef, SMM, no_hash_descr
+from pypy.objspace.std.stdtypedef import StdTypeDef, SMM
from pypy.objspace.std.register_all import register_all
from sys import maxint
@@ -54,7 +54,7 @@
__new__ = gateway.interp2app(descr__new__, unwrap_spec=[gateway.ObjSpace,
gateway.W_Root,
gateway.Arguments]),
- __hash__ = no_hash_descr,
+ __hash__ = None,
)
list_typedef.registermethods(globals())
Modified: pypy/branch/fast-forward/pypy/objspace/std/objecttype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/objecttype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/objecttype.py Fri Oct 1 18:26:11 2010
@@ -4,7 +4,7 @@
from pypy.interpreter.argument import Arguments
from pypy.interpreter.baseobjspace import ObjSpace
from pypy.objspace.descroperation import Object
-from pypy.objspace.std.stdtypedef import StdTypeDef, no_hash_descr
+from pypy.objspace.std.stdtypedef import StdTypeDef
from pypy.objspace.std.register_all import register_all
Modified: pypy/branch/fast-forward/pypy/objspace/std/settype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/settype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/settype.py Fri Oct 1 18:26:11 2010
@@ -1,7 +1,7 @@
from pypy.interpreter.error import OperationError
from pypy.interpreter import gateway
from pypy.objspace.std.register_all import register_all
-from pypy.objspace.std.stdtypedef import StdTypeDef, no_hash_descr, SMM
+from pypy.objspace.std.stdtypedef import StdTypeDef, SMM
set_add = SMM('add', 2,
doc='Add an element to a set.\n\nThis'
@@ -78,7 +78,7 @@
__new__ = gateway.interp2app(descr__new__, unwrap_spec=[gateway.ObjSpace,
gateway.W_Root,
gateway.Arguments]),
- __hash__ = no_hash_descr,
+ __hash__ = None,
)
set_typedef.registermethods(globals())
Modified: pypy/branch/fast-forward/pypy/objspace/std/slicetype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/slicetype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/slicetype.py Fri Oct 1 18:26:11 2010
@@ -1,6 +1,6 @@
from pypy.interpreter import baseobjspace, gateway
from pypy.interpreter.typedef import GetSetProperty
-from pypy.objspace.std.stdtypedef import StdTypeDef, SMM, no_hash_descr
+from pypy.objspace.std.stdtypedef import StdTypeDef, SMM
from pypy.objspace.std.register_all import register_all
from pypy.interpreter.error import OperationError
@@ -89,7 +89,7 @@
Create a slice object. This is used for extended slicing (e.g. a[0:10:2]).''',
__new__ = gateway.interp2app(descr__new__),
- __hash__ = no_hash_descr,
+ __hash__ = None,
start = slicewprop('w_start'),
stop = slicewprop('w_stop'),
step = slicewprop('w_step'),
Modified: pypy/branch/fast-forward/pypy/objspace/std/stdtypedef.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/stdtypedef.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/stdtypedef.py Fri Oct 1 18:26:11 2010
@@ -2,7 +2,7 @@
from pypy.interpreter.error import OperationError, operationerrfmt
from pypy.interpreter.typedef import TypeDef, GetSetProperty, Member
from pypy.interpreter.typedef import descr_get_dict, descr_set_dict
-from pypy.interpreter.typedef import no_hash_descr, descr_del_dict
+from pypy.interpreter.typedef import descr_del_dict
from pypy.interpreter.baseobjspace import SpaceCache
from pypy.objspace.std import model
from pypy.objspace.std.model import StdObjSpaceMultiMethod
@@ -10,7 +10,7 @@
from pypy.rlib import jit
from pypy.tool.sourcetools import compile2
-__all__ = ['StdTypeDef', 'SMM', 'no_hash_descr']
+__all__ = ['StdTypeDef', 'SMM']
SMM = StdObjSpaceMultiMethod
From afa at codespeak.net Fri Oct 1 18:38:41 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 1 Oct 2010 18:38:41 +0200 (CEST)
Subject: [pypy-svn] r77539 - pypy/branch/fast-forward/pypy/objspace
Message-ID: <20101001163841.718ED282BDB@codespeak.net>
Author: afa
Date: Fri Oct 1 18:38:39 2010
New Revision: 77539
Modified:
pypy/branch/fast-forward/pypy/objspace/descroperation.py
Log:
Kill a XXX fixed by the last change
Modified: pypy/branch/fast-forward/pypy/objspace/descroperation.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/descroperation.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/descroperation.py Fri Oct 1 18:38:39 2010
@@ -396,8 +396,6 @@
typename = space.type(w_obj).getname(space, '?')
raise operationerrfmt(space.w_TypeError,
"'%s' objects are unhashable", typename)
- # XXX CPython has a special case for types with "__hash__ = None"
- # to produce a nicer error message, namely "unhashable type: 'X'".
w_result = space.get_and_call_function(w_hash, w_obj)
w_resulttype = space.type(w_result)
if space.is_w(w_resulttype, space.w_int):
From hakanardo at codespeak.net Fri Oct 1 19:36:21 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Fri, 1 Oct 2010 19:36:21 +0200 (CEST)
Subject: [pypy-svn] r77540 - in pypy/branch/jit-unroll-loops/pypy:
jit/metainterp jit/metainterp/optimizeopt jit/metainterp/test rlib
Message-ID: <20101001173621.AF05C282BDB@codespeak.net>
Author: hakanardo
Date: Fri Oct 1 19:36:19 2010
New Revision: 77540
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py
pypy/branch/jit-unroll-loops/pypy/rlib/jit.py
Log:
guards working, test_loop_invariant_mul1 passing
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Fri Oct 1 19:36:19 2010
@@ -14,15 +14,14 @@
if not self.enabled:
self.emit_operation(op)
return
-
+
if op.getopnum() == rop.JUMP:
loop = self.optimizer.loop
loop.preamble.operations = self.optimizer.newoperations
self.optimizer.newoperations = []
jump_args = op.getarglist()
inputargs = self.inline(loop.preamble.operations + [op],
- loop.inputargs, jump_args)
- print "IN: ", inputargs
+ loop.inputargs, jump_args)
loop.inputargs = inputargs
jmp = ResOperation(rop.JUMP, loop.inputargs[:], None)
jmp.setdescr(loop.token)
@@ -36,16 +35,11 @@
for i in range(len(loop_args)):
argmap[loop_args[i]] = jump_args[i]
- print
- print
- print argmap
-
for v in self.optimizer.values.values():
v.fromstart = True
inputargs = jump_args[:]
for op in loop_operations:
- print "I:", op
newop = op.clone()
for i in range(op.numargs()):
a = op.getarg(i)
@@ -63,18 +57,25 @@
args = newop.getarglist()
newop.initarglist(args + inputargs[len(args):])
# FIXME: Assumes no virtuals
- print "N:", newop
current = len(self.optimizer.newoperations)
self.emit_operation(newop)
+
for op in self.optimizer.newoperations[current:]:
- print "E:", op
+ if op.is_guard():
+ args = []
+ for a in op._fail_args:
+ try:
+ a = argmap[a]
+ except KeyError:
+ pass
+ args.append(a)
+ op.setfailargs(args)
+
for a in op.getarglist():
if not isinstance(a, Const) and a in self.optimizer.values:
v = self.getvalue(a)
- print " testing ", a
if v.fromstart and a not in inputargs:
- print " ", a
inputargs.append(a)
return inputargs
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py Fri Oct 1 19:36:19 2010
@@ -298,7 +298,7 @@
self.liveboxes_from_env = liveboxes_from_env
self.liveboxes = {}
storage.rd_numb = numb
- storage.rd_snapshot = None
+ #storage.rd_snapshot = None FIXME!!
# collect liveboxes and virtuals
n = len(liveboxes_from_env) - v
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py Fri Oct 1 19:36:19 2010
@@ -316,8 +316,8 @@
res = self.meta_interp(f, [6, 7])
assert res == 252
self.check_loop_count(2)
- self.check_loops({'guard_true': 1,
- 'int_add': 1, 'int_sub': 1, 'int_gt': 1,
+ self.check_loops({'guard_true': 2,
+ 'int_add': 2, 'int_sub': 2, 'int_gt': 2,
'int_mul': 1,
'jump': 2})
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/jit.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/jit.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/jit.py Fri Oct 1 19:36:19 2010
@@ -236,7 +236,8 @@
'trace_eagerness': 200,
'trace_limit': 10000,
'inlining': False,
- 'optimizer': OPTIMIZER_FULL,
+ #'optimizer': OPTIMIZER_FULL,
+ 'optimizer': OPTIMIZER_NO_PERFECTSPEC,
'debug' : DEBUG_STEPS,
}
unroll_parameters = unrolling_iterable(PARAMETERS.keys())
From afa at codespeak.net Fri Oct 1 22:45:28 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 1 Oct 2010 22:45:28 +0200 (CEST)
Subject: [pypy-svn] r77542 - in pypy/branch/fast-forward/pypy/interpreter: .
test
Message-ID: <20101001204528.99700282BDB@codespeak.net>
Author: afa
Date: Fri Oct 1 22:45:27 2010
New Revision: 77542
Modified:
pypy/branch/fast-forward/pypy/interpreter/argument.py
pypy/branch/fast-forward/pypy/interpreter/test/test_function.py
Log:
**kwargs now accepts any mapping
Modified: pypy/branch/fast-forward/pypy/interpreter/argument.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/argument.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/argument.py Fri Oct 1 22:45:27 2010
@@ -149,14 +149,25 @@
# unpack the ** arguments
if w_starstararg is not None:
space = self.space
- if not space.is_true(space.isinstance(w_starstararg, space.w_dict)):
- raise OperationError(space.w_TypeError,
- space.wrap("argument after ** must be "
- "a dictionary"))
- keywords_w = [None] * space.int_w(space.len(w_starstararg))
- keywords = [None] * space.int_w(space.len(w_starstararg))
+ if space.isinstance_w(w_starstararg, space.w_dict):
+ keys_w = space.unpackiterable(w_starstararg)
+ else:
+ try:
+ w_keys = space.call_method(w_starstararg, "keys")
+ except OperationError, e:
+ if e.match(space, space.w_AttributeError):
+ w_type = space.type(w_starstararg)
+ typename = w_type.getname(space, '?')
+ raise OperationError(
+ space.w_TypeError,
+ space.wrap("argument after ** must be "
+ "a mapping, not %s" % (typename,)))
+ raise
+ keys_w = space.unpackiterable(w_keys)
+ keywords_w = [None] * len(keys_w)
+ keywords = [None] * len(keys_w)
i = 0
- for w_key in space.unpackiterable(w_starstararg):
+ for w_key in keys_w:
try:
key = space.str_w(w_key)
except OperationError, e:
Modified: pypy/branch/fast-forward/pypy/interpreter/test/test_function.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/test/test_function.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/test/test_function.py Fri Oct 1 22:45:27 2010
@@ -164,6 +164,21 @@
raises(
TypeError, func, 42, {'arg1': 23})
+ def test_kwargs_nondict_mapping(self):
+ class Mapping:
+ def keys(self):
+ return ('a', 'b')
+ def __getitem__(self, key):
+ return key
+ def func(arg1, **kw):
+ return arg1, kw
+ res = func(23, **Mapping())
+ assert res[0] == 23
+ assert res[1] == {'a': 'a', 'b': 'b'}
+ error = raises(TypeError, lambda: func(42, **[]))
+ assert error.value.message == ('argument after ** must be a mapping, '
+ 'not list')
+
def test_default_arg(self):
def func(arg1,arg2=42):
return arg1, arg2
From afa at codespeak.net Fri Oct 1 22:53:06 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 1 Oct 2010 22:53:06 +0200 (CEST)
Subject: [pypy-svn] r77543 - in pypy/branch/fast-forward: . lib_pypy/_ctypes
pypy/jit/backend pypy/jit/backend/llgraph
pypy/jit/backend/llsupport pypy/jit/backend/llsupport/test
pypy/jit/backend/test pypy/jit/backend/x86
pypy/jit/metainterp pypy/jit/tl pypy/module/imp
pypy/module/imp/test pypy/module/test_lib_pypy/ctypes_tests
pypy/rpython/lltypesystem pypy/rpython/memory
pypy/rpython/memory/gc pypy/rpython/memory/gc/test
pypy/rpython/memory/gctransform pypy/rpython/memory/test
pypy/translator pypy/translator/goal
Message-ID: <20101001205306.1F56A282BDB@codespeak.net>
Author: afa
Date: Fri Oct 1 22:53:02 2010
New Revision: 77543
Added:
pypy/branch/fast-forward/pypy/jit/backend/conftest.py
- copied unchanged from r77542, pypy/trunk/pypy/jit/backend/conftest.py
pypy/branch/fast-forward/pypy/jit/tl/jittest.py
- copied unchanged from r77542, pypy/trunk/pypy/jit/tl/jittest.py
Removed:
pypy/branch/fast-forward/pypy/jit/backend/test/conftest.py
Modified:
pypy/branch/fast-forward/ (props changed)
pypy/branch/fast-forward/lib_pypy/_ctypes/function.py
pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py
pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py
pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_gc.py
pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py
pypy/branch/fast-forward/pypy/jit/backend/test/test_ll_random.py
pypy/branch/fast-forward/pypy/jit/backend/test/test_random.py
pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py
pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py
pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py
pypy/branch/fast-forward/pypy/jit/metainterp/executor.py
pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py
pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py
pypy/branch/fast-forward/pypy/module/imp/importing.py
pypy/branch/fast-forward/pypy/module/imp/test/test_import.py
pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py
pypy/branch/fast-forward/pypy/rpython/memory/gc/base.py
pypy/branch/fast-forward/pypy/rpython/memory/gc/generation.py
pypy/branch/fast-forward/pypy/rpython/memory/gc/markcompact.py
pypy/branch/fast-forward/pypy/rpython/memory/gc/marksweep.py
pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py
pypy/branch/fast-forward/pypy/rpython/memory/gc/minimarkpage.py
pypy/branch/fast-forward/pypy/rpython/memory/gc/semispace.py
pypy/branch/fast-forward/pypy/rpython/memory/gc/test/test_direct.py
pypy/branch/fast-forward/pypy/rpython/memory/gc/test/test_minimarkpage.py
pypy/branch/fast-forward/pypy/rpython/memory/gctransform/framework.py
pypy/branch/fast-forward/pypy/rpython/memory/gcwrapper.py
pypy/branch/fast-forward/pypy/rpython/memory/test/test_transformed_gc.py
pypy/branch/fast-forward/pypy/translator/driver.py
pypy/branch/fast-forward/pypy/translator/goal/translate.py
Log:
Merge from trunk
Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/function.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/_ctypes/function.py (original)
+++ pypy/branch/fast-forward/lib_pypy/_ctypes/function.py Fri Oct 1 22:53:02 2010
@@ -171,7 +171,7 @@
return self._build_result(restype, resbuffer, argtypes, argsandobjs)
def _getfuncptr(self, argtypes, restype, thisarg=None):
- if self._ptr is not None:
+ if self._ptr is not None and argtypes is self._argtypes_:
return self._ptr
if restype is None or not isinstance(restype, _CDataMeta):
import ctypes
Modified: pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py Fri Oct 1 22:53:02 2010
@@ -129,7 +129,7 @@
'arraylen_gc' : (('ref',), 'int'),
'call' : (('ref', 'varargs'), 'intorptr'),
'call_assembler' : (('varargs',), 'intorptr'),
- 'cond_call_gc_wb' : (('ptr',), None),
+ 'cond_call_gc_wb' : (('ptr', 'ptr'), None),
'oosend' : (('varargs',), 'intorptr'),
'oosend_pure' : (('varargs',), 'intorptr'),
'guard_true' : (('bool',), None),
@@ -810,7 +810,7 @@
FLOAT: 0.0}
return d[calldescr.typeinfo]
- def op_cond_call_gc_wb(self, descr, a):
+ def op_cond_call_gc_wb(self, descr, a, b):
py.test.skip("cond_call_gc_wb not supported")
def op_oosend(self, descr, obj, *args):
Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py Fri Oct 1 22:53:02 2010
@@ -404,7 +404,7 @@
self.GC_MALLOC_BASIC = lltype.Ptr(lltype.FuncType(
[lltype.Signed, lltype.Signed], llmemory.GCREF))
self.WB_FUNCPTR = lltype.Ptr(lltype.FuncType(
- [llmemory.Address], lltype.Void))
+ [llmemory.Address, llmemory.Address], lltype.Void))
self.write_barrier_descr = WriteBarrierDescr(self)
#
def malloc_array(itemsize, tid, num_elem):
@@ -550,7 +550,8 @@
# the GC, and call it immediately
llop1 = self.llop1
funcptr = llop1.get_write_barrier_failing_case(self.WB_FUNCPTR)
- funcptr(llmemory.cast_ptr_to_adr(gcref_struct))
+ funcptr(llmemory.cast_ptr_to_adr(gcref_struct),
+ llmemory.cast_ptr_to_adr(gcref_newptr))
def rewrite_assembler(self, cpu, operations):
# Perform two kinds of rewrites in parallel:
@@ -589,7 +590,7 @@
v = op.getarg(1)
if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and
bool(v.value)): # store a non-NULL
- self._gen_write_barrier(newops, op.getarg(0))
+ self._gen_write_barrier(newops, op.getarg(0), v)
op = op.copy_and_change(rop.SETFIELD_RAW)
# ---------- write barrier for SETARRAYITEM_GC ----------
if op.getopnum() == rop.SETARRAYITEM_GC:
@@ -598,15 +599,15 @@
bool(v.value)): # store a non-NULL
# XXX detect when we should produce a
# write_barrier_from_array
- self._gen_write_barrier(newops, op.getarg(0))
+ self._gen_write_barrier(newops, op.getarg(0), v)
op = op.copy_and_change(rop.SETARRAYITEM_RAW)
# ----------
newops.append(op)
del operations[:]
operations.extend(newops)
- def _gen_write_barrier(self, newops, v_base):
- args = [v_base]
+ def _gen_write_barrier(self, newops, v_base, v_value):
+ args = [v_base, v_value]
newops.append(ResOperation(rop.COND_CALL_GC_WB, args, None,
descr=self.write_barrier_descr))
Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_gc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_gc.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_gc.py Fri Oct 1 22:53:02 2010
@@ -141,8 +141,8 @@
repr(offset_to_length), p))
return p
- def _write_barrier_failing_case(self, adr_struct):
- self.record.append(('barrier', adr_struct))
+ def _write_barrier_failing_case(self, adr_struct, adr_newptr):
+ self.record.append(('barrier', adr_struct, adr_newptr))
def get_write_barrier_failing_case(self, FPTRTYPE):
return llhelper(FPTRTYPE, self._write_barrier_failing_case)
@@ -239,6 +239,7 @@
s_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
r_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, r)
s_adr = llmemory.cast_ptr_to_adr(s)
+ r_adr = llmemory.cast_ptr_to_adr(r)
#
s_hdr.tid &= ~gc_ll_descr.GCClass.JIT_WB_IF_FLAG
gc_ll_descr.do_write_barrier(s_gcref, r_gcref)
@@ -246,7 +247,7 @@
#
s_hdr.tid |= gc_ll_descr.GCClass.JIT_WB_IF_FLAG
gc_ll_descr.do_write_barrier(s_gcref, r_gcref)
- assert self.llop1.record == [('barrier', s_adr)]
+ assert self.llop1.record == [('barrier', s_adr, r_adr)]
def test_gen_write_barrier(self):
gc_ll_descr = self.gc_ll_descr
@@ -254,11 +255,13 @@
#
newops = []
v_base = BoxPtr()
- gc_ll_descr._gen_write_barrier(newops, v_base)
+ v_value = BoxPtr()
+ gc_ll_descr._gen_write_barrier(newops, v_base, v_value)
assert llop1.record == []
assert len(newops) == 1
assert newops[0].getopnum() == rop.COND_CALL_GC_WB
assert newops[0].getarg(0) == v_base
+ assert newops[0].getarg(1) == v_value
assert newops[0].result is None
wbdescr = newops[0].getdescr()
assert isinstance(wbdescr.jit_wb_if_flag, int)
@@ -358,6 +361,7 @@
#
assert operations[0].getopnum() == rop.COND_CALL_GC_WB
assert operations[0].getarg(0) == v_base
+ assert operations[0].getarg(1) == v_value
assert operations[0].result is None
#
assert operations[1].getopnum() == rop.SETFIELD_RAW
@@ -381,6 +385,7 @@
#
assert operations[0].getopnum() == rop.COND_CALL_GC_WB
assert operations[0].getarg(0) == v_base
+ assert operations[0].getarg(1) == v_value
assert operations[0].result is None
#
assert operations[1].getopnum() == rop.SETARRAYITEM_RAW
Modified: pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py Fri Oct 1 22:53:02 2010
@@ -1427,12 +1427,12 @@
assert not excvalue
def test_cond_call_gc_wb(self):
- def func_void(a):
- record.append(a)
+ def func_void(a, b):
+ record.append((a, b))
record = []
#
S = lltype.GcStruct('S', ('tid', lltype.Signed))
- FUNC = self.FuncType([lltype.Ptr(S)], lltype.Void)
+ FUNC = self.FuncType([lltype.Ptr(S), lltype.Signed], lltype.Void)
func_ptr = llhelper(lltype.Ptr(FUNC), func_void)
funcbox = self.get_funcbox(self.cpu, func_ptr)
class WriteBarrierDescr(AbstractDescr):
@@ -1453,10 +1453,10 @@
sgcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
del record[:]
self.execute_operation(rop.COND_CALL_GC_WB,
- [BoxPtr(sgcref)],
+ [BoxPtr(sgcref), ConstInt(-2121)],
'void', descr=WriteBarrierDescr())
if cond:
- assert record == [s]
+ assert record == [(s, -2121)]
else:
assert record == []
Modified: pypy/branch/fast-forward/pypy/jit/backend/test/test_ll_random.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/test/test_ll_random.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/test/test_ll_random.py Fri Oct 1 22:53:02 2010
@@ -386,6 +386,20 @@
v_string = self.get_string(builder, r)
builder.do(self.opnum, [v_string])
+class AbstractCopyContentOperation(AbstractStringOperation):
+ def produce_into(self, builder, r):
+ v_srcstring = self.get_string(builder, r)
+ v_dststring = self.get_string(builder, r)
+ if v_srcstring.value == v_dststring.value: # because it's not a
+ raise test_random.CannotProduceOperation # memmove(), but memcpy()
+ srclen = len(v_srcstring.getref(self.ptr).chars)
+ dstlen = len(v_dststring.getref(self.ptr).chars)
+ v_length = builder.get_index(min(srclen, dstlen), r)
+ v_srcstart = builder.get_index(srclen - v_length.value + 1, r)
+ v_dststart = builder.get_index(dstlen - v_length.value + 1, r)
+ builder.do(self.opnum, [v_srcstring, v_dststring,
+ v_srcstart, v_dststart, v_length])
+
class StrGetItemOperation(AbstractGetItemOperation, _StrOperation):
pass
@@ -404,6 +418,13 @@
class UnicodeLenOperation(AbstractStringLenOperation, _UnicodeOperation):
pass
+class CopyStrContentOperation(AbstractCopyContentOperation, _StrOperation):
+ pass
+
+class CopyUnicodeContentOperation(AbstractCopyContentOperation,
+ _UnicodeOperation):
+ pass
+
# there are five options in total:
# 1. non raising call and guard_no_exception
@@ -577,6 +598,8 @@
OPERATIONS.append(UnicodeSetItemOperation(rop.UNICODESETITEM))
OPERATIONS.append(StrLenOperation(rop.STRLEN))
OPERATIONS.append(UnicodeLenOperation(rop.UNICODELEN))
+ OPERATIONS.append(CopyStrContentOperation(rop.COPYSTRCONTENT))
+ #OPERATIONS.append(CopyUnicodeContentOperation(rop.COPYUNICODECONTENT))
for i in range(2):
OPERATIONS.append(GuardClassOperation(rop.GUARD_CLASS))
Modified: pypy/branch/fast-forward/pypy/jit/backend/test/test_random.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/test/test_random.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/test/test_random.py Fri Oct 1 22:53:02 2010
@@ -1,7 +1,7 @@
import py, sys
from pypy.rlib.rarithmetic import intmask, LONG_BIT
from pypy.rpython.lltypesystem import llmemory
-from pypy.jit.backend.test import conftest as demo_conftest
+from pypy.jit.backend import conftest as demo_conftest
from pypy.jit.metainterp.history import BasicFailDescr, TreeLoop
from pypy.jit.metainterp.history import BoxInt, ConstInt, LoopToken
from pypy.jit.metainterp.history import BoxPtr, ConstPtr
@@ -102,7 +102,7 @@
elif isinstance(v, ConstFloat):
args.append('ConstFloat(%r)' % v.value)
elif isinstance(v, ConstInt):
- args.append('ConstInt(%d)' % v.value)
+ args.append('ConstInt(%s)' % v.value)
else:
raise NotImplementedError(v)
if op.getdescr() is None:
@@ -113,7 +113,7 @@
except AttributeError:
descrstr = ', descr=...'
print >>s, ' ResOperation(rop.%s, [%s], %s%s),' % (
- opname[op.opnum], ', '.join(args), names[op.result], descrstr)
+ opname[op.getopnum()], ', '.join(args), names[op.result], descrstr)
#if getattr(op, 'suboperations', None) is not None:
# subops.append(op)
@@ -189,7 +189,7 @@
v.value)
print >>s, ' op = cpu.execute_token(looptoken)'
if self.should_fail_by is None:
- fail_args = self.loop.operations[-1].args
+ fail_args = self.loop.operations[-1].getarglist()
else:
fail_args = self.should_fail_by.getfailargs()
for i, v in enumerate(fail_args):
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py Fri Oct 1 22:53:02 2010
@@ -1780,11 +1780,12 @@
self.mc.PUSH_i32(loc.getint())
if IS_X86_64:
- # We clobber this register to pass the arguments, but that's
+ # We clobber these registers to pass the arguments, but that's
# okay, because consider_cond_call_gc_wb makes sure that any
# caller-save registers with values in them are present in arglocs,
# so they are saved on the stack above and restored below
self.mc.MOV_rs(edi.value, 0)
+ self.mc.MOV_rs(esi.value, 8)
# misaligned stack in the call, but it's ok because the write barrier
# is not going to call anything more. Also, this assumes that the
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py Fri Oct 1 22:53:02 2010
@@ -696,9 +696,13 @@
def consider_cond_call_gc_wb(self, op):
assert op.result is None
args = op.getarglist()
+ loc_newvalue = self.rm.make_sure_var_in_reg(op.getarg(1), args)
+ # ^^^ we force loc_newvalue in a reg (unless it's a Const),
+ # because it will be needed anyway by the following setfield_gc.
+ # It avoids loading it twice from the memory.
loc_base = self.rm.make_sure_var_in_reg(op.getarg(0), args,
imm_fine=False)
- arglocs = [loc_base]
+ arglocs = [loc_base, loc_newvalue]
# add eax, ecx and edx as extra "arguments" to ensure they are
# saved and restored. Fish in self.rm to know which of these
# registers really need to be saved (a bit of a hack). Moreover,
@@ -959,18 +963,23 @@
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(args[0], args)
ofs_loc = self.rm.make_sure_var_in_reg(args[2], args)
+ assert args[0] is not args[1] # forbidden case of aliasing
self.rm.possibly_free_var(args[0])
- self.rm.possibly_free_var(args[2])
+ if args[3] is not args[2] is not args[4]: # MESS MESS MESS: don't free
+ self.rm.possibly_free_var(args[2]) # it if ==args[3] or args[4]
srcaddr_box = TempBox()
- srcaddr_loc = self.rm.force_allocate_reg(srcaddr_box)
+ forbidden_vars = [args[1], args[3], args[4], srcaddr_box]
+ srcaddr_loc = self.rm.force_allocate_reg(srcaddr_box, forbidden_vars)
self._gen_address_inside_string(base_loc, ofs_loc, srcaddr_loc)
# compute the destination address
- base_loc = self.rm.make_sure_var_in_reg(args[1], args)
- ofs_loc = self.rm.make_sure_var_in_reg(args[3], args)
+ base_loc = self.rm.make_sure_var_in_reg(args[1], forbidden_vars)
+ ofs_loc = self.rm.make_sure_var_in_reg(args[3], forbidden_vars)
self.rm.possibly_free_var(args[1])
- self.rm.possibly_free_var(args[3])
+ if args[3] is not args[4]: # more of the MESS described above
+ self.rm.possibly_free_var(args[3])
+ forbidden_vars = [args[4], srcaddr_box]
dstaddr_box = TempBox()
- dstaddr_loc = self.rm.force_allocate_reg(dstaddr_box)
+ dstaddr_loc = self.rm.force_allocate_reg(dstaddr_box, forbidden_vars)
self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc)
# call memcpy()
length_loc = self.loc(args[4])
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py Fri Oct 1 22:53:02 2010
@@ -87,7 +87,9 @@
def execute_token(self, executable_token):
addr = executable_token._x86_bootstrap_code
+ #llop.debug_print(lltype.Void, ">>>> Entering", addr)
func = rffi.cast(lltype.Ptr(self.BOOTSTRAP_TP), addr)
+ #llop.debug_print(lltype.Void, "<<<< Back")
fail_index = self._execute_call(func)
return self.get_fail_descr_from_number(fail_index)
@@ -99,10 +101,7 @@
LLInterpreter.current_interpreter = self.debug_ll_interpreter
res = 0
try:
- #llop.debug_print(lltype.Void, ">>>> Entering",
- # rffi.cast(lltype.Signed, func))
res = func()
- #llop.debug_print(lltype.Void, "<<<< Back")
finally:
if not self.translate_support_code:
LLInterpreter.current_interpreter = prev_interpreter
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/executor.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/executor.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/executor.py Fri Oct 1 22:53:02 2010
@@ -205,8 +205,8 @@
def do_copystrcontent(cpu, _, srcbox, dstbox,
srcstartbox, dststartbox, lengthbox):
- src = srcbox.getptr(lltype.Ptr(rstr.STR))
- dst = dstbox.getptr(lltype.Ptr(rstr.STR))
+ src = srcbox.getref(lltype.Ptr(rstr.STR))
+ dst = dstbox.getref(lltype.Ptr(rstr.STR))
srcstart = srcstartbox.getint()
dststart = dststartbox.getint()
length = lengthbox.getint()
@@ -214,8 +214,8 @@
def do_copyunicodecontent(cpu, _, srcbox, dstbox,
srcstartbox, dststartbox, lengthbox):
- src = srcbox.getptr(lltype.Ptr(rstr.UNICODE))
- dst = dstbox.getptr(lltype.Ptr(rstr.UNICODE))
+ src = srcbox.getref(lltype.Ptr(rstr.UNICODE))
+ dst = dstbox.getref(lltype.Ptr(rstr.UNICODE))
srcstart = srcstartbox.getint()
dststart = dststartbox.getint()
length = lengthbox.getint()
@@ -428,6 +428,10 @@
if arity == 3:
func = get_execute_funclist(3, False)[opnum]
return func(cpu, metainterp, argboxes[0], argboxes[1], argboxes[2])
+ if arity == 5: # copystrcontent, copyunicodecontent
+ func = get_execute_funclist(5, False)[opnum]
+ return func(cpu, metainterp, argboxes[0], argboxes[1],
+ argboxes[2], argboxes[3], argboxes[4])
raise NotImplementedError
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py Fri Oct 1 22:53:02 2010
@@ -455,7 +455,7 @@
'UNICODESETITEM/3',
'NEWUNICODE/1',
#'RUNTIMENEW/1', # ootype operation
- 'COND_CALL_GC_WB/1d', # [objptr] (for the write barrier)
+ 'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier)
'DEBUG_MERGE_POINT/1', # debugging only
'VIRTUAL_REF_FINISH/2', # removed before it's passed to the backend
'COPYSTRCONTENT/5', # src, dst, srcstart, dststart, length
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py Fri Oct 1 22:53:02 2010
@@ -67,9 +67,16 @@
def jittify_and_run(interp, graph, args, repeat=1,
backendopt=False, trace_limit=sys.maxint,
debug_level=DEBUG_STEPS, inline=False, **kwds):
+ from pypy.config.config import ConfigError
translator = interp.typer.annotator.translator
- translator.config.translation.gc = "boehm"
- translator.config.translation.list_comprehension_operations = True
+ try:
+ translator.config.translation.gc = "boehm"
+ except ConfigError:
+ pass
+ try:
+ translator.config.translation.list_comprehension_operations = True
+ except ConfigError:
+ pass
warmrunnerdesc = WarmRunnerDesc(translator, backendopt=backendopt, **kwds)
for jd in warmrunnerdesc.jitdrivers_sd:
jd.warmstate.set_param_threshold(3) # for tests
Modified: pypy/branch/fast-forward/pypy/module/imp/importing.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/imp/importing.py (original)
+++ pypy/branch/fast-forward/pypy/module/imp/importing.py Fri Oct 1 22:53:02 2010
@@ -122,7 +122,7 @@
n = len(ctxt_name_prefix_parts)-level+1
assert n>=0
ctxt_name_prefix_parts = ctxt_name_prefix_parts[:n]
- if ctxt_w_path is None: # plain module
+ if ctxt_name_prefix_parts and ctxt_w_path is None: # plain module
ctxt_name_prefix_parts.pop()
if ctxt_name_prefix_parts:
rel_modulename = '.'.join(ctxt_name_prefix_parts)
Modified: pypy/branch/fast-forward/pypy/module/imp/test/test_import.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/imp/test/test_import.py (original)
+++ pypy/branch/fast-forward/pypy/module/imp/test/test_import.py Fri Oct 1 22:53:02 2010
@@ -360,6 +360,12 @@
""".rstrip()
raises(ValueError, imp)
+ def test_future_relative_import_error_when_in_non_package2(self):
+ exec """def imp():
+ from .. import inpackage
+ """.rstrip()
+ raises(ValueError, imp)
+
def test_relative_import_with___name__(self):
import sys
mydict = {'__name__': 'sys.foo'}
Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py (original)
+++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py Fri Oct 1 22:53:02 2010
@@ -29,6 +29,7 @@
class RECT(Structure):
_fields_ = [("left", c_int), ("top", c_int),
("right", c_int), ("bottom", c_int)]
+
class TestFunctions(BaseCTypesTestChecker):
def test_mro(self):
@@ -392,6 +393,18 @@
result = f("abcd", ord("b"))
assert result == "bcd"
+ def test_caching_bug_1(self):
+ # the same test as test_call_some_args, with two extra lines
+ # in the middle that trigger caching in f._ptr, which then
+ # makes the last two lines fail
+ f = dll.my_strchr
+ f.argtypes = [c_char_p, c_int]
+ f.restype = c_char_p
+ result = f("abcd", ord("b"))
+ assert result == "bcd"
+ result = f("abcd", ord("b"), 42)
+ assert result == "bcd"
+
def test_sf1651235(self):
py.test.skip("we are less strict in checking callback parameters")
# see http://www.python.org/sf/1651235
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py Fri Oct 1 22:53:02 2010
@@ -124,6 +124,9 @@
assert self.usagemap[i] == 'x'
self.usagemap[i] = '#'
+ def mark_freed(self):
+ self.freed = True # this method is a hook for tests
+
class fakearenaaddress(llmemory.fakeaddress):
def __init__(self, arena, offset):
@@ -314,7 +317,7 @@
assert arena_addr.offset == 0
arena_addr.arena.reset(False)
assert not arena_addr.arena.objectptrs
- arena_addr.arena.freed = True
+ arena_addr.arena.mark_freed()
def arena_reset(arena_addr, size, zero):
"""Free all objects in the arena, which can then be reused.
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/base.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gc/base.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gc/base.py Fri Oct 1 22:53:02 2010
@@ -20,12 +20,15 @@
prebuilt_gc_objects_are_static_roots = True
object_minimal_size = 0
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE):
+ def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
+ translated_to_c=True):
self.gcheaderbuilder = GCHeaderBuilder(self.HDR)
self.AddressStack = get_address_stack(chunk_size)
self.AddressDeque = get_address_deque(chunk_size)
self.AddressDict = AddressDict
self.config = config
+ assert isinstance(translated_to_c, bool)
+ self.translated_to_c = translated_to_c
def setup(self):
# all runtime mutable values' setup should happen here
@@ -79,7 +82,7 @@
def set_root_walker(self, root_walker):
self.root_walker = root_walker
- def write_barrier(self, addr_struct):
+ def write_barrier(self, newvalue, addr_struct):
pass
def statistics(self, index):
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/generation.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gc/generation.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gc/generation.py Fri Oct 1 22:53:02 2010
@@ -5,7 +5,6 @@
from pypy.rpython.memory.gc.base import read_from_env
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
from pypy.rpython.lltypesystem import lltype, llmemory, llarena
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rlib.objectmodel import free_non_gc_object
from pypy.rlib.debug import ll_assert
from pypy.rlib.debug import debug_print, debug_start, debug_stop
@@ -49,15 +48,17 @@
nursery_hash_base = -1
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
+ def __init__(self, config,
nursery_size=32*WORD,
min_nursery_size=32*WORD,
auto_nursery_size=False,
space_size=1024*WORD,
- max_space_size=sys.maxint//2+1):
- SemiSpaceGC.__init__(self, config, chunk_size = chunk_size,
+ max_space_size=sys.maxint//2+1,
+ **kwds):
+ SemiSpaceGC.__init__(self, config,
space_size = space_size,
- max_space_size = max_space_size)
+ max_space_size = max_space_size,
+ **kwds)
assert min_nursery_size <= nursery_size <= space_size // 2
self.initial_nursery_size = nursery_size
self.auto_nursery_size = auto_nursery_size
@@ -157,6 +158,14 @@
"odd-valued (i.e. tagged) pointer unexpected here")
return self.nursery <= addr < self.nursery_top
+ def appears_to_be_in_nursery(self, addr):
+ # same as is_in_nursery(), but may return True accidentally if
+ # 'addr' is a tagged pointer with just the wrong value.
+ if not self.translated_to_c:
+ if not self.is_valid_gc_object(addr):
+ return False
+ return self.nursery <= addr < self.nursery_top
+
def malloc_fixedsize_clear(self, typeid, size, can_collect,
has_finalizer=False, contains_weakptr=False):
if (has_finalizer or not can_collect or
@@ -326,7 +335,7 @@
addr = pointer.address[0]
newaddr = self.copy(addr)
pointer.address[0] = newaddr
- self.write_into_last_generation_obj(obj)
+ self.write_into_last_generation_obj(obj, newaddr)
# ____________________________________________________________
# Implementation of nursery-only collections
@@ -457,9 +466,9 @@
# "if addr_struct.int0 & JIT_WB_IF_FLAG: remember_young_pointer()")
JIT_WB_IF_FLAG = GCFLAG_NO_YOUNG_PTRS
- def write_barrier(self, addr_struct):
- if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS:
- self.remember_young_pointer(addr_struct)
+ def write_barrier(self, newvalue, addr_struct):
+ if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS:
+ self.remember_young_pointer(addr_struct, newvalue)
def _setup_wb(self):
DEBUG = self.DEBUG
@@ -470,23 +479,33 @@
# For x86, there is also an extra requirement: when the JIT calls
# remember_young_pointer(), it assumes that it will not touch the SSE
# registers, so it does not save and restore them (that's a *hack*!).
- def remember_young_pointer(addr_struct):
+ def remember_young_pointer(addr_struct, addr):
#llop.debug_print(lltype.Void, "\tremember_young_pointer",
# addr_struct, "<-", addr)
if DEBUG:
ll_assert(not self.is_in_nursery(addr_struct),
"nursery object with GCFLAG_NO_YOUNG_PTRS")
- self.old_objects_pointing_to_young.append(addr_struct)
- self.header(addr_struct).tid &= ~GCFLAG_NO_YOUNG_PTRS
- self.write_into_last_generation_obj(addr_struct)
+ #
+ # What is important in this function is that it *must*
+ # clear the flag GCFLAG_NO_YOUNG_PTRS from 'addr_struct'
+ # if 'addr' is in the nursery. It is ok if, accidentally,
+ # it also clears the flag in some more rare cases, like
+ # 'addr' being a tagged pointer whose value happens to be
+ # a large integer that fools is_in_nursery().
+ if self.appears_to_be_in_nursery(addr):
+ self.old_objects_pointing_to_young.append(addr_struct)
+ self.header(addr_struct).tid &= ~GCFLAG_NO_YOUNG_PTRS
+ self.write_into_last_generation_obj(addr_struct, addr)
remember_young_pointer._dont_inline_ = True
self.remember_young_pointer = remember_young_pointer
- def write_into_last_generation_obj(self, addr_struct):
+ def write_into_last_generation_obj(self, addr_struct, addr):
objhdr = self.header(addr_struct)
if objhdr.tid & GCFLAG_NO_HEAP_PTRS:
- objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
- self.last_generation_root_objects.append(addr_struct)
+ if (self.is_valid_gc_object(addr) and
+ not self.is_last_generation(addr)):
+ objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
+ self.last_generation_root_objects.append(addr_struct)
write_into_last_generation_obj._always_inline_ = True
def assume_young_pointers(self, addr_struct):
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/markcompact.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gc/markcompact.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gc/markcompact.py Fri Oct 1 22:53:02 2010
@@ -2,7 +2,6 @@
from pypy.rpython.memory.gc.base import MovingGCBase, read_from_env
from pypy.rlib.debug import ll_assert, have_debug_prints
from pypy.rlib.debug import debug_print, debug_start, debug_stop
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rpython.memory.support import get_address_stack, get_address_deque
from pypy.rpython.memory.support import AddressDict
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
@@ -86,9 +85,9 @@
free = NULL
next_collect_after = -1
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, space_size=4096,
- min_next_collect_after=128):
- MovingGCBase.__init__(self, config, chunk_size)
+ def __init__(self, config, space_size=4096,
+ min_next_collect_after=128, **kwds):
+ MovingGCBase.__init__(self, config, **kwds)
self.space_size = space_size
self.min_next_collect_after = min_next_collect_after
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/marksweep.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gc/marksweep.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gc/marksweep.py Fri Oct 1 22:53:02 2010
@@ -1,7 +1,6 @@
from pypy.rpython.lltypesystem.llmemory import raw_malloc, raw_free
from pypy.rpython.lltypesystem.llmemory import raw_memcopy, raw_memclear
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rpython.memory.support import get_address_stack
from pypy.rpython.memory.gcheader import GCHeaderBuilder
from pypy.rpython.lltypesystem import lltype, llmemory, rffi, llgroup
@@ -48,9 +47,9 @@
# translating to a real backend.
TRANSLATION_PARAMS = {'start_heap_size': 8*1024*1024} # XXX adjust
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, start_heap_size=4096):
+ def __init__(self, config, start_heap_size=4096, **kwds):
self.param_start_heap_size = start_heap_size
- GCBase.__init__(self, config, chunk_size)
+ GCBase.__init__(self, config, **kwds)
def setup(self):
GCBase.setup(self)
@@ -714,8 +713,8 @@
_alloc_flavor_ = "raw"
COLLECT_EVERY = 2000
- def __init__(self, chunk_size=DEFAULT_CHUNK_SIZE, start_heap_size=4096):
- MarkSweepGC.__init__(self, chunk_size, start_heap_size)
+ def __init__(self, config, **kwds):
+ MarkSweepGC.__init__(self, config, **kwds)
self.count_mallocs = 0
def maybe_collect(self):
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py Fri Oct 1 22:53:02 2010
@@ -4,7 +4,6 @@
from pypy.rpython.lltypesystem.llmemory import raw_malloc_usage
from pypy.rpython.memory.gc.base import GCBase, MovingGCBase
from pypy.rpython.memory.gc import minimarkpage, base, generation
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask, r_uint
from pypy.rlib.rarithmetic import LONG_BIT_SHIFT
from pypy.rlib.debug import ll_assert, debug_print, debug_start, debug_stop
@@ -140,7 +139,7 @@
"large_object_gcptrs": 8250*WORD,
}
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
+ def __init__(self, config,
read_from_env=False,
nursery_size=32*WORD,
page_size=16*WORD,
@@ -150,8 +149,9 @@
card_page_indices=0,
large_object=8*WORD,
large_object_gcptrs=10*WORD,
- ArenaCollectionClass=None):
- MovingGCBase.__init__(self, config, chunk_size)
+ ArenaCollectionClass=None,
+ **kwds):
+ MovingGCBase.__init__(self, config, **kwds)
assert small_request_threshold % WORD == 0
self.read_from_env = read_from_env
self.nursery_size = nursery_size
@@ -636,6 +636,14 @@
"odd-valued (i.e. tagged) pointer unexpected here")
return self.nursery <= addr < self.nursery_top
+ def appears_to_be_in_nursery(self, addr):
+ # same as is_in_nursery(), but may return True accidentally if
+ # 'addr' is a tagged pointer with just the wrong value.
+ if not self.translated_to_c:
+ if not self.is_valid_gc_object(addr):
+ return False
+ return self.nursery <= addr < self.nursery_top
+
def is_forwarded(self, obj):
"""Returns True if the nursery obj is marked as forwarded.
Implemented a bit obscurely by checking an unrelated flag
@@ -726,16 +734,16 @@
def JIT_max_size_of_young_obj(cls):
return cls.TRANSLATION_PARAMS['large_object']
- def write_barrier(self, addr_struct):
+ def write_barrier(self, newvalue, addr_struct):
if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS:
- self.remember_young_pointer(addr_struct)
+ self.remember_young_pointer(addr_struct, newvalue)
- def write_barrier_from_array(self, addr_array, index):
+ def write_barrier_from_array(self, newvalue, addr_array, index):
if self.header(addr_array).tid & GCFLAG_NO_YOUNG_PTRS:
if self.card_page_indices > 0: # <- constant-folded
self.remember_young_pointer_from_array(addr_array, index)
else:
- self.remember_young_pointer(addr_array)
+ self.remember_young_pointer(addr_array, newvalue)
def _init_writebarrier_logic(self):
DEBUG = self.DEBUG
@@ -746,27 +754,28 @@
# For x86, there is also an extra requirement: when the JIT calls
# remember_young_pointer(), it assumes that it will not touch the SSE
# registers, so it does not save and restore them (that's a *hack*!).
- def remember_young_pointer(addr_struct):
+ def remember_young_pointer(addr_struct, newvalue):
# 'addr_struct' is the address of the object in which we write.
+ # 'newvalue' is the address that we are going to write in there.
if DEBUG:
ll_assert(not self.is_in_nursery(addr_struct),
"nursery object with GCFLAG_NO_YOUNG_PTRS")
#
- # We assume that what we are writing is a pointer to the nursery
- # (and don't care for the fact that this new pointer may not
- # actually point to the nursery, which seems ok). What we need is
+ # If it seems that what we are writing is a pointer to the nursery
+ # (as checked with appears_to_be_in_nursery()), then we need
# to remove the flag GCFLAG_NO_YOUNG_PTRS and add the old object
# to the list 'old_objects_pointing_to_young'. We know that
# 'addr_struct' cannot be in the nursery, because nursery objects
# never have the flag GCFLAG_NO_YOUNG_PTRS to start with.
- self.old_objects_pointing_to_young.append(addr_struct)
objhdr = self.header(addr_struct)
- objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
+ if self.appears_to_be_in_nursery(newvalue):
+ self.old_objects_pointing_to_young.append(addr_struct)
+ objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
#
# Second part: if 'addr_struct' is actually a prebuilt GC
# object and it's the first time we see a write to it, we
# add it to the list 'prebuilt_root_objects'. Note that we
- # do it even in the (rare?) case of 'addr' being another
+ # do it even in the (rare?) case of 'addr' being NULL or another
# prebuilt object, to simplify code.
if objhdr.tid & GCFLAG_NO_HEAP_PTRS:
objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
@@ -780,16 +789,24 @@
def _init_writebarrier_with_card_marker(self):
+ DEBUG = self.DEBUG
def remember_young_pointer_from_array(addr_array, index):
# 'addr_array' is the address of the object in which we write,
# which must have an array part; 'index' is the index of the
# item that is (or contains) the pointer that we write.
+ if DEBUG:
+ ll_assert(not self.is_in_nursery(addr_array),
+ "nursery array with GCFLAG_NO_YOUNG_PTRS")
objhdr = self.header(addr_array)
if objhdr.tid & GCFLAG_HAS_CARDS == 0:
#
- # no cards, use default logic. The 'nocard_logic()' is just
- # 'remember_young_pointer()', but forced to be inlined here.
- nocard_logic(addr_array)
+ # no cards, use default logic. Mostly copied from above.
+ self.old_objects_pointing_to_young.append(addr_array)
+ objhdr = self.header(addr_array)
+ objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
+ if objhdr.tid & GCFLAG_NO_HEAP_PTRS:
+ objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
+ self.prebuilt_root_objects.append(addr_array)
return
#
# 'addr_array' is a raw_malloc'ed array with card markers
@@ -807,17 +824,15 @@
return
#
# We set the flag (even if the newly written address does not
- # actually point to the nursery -- like remember_young_pointer()).
+ # actually point to the nursery, which seems to be ok -- actually
+ # it seems more important that remember_young_pointer_from_array()
+ # does not take 3 arguments).
addr_byte.char[0] = chr(byte | bitmask)
#
if objhdr.tid & GCFLAG_CARDS_SET == 0:
self.old_objects_with_cards_set.append(addr_array)
objhdr.tid |= GCFLAG_CARDS_SET
- nocard_logic = func_with_new_name(self.remember_young_pointer,
- 'remember_young_pointer_nocard')
- del nocard_logic._dont_inline_
- nocard_logic._always_inline_ = True
remember_young_pointer_from_array._dont_inline_ = True
self.remember_young_pointer_from_array = (
remember_young_pointer_from_array)
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/minimarkpage.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gc/minimarkpage.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gc/minimarkpage.py Fri Oct 1 22:53:02 2010
@@ -4,15 +4,45 @@
from pypy.rlib.debug import ll_assert
WORD = LONG_BIT // 8
-WORD_POWER_2 = {32: 2, 64: 3}[LONG_BIT]
NULL = llmemory.NULL
+WORD_POWER_2 = {32: 2, 64: 3}[LONG_BIT]
+assert 1 << WORD_POWER_2 == WORD
-# Terminology: the memory is subdivided into "pages".
+# Terminology: the memory is subdivided into "arenas" containing "pages".
# A page contains a number of allocated objects, called "blocks".
-# The actual allocation occurs in whole arenas, which are subdivided
-# into pages. We don't keep track of the arenas. A page can be:
+# The actual allocation occurs in whole arenas, which are then subdivided
+# into pages. For each arena we allocate one of the following structures:
+
+ARENA_PTR = lltype.Ptr(lltype.ForwardReference())
+ARENA = lltype.Struct('ArenaReference',
+ # -- The address of the arena, as returned by malloc()
+ ('base', llmemory.Address),
+ # -- The number of free and the total number of pages in the arena
+ ('nfreepages', lltype.Signed),
+ ('totalpages', lltype.Signed),
+ # -- A chained list of free pages in the arena. Ends with NULL.
+ ('freepages', llmemory.Address),
+ # -- A linked list of arenas. See below.
+ ('nextarena', ARENA_PTR),
+ )
+ARENA_PTR.TO.become(ARENA)
+ARENA_NULL = lltype.nullptr(ARENA)
+
+# The idea is that when we need a free page, we take it from the arena
+# which currently has the *lowest* number of free pages. This allows
+# arenas with a lot of free pages to eventually become entirely free, at
+# which point they are returned to the OS. If an arena has a total of
+# 64 pages, then we have 64 global lists, arenas_lists[0] to
+# arenas_lists[63], such that arenas_lists[i] contains exactly those
+# arenas that have 'nfreepages == i'. We allocate pages out of the
+# arena in 'current_arena'; when it is exhausted we pick another arena
+# with the smallest value for nfreepages (but > 0).
+
+# ____________________________________________________________
+#
+# Each page in an arena can be:
#
# - uninitialized: never touched so far.
#
@@ -21,10 +51,11 @@
# room for objects of that size, unless it is completely full.
#
# - free: used to be partially full, and is now free again. The page is
-# on the chained list of free pages.
+# on the chained list of free pages 'freepages' from its arena.
-# Similarily, each allocated page contains blocks of a given size, which can
-# be either uninitialized, allocated or free.
+# Each allocated page contains blocks of a given size, which can again be in
+# one of three states: allocated, free, or uninitialized. The uninitialized
+# blocks (initially all of them) are at the tail of the page.
PAGE_PTR = lltype.Ptr(lltype.ForwardReference())
PAGE_HEADER = lltype.Struct('PageHeader',
@@ -32,13 +63,16 @@
# pages, it is a chained list of pages having the same size class,
# rooted in 'page_for_size[size_class]'. For full pages, it is a
# different chained list rooted in 'full_page_for_size[size_class]'.
+ # For free pages, it is the list 'freepages' in the arena header.
('nextpage', PAGE_PTR),
- # -- The number of free blocks, and the number of uninitialized blocks.
- # The number of allocated blocks is the rest.
- ('nuninitialized', lltype.Signed),
+ # -- The arena this page is part of.
+ ('arena', ARENA_PTR),
+ # -- The number of free blocks. The numbers of uninitialized and
+ # allocated blocks can be deduced from the context if needed.
('nfree', lltype.Signed),
- # -- The chained list of free blocks. If there are none, points to the
- # first uninitialized block.
+ # -- The chained list of free blocks. It ends as a pointer to the
+ # first uninitialized block (pointing to data that is uninitialized,
+ # or to the end of the page).
('freeblock', llmemory.Address),
# -- The structure above is 4 words, which is a good value:
# '(1024-4) % N' is zero or very small for various small N's,
@@ -72,13 +106,35 @@
self.nblocks_for_size = lltype.malloc(rffi.CArray(lltype.Signed),
length, flavor='raw')
self.hdrsize = llmemory.raw_malloc_usage(llmemory.sizeof(PAGE_HEADER))
+ assert page_size > self.hdrsize
self.nblocks_for_size[0] = 0 # unused
for i in range(1, length):
self.nblocks_for_size[i] = (page_size - self.hdrsize) // (WORD * i)
#
- self.uninitialized_pages = NULL
+ self.max_pages_per_arena = arena_size // page_size
+ self.arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
+ self.max_pages_per_arena,
+ flavor='raw', zero=True)
+ # this is used in mass_free() only
+ self.old_arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
+ self.max_pages_per_arena,
+ flavor='raw', zero=True)
+ #
+ # the arena currently consumed; it must have at least one page
+ # available, or be NULL. The arena object that we point to is
+ # not in any 'arenas_lists'. We will consume all its pages before
+ # we choose a next arena, even if there is a major collection
+ # in-between.
+ self.current_arena = ARENA_NULL
+ #
+ # guarantee that 'arenas_lists[1:min_empty_nfreepages]' are all empty
+ self.min_empty_nfreepages = self.max_pages_per_arena
+ #
+ # part of current_arena might still contain uninitialized pages
self.num_uninitialized_pages = 0
- self.free_pages = NULL
+ #
+ # the total memory used, counting every block in use, without
+ # the additional bookkeeping stuff.
self.total_memory_used = r_uint(0)
@@ -109,16 +165,12 @@
#
else:
# The 'result' is part of the uninitialized blocks.
- ll_assert(page.nuninitialized > 0,
- "fully allocated page found in the page_for_size list")
- page.nuninitialized -= 1
- if page.nuninitialized > 0:
- freeblock = result + nsize
- else:
- freeblock = NULL
+ freeblock = result + nsize
#
page.freeblock = freeblock
- if freeblock == NULL:
+ #
+ pageaddr = llarena.getfakearenaaddress(llmemory.cast_ptr_to_adr(page))
+ if freeblock - pageaddr > self.page_size - nsize:
# This was the last free block, so unlink the page from the
# chained list and put it in the 'full_page_for_size' list.
self.page_for_size[size_class] = page.nextpage
@@ -132,37 +184,88 @@
def allocate_new_page(self, size_class):
"""Allocate and return a new page for the given size_class."""
#
- if self.free_pages != NULL:
+ # Allocate a new arena if needed.
+ if self.current_arena == ARENA_NULL:
+ self.allocate_new_arena()
+ #
+ # The result is simply 'current_arena.freepages'.
+ arena = self.current_arena
+ result = arena.freepages
+ if arena.nfreepages > 0:
+ #
+ # The 'result' was part of the chained list; read the next.
+ arena.nfreepages -= 1
+ freepages = result.address[0]
+ llarena.arena_reset(result,
+ llmemory.sizeof(llmemory.Address),
+ 0)
#
- # Get the page from the chained list 'free_pages'.
- page = self.free_pages
- self.free_pages = page.address[0]
- llarena.arena_reset(page, llmemory.sizeof(llmemory.Address), 0)
else:
- # Get the next free page from the uninitialized pages.
- if self.num_uninitialized_pages == 0:
- self.allocate_new_arena() # Out of memory. Get a new arena.
- page = self.uninitialized_pages
- self.uninitialized_pages += self.page_size
+ # The 'result' is part of the uninitialized pages.
+ ll_assert(self.num_uninitialized_pages > 0,
+ "fully allocated arena found in self.current_arena")
self.num_uninitialized_pages -= 1
+ if self.num_uninitialized_pages > 0:
+ freepages = result + self.page_size
+ else:
+ freepages = NULL
#
- # Initialize the fields of the resulting page
- llarena.arena_reserve(page, llmemory.sizeof(PAGE_HEADER))
- result = llmemory.cast_adr_to_ptr(page, PAGE_PTR)
+ arena.freepages = freepages
+ if freepages == NULL:
+ # This was the last page, so put the arena away into
+ # arenas_lists[0].
+ ll_assert(arena.nfreepages == 0,
+ "freepages == NULL but nfreepages > 0")
+ arena.nextarena = self.arenas_lists[0]
+ self.arenas_lists[0] = arena
+ self.current_arena = ARENA_NULL
#
- result.nuninitialized = self.nblocks_for_size[size_class]
- result.nfree = 0
- result.freeblock = page + self.hdrsize
- result.nextpage = PAGE_NULL
+ # Initialize the fields of the resulting page
+ llarena.arena_reserve(result, llmemory.sizeof(PAGE_HEADER))
+ page = llmemory.cast_adr_to_ptr(result, PAGE_PTR)
+ page.arena = arena
+ page.nfree = 0
+ page.freeblock = result + self.hdrsize
+ page.nextpage = PAGE_NULL
ll_assert(self.page_for_size[size_class] == PAGE_NULL,
"allocate_new_page() called but a page is already waiting")
- self.page_for_size[size_class] = result
- return result
+ self.page_for_size[size_class] = page
+ return page
+
+
+ def _all_arenas(self):
+ """For testing. Enumerates all arenas."""
+ if self.current_arena:
+ yield self.current_arena
+ for arena in self.arenas_lists:
+ while arena:
+ yield arena
+ arena = arena.nextarena
def allocate_new_arena(self):
- ll_assert(self.num_uninitialized_pages == 0,
- "some uninitialized pages are already waiting")
+ """Loads in self.current_arena the arena to allocate from next."""
+ #
+ # Pick an arena from 'arenas_lists[i]', with i as small as possible
+ # but > 0. Use caching with 'min_empty_nfreepages', which guarantees
+ # that 'arenas_lists[1:min_empty_nfreepages]' are all empty.
+ i = self.min_empty_nfreepages
+ while i < self.max_pages_per_arena:
+ #
+ if self.arenas_lists[i] != ARENA_NULL:
+ #
+ # Found it.
+ self.current_arena = self.arenas_lists[i]
+ self.arenas_lists[i] = self.current_arena.nextarena
+ return
+ #
+ i += 1
+ self.min_empty_nfreepages = i
+ #
+ # No more arena with any free page. We must allocate a new arena.
+ if not we_are_translated():
+ for a in self._all_arenas():
+ assert a.nfreepages == 0
#
# 'arena_base' points to the start of malloced memory; it might not
# be a page-aligned address
@@ -177,13 +280,15 @@
# 'npages' is the number of full pages just allocated
npages = (arena_end - firstpage) // self.page_size
#
- # add these pages to the list
- self.uninitialized_pages = firstpage
+ # Allocate an ARENA object and initialize it
+ arena = lltype.malloc(ARENA, flavor='raw')
+ arena.base = arena_base
+ arena.nfreepages = 0 # they are all uninitialized pages
+ arena.totalpages = npages
+ arena.freepages = firstpage
self.num_uninitialized_pages = npages
+ self.current_arena = arena
#
- # increase a bit arena_size for the next time
- self.arena_size = (self.arena_size // 4 * 5) + (self.page_size - 1)
- self.arena_size = (self.arena_size // self.page_size) * self.page_size
allocate_new_arena._dont_inline_ = True
@@ -199,16 +304,51 @@
#
# Walk the pages in 'page_for_size[size_class]' and
# 'full_page_for_size[size_class]' and free some objects.
- # Pages completely freed are added to 'self.free_pages', and
- # become available for reuse by any size class. Pages not
- # completely freed are re-chained either in
+ # Pages completely freed are added to 'page.arena.freepages',
+ # and become available for reuse by any size class. Pages
+ # not completely freed are re-chained either in
# 'full_page_for_size[]' or 'page_for_size[]'.
- self.mass_free_in_page(size_class, ok_to_free_func)
+ self.mass_free_in_pages(size_class, ok_to_free_func)
#
size_class -= 1
+ #
+ # Rehash arenas into the correct arenas_lists[i]. If
+ # 'self.current_arena' contains an arena too, it remains there.
+ (self.old_arenas_lists, self.arenas_lists) = (
+ self.arenas_lists, self.old_arenas_lists)
+ #
+ i = 0
+ while i < self.max_pages_per_arena:
+ self.arenas_lists[i] = ARENA_NULL
+ i += 1
+ #
+ i = 0
+ while i < self.max_pages_per_arena:
+ arena = self.old_arenas_lists[i]
+ while arena != ARENA_NULL:
+ nextarena = arena.nextarena
+ #
+ if arena.nfreepages == arena.totalpages:
+ #
+ # The whole arena is empty. Free it.
+ llarena.arena_free(arena.base)
+ lltype.free(arena, flavor='raw')
+ #
+ else:
+ # Insert 'arena' in the correct arenas_lists[n]
+ n = arena.nfreepages
+ ll_assert(n < self.max_pages_per_arena,
+ "totalpages != nfreepages >= max_pages_per_arena")
+ arena.nextarena = self.arenas_lists[n]
+ self.arenas_lists[n] = arena
+ #
+ arena = nextarena
+ i += 1
+ #
+ self.min_empty_nfreepages = 1
- def mass_free_in_page(self, size_class, ok_to_free_func):
+ def mass_free_in_pages(self, size_class, ok_to_free_func):
nblocks = self.nblocks_for_size[size_class]
block_size = size_class * WORD
remaining_partial_pages = PAGE_NULL
@@ -224,8 +364,7 @@
while page != PAGE_NULL:
#
# Collect the page.
- surviving = self.walk_page(page, block_size,
- nblocks, ok_to_free_func)
+ surviving = self.walk_page(page, block_size, ok_to_free_func)
nextpage = page.nextpage
#
if surviving == nblocks:
@@ -259,19 +398,23 @@
def free_page(self, page):
"""Free a whole page."""
#
- # Done by inserting it in the 'free_pages' list.
+ # Insert the freed page in the arena's 'freepages' list.
+ # If nfreepages == totalpages, then it will be freed at the
+ # end of mass_free().
+ arena = page.arena
+ arena.nfreepages += 1
pageaddr = llmemory.cast_ptr_to_adr(page)
pageaddr = llarena.getfakearenaaddress(pageaddr)
llarena.arena_reset(pageaddr, self.page_size, 0)
llarena.arena_reserve(pageaddr, llmemory.sizeof(llmemory.Address))
- pageaddr.address[0] = self.free_pages
- self.free_pages = pageaddr
+ pageaddr.address[0] = arena.freepages
+ arena.freepages = pageaddr
- def walk_page(self, page, block_size, nblocks, ok_to_free_func):
+ def walk_page(self, page, block_size, ok_to_free_func):
"""Walk over all objects in a page, and ask ok_to_free_func()."""
#
- # 'freeblock' is the next free block, or NULL if there isn't any more.
+ # 'freeblock' is the next free block
freeblock = page.freeblock
#
# 'prevfreeblockat' is the address of where 'freeblock' was read from.
@@ -281,22 +424,28 @@
obj = llarena.getfakearenaaddress(llmemory.cast_ptr_to_adr(page))
obj += self.hdrsize
surviving = 0 # initially
+ skip_free_blocks = page.nfree
#
- nblocks -= page.nuninitialized
- index = nblocks
- while index > 0:
+ while True:
#
if obj == freeblock:
#
+ if skip_free_blocks == 0:
+ #
+ # 'obj' points to the first uninitialized block,
+ # or to the end of the page if there are none.
+ break
+ #
# 'obj' points to a free block. It means that
# 'prevfreeblockat.address[0]' does not need to be updated.
# Just read the next free block from 'obj.address[0]'.
+ skip_free_blocks -= 1
prevfreeblockat = obj
freeblock = obj.address[0]
#
else:
# 'obj' points to a valid object.
- ll_assert(not freeblock or freeblock > obj,
+ ll_assert(freeblock > obj,
"freeblocks are linked out of order")
#
if ok_to_free_func(obj):
@@ -310,15 +459,14 @@
prevfreeblockat = obj
obj.address[0] = freeblock
#
+ # Update the number of free objects in the page.
+ page.nfree += 1
+ #
else:
# The object survives.
surviving += 1
#
obj += block_size
- index -= 1
- #
- # Update the number of free objects in the page.
- page.nfree = nblocks - surviving
#
# Update the global total size of objects.
self.total_memory_used += surviving * block_size
@@ -327,6 +475,20 @@
return surviving
+ def _nuninitialized(self, page, size_class):
+ # Helper for debugging: count the number of uninitialized blocks
+ freeblock = page.freeblock
+ for i in range(page.nfree):
+ freeblock = freeblock.address[0]
+ assert freeblock != NULL
+ pageaddr = llarena.getfakearenaaddress(llmemory.cast_ptr_to_adr(page))
+ num_initialized_blocks, rem = divmod(
+ freeblock - pageaddr - self.hdrsize, size_class * WORD)
+ assert rem == 0, "page size_class misspecified?"
+ nblocks = self.nblocks_for_size[size_class]
+ return nblocks - num_initialized_blocks
+
+
# ____________________________________________________________
# Helpers to go from a pointer to the start of its page
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/semispace.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gc/semispace.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gc/semispace.py Fri Oct 1 22:53:02 2010
@@ -1,7 +1,6 @@
from pypy.rpython.lltypesystem.llmemory import raw_malloc, raw_free
from pypy.rpython.lltypesystem.llmemory import raw_memcopy, raw_memclear
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rpython.memory.support import get_address_stack, get_address_deque
from pypy.rpython.memory.support import AddressDict
from pypy.rpython.lltypesystem import lltype, llmemory, llarena, rffi, llgroup
@@ -59,11 +58,11 @@
# translating to a real backend.
TRANSLATION_PARAMS = {'space_size': 8*1024*1024} # XXX adjust
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, space_size=4096,
- max_space_size=sys.maxint//2+1):
+ def __init__(self, config, space_size=4096, max_space_size=sys.maxint//2+1,
+ **kwds):
self.param_space_size = space_size
self.param_max_space_size = max_space_size
- MovingGCBase.__init__(self, config, chunk_size)
+ MovingGCBase.__init__(self, config, **kwds)
def setup(self):
#self.total_collection_time = 0.0
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/test/test_direct.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gc/test/test_direct.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gc/test/test_direct.py Fri Oct 1 22:53:02 2010
@@ -70,6 +70,7 @@
GC_PARAMS = self.GC_PARAMS.copy()
if hasattr(meth, 'GC_PARAMS'):
GC_PARAMS.update(meth.GC_PARAMS)
+ GC_PARAMS['translated_to_c'] = False
self.gc = self.GCClass(config, **GC_PARAMS)
self.gc.DEBUG = True
self.rootwalker = DirectRootWalker(self)
@@ -86,17 +87,19 @@
def write(self, p, fieldname, newvalue):
if self.gc.needs_write_barrier:
+ newaddr = llmemory.cast_ptr_to_adr(newvalue)
addr_struct = llmemory.cast_ptr_to_adr(p)
- self.gc.write_barrier(addr_struct)
+ self.gc.write_barrier(newaddr, addr_struct)
setattr(p, fieldname, newvalue)
def writearray(self, p, index, newvalue):
if self.gc.needs_write_barrier:
+ newaddr = llmemory.cast_ptr_to_adr(newvalue)
addr_struct = llmemory.cast_ptr_to_adr(p)
if hasattr(self.gc, 'write_barrier_from_array'):
- self.gc.write_barrier_from_array(addr_struct, index)
+ self.gc.write_barrier_from_array(newaddr, addr_struct, index)
else:
- self.gc.write_barrier(addr_struct)
+ self.gc.write_barrier(newaddr, addr_struct)
p[index] = newvalue
def malloc(self, TYPE, n=None):
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/test/test_minimarkpage.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gc/test/test_minimarkpage.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gc/test/test_minimarkpage.py Fri Oct 1 22:53:02 2010
@@ -12,17 +12,19 @@
def test_allocate_arena():
- ac = ArenaCollection(SHIFT + 16*20, 16, 1)
+ ac = ArenaCollection(SHIFT + 64*20, 64, 1)
ac.allocate_new_arena()
assert ac.num_uninitialized_pages == 20
- ac.uninitialized_pages + 16*20 # does not raise
- py.test.raises(llarena.ArenaError, "ac.uninitialized_pages + 16*20 + 1")
+ upages = ac.current_arena.freepages
+ upages + 64*20 # does not raise
+ py.test.raises(llarena.ArenaError, "upages + 64*20 + 1")
#
- ac = ArenaCollection(SHIFT + 16*20 + 7, 16, 1)
+ ac = ArenaCollection(SHIFT + 64*20 + 7, 64, 1)
ac.allocate_new_arena()
assert ac.num_uninitialized_pages == 20
- ac.uninitialized_pages + 16*20 + 7 # does not raise
- py.test.raises(llarena.ArenaError, "ac.uninitialized_pages + 16*20 + 16")
+ upages = ac.current_arena.freepages
+ upages + 64*20 + 7 # does not raise
+ py.test.raises(llarena.ArenaError, "upages + 64*20 + 64")
def test_allocate_new_page():
@@ -31,7 +33,8 @@
#
def checknewpage(page, size_class):
size = WORD * size_class
- assert page.nuninitialized == (pagesize - hdrsize) // size
+ assert (ac._nuninitialized(page, size_class) ==
+ (pagesize - hdrsize) // size)
assert page.nfree == 0
page1 = page.freeblock - hdrsize
assert llmemory.cast_ptr_to_adr(page) == page1
@@ -44,13 +47,13 @@
page = ac.allocate_new_page(5)
checknewpage(page, 5)
assert ac.num_uninitialized_pages == 2
- assert ac.uninitialized_pages - pagesize == cast_ptr_to_adr(page)
+ assert ac.current_arena.freepages - pagesize == cast_ptr_to_adr(page)
assert ac.page_for_size[5] == page
#
page = ac.allocate_new_page(3)
checknewpage(page, 3)
assert ac.num_uninitialized_pages == 1
- assert ac.uninitialized_pages - pagesize == cast_ptr_to_adr(page)
+ assert ac.current_arena.freepages - pagesize == cast_ptr_to_adr(page)
assert ac.page_for_size[3] == page
#
page = ac.allocate_new_page(4)
@@ -71,17 +74,17 @@
page = llmemory.cast_adr_to_ptr(pageaddr, PAGE_PTR)
if step == 1:
page.nfree = 0
- page.nuninitialized = nblocks - nusedblocks
+ nuninitialized = nblocks - nusedblocks
else:
page.nfree = nusedblocks
- page.nuninitialized = nblocks - 2*nusedblocks
+ nuninitialized = nblocks - 2*nusedblocks
+ page.freeblock = pageaddr + hdrsize + nusedblocks * size_block
if nusedblocks < nblocks:
- page.freeblock = pageaddr + hdrsize + nusedblocks * size_block
chainedlists = ac.page_for_size
else:
- page.freeblock = NULL
chainedlists = ac.full_page_for_size
page.nextpage = chainedlists[size_class]
+ page.arena = ac.current_arena
chainedlists[size_class] = page
if fill_with_objects:
for i in range(0, nusedblocks*step, step):
@@ -98,11 +101,15 @@
prev = 'prevhole.address[0]'
endaddr = pageaddr + hdrsize + 2*nusedblocks * size_block
exec '%s = endaddr' % prev in globals(), locals()
+ assert ac._nuninitialized(page, size_class) == nuninitialized
#
ac.allocate_new_arena()
num_initialized_pages = len(pagelayout.rstrip(" "))
- ac._startpageaddr = ac.uninitialized_pages
- ac.uninitialized_pages += pagesize * num_initialized_pages
+ ac._startpageaddr = ac.current_arena.freepages
+ if pagelayout.endswith(" "):
+ ac.current_arena.freepages += pagesize * num_initialized_pages
+ else:
+ ac.current_arena.freepages = NULL
ac.num_uninitialized_pages -= num_initialized_pages
#
for i in reversed(range(num_initialized_pages)):
@@ -115,8 +122,9 @@
link(pageaddr, size_class, size_block, nblocks, nblocks-1)
elif c == '.': # a free, but initialized, page
llarena.arena_reserve(pageaddr, llmemory.sizeof(llmemory.Address))
- pageaddr.address[0] = ac.free_pages
- ac.free_pages = pageaddr
+ pageaddr.address[0] = ac.current_arena.freepages
+ ac.current_arena.freepages = pageaddr
+ ac.current_arena.nfreepages += 1
elif c == '#': # a random full page, in the list 'full_pages'
size_class = fill_with_objects or 1
size_block = WORD * size_class
@@ -142,26 +150,29 @@
def checkpage(ac, page, expected_position):
assert llmemory.cast_ptr_to_adr(page) == pagenum(ac, expected_position)
+def freepages(ac):
+ return ac.current_arena.freepages
+
def test_simple_arena_collection():
pagesize = hdrsize + 16
ac = arena_collection_for_test(pagesize, "##....# ")
#
- assert ac.free_pages == pagenum(ac, 2)
+ assert freepages(ac) == pagenum(ac, 2)
page = ac.allocate_new_page(1); checkpage(ac, page, 2)
- assert ac.free_pages == pagenum(ac, 3)
+ assert freepages(ac) == pagenum(ac, 3)
page = ac.allocate_new_page(2); checkpage(ac, page, 3)
- assert ac.free_pages == pagenum(ac, 4)
+ assert freepages(ac) == pagenum(ac, 4)
page = ac.allocate_new_page(3); checkpage(ac, page, 4)
- assert ac.free_pages == pagenum(ac, 5)
+ assert freepages(ac) == pagenum(ac, 5)
page = ac.allocate_new_page(4); checkpage(ac, page, 5)
- assert ac.free_pages == NULL and ac.num_uninitialized_pages == 3
+ assert freepages(ac) == pagenum(ac, 7) and ac.num_uninitialized_pages == 3
page = ac.allocate_new_page(5); checkpage(ac, page, 7)
- assert ac.free_pages == NULL and ac.num_uninitialized_pages == 2
+ assert freepages(ac) == pagenum(ac, 8) and ac.num_uninitialized_pages == 2
page = ac.allocate_new_page(6); checkpage(ac, page, 8)
- assert ac.free_pages == NULL and ac.num_uninitialized_pages == 1
+ assert freepages(ac) == pagenum(ac, 9) and ac.num_uninitialized_pages == 1
page = ac.allocate_new_page(7); checkpage(ac, page, 9)
- assert ac.free_pages == NULL and ac.num_uninitialized_pages == 0
+ assert not ac.current_arena and ac.num_uninitialized_pages == 0
def chkob(ac, num_page, pos_obj, obj):
@@ -205,47 +216,47 @@
ac = arena_collection_for_test(pagesize, "/.", fill_with_objects=2)
page = getpage(ac, 0)
assert page.nfree == 3
- assert page.nuninitialized == 3
+ assert ac._nuninitialized(page, 2) == 3
chkob(ac, 0, 2*WORD, page.freeblock)
#
obj = ac.malloc(2*WORD); chkob(ac, 0, 2*WORD, obj)
obj = ac.malloc(2*WORD); chkob(ac, 0, 6*WORD, obj)
assert page.nfree == 1
- assert page.nuninitialized == 3
+ assert ac._nuninitialized(page, 2) == 3
chkob(ac, 0, 10*WORD, page.freeblock)
#
obj = ac.malloc(2*WORD); chkob(ac, 0, 10*WORD, obj)
assert page.nfree == 0
- assert page.nuninitialized == 3
+ assert ac._nuninitialized(page, 2) == 3
chkob(ac, 0, 12*WORD, page.freeblock)
#
obj = ac.malloc(2*WORD); chkob(ac, 0, 12*WORD, obj)
- assert page.nuninitialized == 2
+ assert ac._nuninitialized(page, 2) == 2
obj = ac.malloc(2*WORD); chkob(ac, 0, 14*WORD, obj)
obj = ac.malloc(2*WORD); chkob(ac, 0, 16*WORD, obj)
assert page.nfree == 0
- assert page.nuninitialized == 0
+ assert ac._nuninitialized(page, 2) == 0
obj = ac.malloc(2*WORD); chkob(ac, 1, 0*WORD, obj)
def test_malloc_new_arena():
pagesize = hdrsize + 7*WORD
ac = arena_collection_for_test(pagesize, "### ")
+ arena_size = ac.arena_size
obj = ac.malloc(2*WORD); chkob(ac, 3, 0*WORD, obj) # 3rd page -> size 2
#
del ac.allocate_new_arena # restore the one from the class
- arena_size = ac.arena_size
obj = ac.malloc(3*WORD) # need a new arena
assert ac.num_uninitialized_pages == (arena_size // ac.page_size
- - 1 # for start_of_page()
- 1 # the just-allocated page
)
class OkToFree(object):
- def __init__(self, ac, answer):
+ def __init__(self, ac, answer, multiarenas=False):
assert callable(answer) or 0.0 <= answer <= 1.0
self.ac = ac
self.answer = answer
+ self.multiarenas = multiarenas
self.lastnum = 0.0
self.seen = {}
@@ -257,7 +268,10 @@
ok_to_free = self.lastnum >= 1.0
if ok_to_free:
self.lastnum -= 1.0
- key = addr - self.ac._startpageaddr
+ if self.multiarenas:
+ key = (addr.arena, addr.offset)
+ else:
+ key = addr - self.ac._startpageaddr
assert key not in self.seen
self.seen[key] = ok_to_free
return ok_to_free
@@ -272,10 +286,10 @@
page = getpage(ac, 0)
assert page == ac.page_for_size[2]
assert page.nextpage == PAGE_NULL
- assert page.nuninitialized == 1
+ assert ac._nuninitialized(page, 2) == 1
assert page.nfree == 0
chkob(ac, 0, 4*WORD, page.freeblock)
- assert ac.free_pages == NULL
+ assert freepages(ac) == NULL
def test_mass_free_emptied_page():
pagesize = hdrsize + 7*WORD
@@ -285,7 +299,7 @@
assert ok_to_free.seen == {hdrsize + 0*WORD: True,
hdrsize + 2*WORD: True}
pageaddr = pagenum(ac, 0)
- assert pageaddr == ac.free_pages
+ assert pageaddr == freepages(ac)
assert pageaddr.address[0] == NULL
assert ac.page_for_size[2] == PAGE_NULL
@@ -300,10 +314,9 @@
page = getpage(ac, 0)
assert page == ac.full_page_for_size[2]
assert page.nextpage == PAGE_NULL
- assert page.nuninitialized == 0
+ assert ac._nuninitialized(page, 2) == 0
assert page.nfree == 0
- assert page.freeblock == NULL
- assert ac.free_pages == NULL
+ assert freepages(ac) == NULL
assert ac.page_for_size[2] == PAGE_NULL
def test_mass_free_full_is_partially_emptied():
@@ -319,19 +332,19 @@
pageaddr = pagenum(ac, 0)
assert page == ac.page_for_size[2]
assert page.nextpage == PAGE_NULL
- assert page.nuninitialized == 0
+ assert ac._nuninitialized(page, 2) == 0
assert page.nfree == 2
assert page.freeblock == pageaddr + hdrsize + 2*WORD
assert page.freeblock.address[0] == pageaddr + hdrsize + 6*WORD
- assert page.freeblock.address[0].address[0] == NULL
- assert ac.free_pages == NULL
+ assert page.freeblock.address[0].address[0] == pageaddr + hdrsize + 8*WORD
+ assert freepages(ac) == NULL
assert ac.full_page_for_size[2] == PAGE_NULL
def test_mass_free_half_page_remains():
pagesize = hdrsize + 24*WORD
ac = arena_collection_for_test(pagesize, "/", fill_with_objects=2)
page = getpage(ac, 0)
- assert page.nuninitialized == 4
+ assert ac._nuninitialized(page, 2) == 4
assert page.nfree == 4
#
ok_to_free = OkToFree(ac, False)
@@ -344,7 +357,7 @@
pageaddr = pagenum(ac, 0)
assert page == ac.page_for_size[2]
assert page.nextpage == PAGE_NULL
- assert page.nuninitialized == 4
+ assert ac._nuninitialized(page, 2) == 4
assert page.nfree == 4
assert page.freeblock == pageaddr + hdrsize + 2*WORD
assert page.freeblock.address[0] == pageaddr + hdrsize + 6*WORD
@@ -352,14 +365,14 @@
pageaddr + hdrsize + 10*WORD
assert page.freeblock.address[0].address[0].address[0] == \
pageaddr + hdrsize + 14*WORD
- assert ac.free_pages == NULL
+ assert freepages(ac) == NULL
assert ac.full_page_for_size[2] == PAGE_NULL
def test_mass_free_half_page_becomes_more_free():
pagesize = hdrsize + 24*WORD
ac = arena_collection_for_test(pagesize, "/", fill_with_objects=2)
page = getpage(ac, 0)
- assert page.nuninitialized == 4
+ assert ac._nuninitialized(page, 2) == 4
assert page.nfree == 4
#
ok_to_free = OkToFree(ac, 0.5)
@@ -372,7 +385,7 @@
pageaddr = pagenum(ac, 0)
assert page == ac.page_for_size[2]
assert page.nextpage == PAGE_NULL
- assert page.nuninitialized == 4
+ assert ac._nuninitialized(page, 2) == 4
assert page.nfree == 6
fb = page.freeblock
assert fb == pageaddr + hdrsize + 2*WORD
@@ -384,7 +397,7 @@
pageaddr + hdrsize + 12*WORD
assert fb.address[0].address[0].address[0].address[0].address[0] == \
pageaddr + hdrsize + 14*WORD
- assert ac.free_pages == NULL
+ assert freepages(ac) == NULL
assert ac.full_page_for_size[2] == PAGE_NULL
# ____________________________________________________________
@@ -392,17 +405,29 @@
def test_random():
import random
pagesize = hdrsize + 24*WORD
- num_pages = 28
+ num_pages = 3
ac = arena_collection_for_test(pagesize, " " * num_pages)
live_objects = {}
#
- # Run the test until ac.allocate_new_arena() is called.
+ # Run the test until three arenas are freed. This is a quick test
+ # that the arenas are really freed by the logic.
class DoneTesting(Exception):
- pass
- def done_testing():
- raise DoneTesting
- ac.allocate_new_arena = done_testing
- #
+ counter = 0
+ def my_allocate_new_arena():
+ # the following output looks cool on a 112-character-wide terminal.
+ lst = sorted(ac._all_arenas(), key=lambda a: a.base.arena._arena_index)
+ for a in lst:
+ print a.base.arena, a.base.arena.usagemap
+ print '-' * 80
+ ac.__class__.allocate_new_arena(ac)
+ a = ac.current_arena.base.arena
+ def my_mark_freed():
+ a.freed = True
+ DoneTesting.counter += 1
+ if DoneTesting.counter > 3:
+ raise DoneTesting
+ a.mark_freed = my_mark_freed
+ ac.allocate_new_arena = my_allocate_new_arena
try:
while True:
#
@@ -410,12 +435,13 @@
for i in range(random.randrange(50, 100)):
size_class = random.randrange(1, 7)
obj = ac.malloc(size_class * WORD)
- at = obj - ac._startpageaddr
+ at = (obj.arena, obj.offset)
assert at not in live_objects
live_objects[at] = size_class * WORD
#
# Free half the objects, randomly
- ok_to_free = OkToFree(ac, lambda obj: random.random() < 0.5)
+ ok_to_free = OkToFree(ac, lambda obj: random.random() < 0.5,
+ multiarenas=True)
ac.mass_free(ok_to_free)
#
# Check that we have seen all objects
@@ -428,5 +454,4 @@
surviving_total_size += live_objects[at]
assert ac.total_memory_used == surviving_total_size
except DoneTesting:
- # the following output looks cool on a 112-character-wide terminal.
- print ac._startpageaddr.arena.usagemap
+ pass
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gctransform/framework.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gctransform/framework.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gctransform/framework.py Fri Oct 1 22:53:02 2010
@@ -426,6 +426,7 @@
if GCClass.needs_write_barrier:
self.write_barrier_ptr = getfn(GCClass.write_barrier.im_func,
[s_gc,
+ annmodel.SomeAddress(),
annmodel.SomeAddress()],
annmodel.s_None,
inline=True)
@@ -434,13 +435,15 @@
# func should not be a bound method, but a real function
assert isinstance(func, types.FunctionType)
self.write_barrier_failing_case_ptr = getfn(func,
- [annmodel.SomeAddress()],
+ [annmodel.SomeAddress(),
+ annmodel.SomeAddress()],
annmodel.s_None)
func = getattr(GCClass, 'write_barrier_from_array', None)
if func is not None:
self.write_barrier_from_array_ptr = getfn(func.im_func,
[s_gc,
annmodel.SomeAddress(),
+ annmodel.SomeAddress(),
annmodel.SomeInteger()],
annmodel.s_None,
inline=True)
@@ -1021,6 +1024,8 @@
and not isinstance(v_newvalue, Constant)
and v_struct.concretetype.TO._gckind == "gc"
and hop.spaceop not in self.clean_sets):
+ v_newvalue = hop.genop("cast_ptr_to_adr", [v_newvalue],
+ resulttype = llmemory.Address)
v_structaddr = hop.genop("cast_ptr_to_adr", [v_struct],
resulttype = llmemory.Address)
if (self.write_barrier_from_array_ptr is not None and
@@ -1030,12 +1035,14 @@
assert v_index.concretetype == lltype.Signed
hop.genop("direct_call", [self.write_barrier_from_array_ptr,
self.c_const_gc,
+ v_newvalue,
v_structaddr,
v_index])
else:
self.write_barrier_calls += 1
hop.genop("direct_call", [self.write_barrier_ptr,
self.c_const_gc,
+ v_newvalue,
v_structaddr])
hop.rename('bare_' + opname)
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gcwrapper.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gcwrapper.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gcwrapper.py Fri Oct 1 22:53:02 2010
@@ -9,7 +9,10 @@
def __init__(self, llinterp, flowgraphs, gc_class, GC_PARAMS={}):
translator = llinterp.typer.annotator.translator
config = translator.config.translation
- self.gc = gc_class(config, chunk_size = 10, **GC_PARAMS)
+ self.gc = gc_class(config,
+ chunk_size = 10,
+ translated_to_c = False,
+ **GC_PARAMS)
self.gc.set_root_walker(LLInterpRootWalker(self))
self.gc.DEBUG = True
self.llinterp = llinterp
@@ -94,6 +97,7 @@
assert (type(index) is int # <- fast path
or lltype.typeOf(index) == lltype.Signed)
self.gc.write_barrier_from_array(
+ llmemory.cast_ptr_to_adr(newvalue),
llmemory.cast_ptr_to_adr(toplevelcontainer),
index)
wb = False
@@ -101,6 +105,7 @@
#
if wb:
self.gc.write_barrier(
+ llmemory.cast_ptr_to_adr(newvalue),
llmemory.cast_ptr_to_adr(toplevelcontainer))
llheap.setinterior(toplevelcontainer, inneraddr, INNERTYPE, newvalue)
Modified: pypy/branch/fast-forward/pypy/rpython/memory/test/test_transformed_gc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/test/test_transformed_gc.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/test/test_transformed_gc.py Fri Oct 1 22:53:02 2010
@@ -906,7 +906,8 @@
gcname = "marksweep"
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
- GC_PARAMS = {'start_heap_size': 1024*WORD }
+ GC_PARAMS = {'start_heap_size': 1024*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
@@ -1144,7 +1145,8 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.marksweep import PrintingMarkSweepGC as GCClass
- GC_PARAMS = {'start_heap_size': 1024*WORD }
+ GC_PARAMS = {'start_heap_size': 1024*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestSemiSpaceGC(GenericMovingGCTests):
@@ -1154,7 +1156,8 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.semispace import SemiSpaceGC as GCClass
- GC_PARAMS = {'space_size': 512*WORD}
+ GC_PARAMS = {'space_size': 512*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestMarkCompactGC(GenericMovingGCTests):
@@ -1163,7 +1166,8 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.markcompact import MarkCompactGC as GCClass
- GC_PARAMS = {'space_size': 4096*WORD}
+ GC_PARAMS = {'space_size': 4096*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestGenerationGC(GenericMovingGCTests):
@@ -1175,7 +1179,8 @@
from pypy.rpython.memory.gc.generation import GenerationGC as \
GCClass
GC_PARAMS = {'space_size': 512*WORD,
- 'nursery_size': 32*WORD}
+ 'nursery_size': 32*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
def define_weakref_across_minor_collection(cls):
@@ -1372,7 +1377,8 @@
GenerationGC._teardown(self)
GC_PARAMS = {'space_size': 512*WORD,
- 'nursery_size': 128*WORD}
+ 'nursery_size': 128*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
def define_working_nursery(cls):
@@ -1404,7 +1410,8 @@
from pypy.rpython.memory.gc.hybrid import HybridGC as GCClass
GC_PARAMS = {'space_size': 512*WORD,
'nursery_size': 32*WORD,
- 'large_object': 8*WORD}
+ 'large_object': 8*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
def define_ref_from_rawmalloced_to_regular(cls):
@@ -1477,6 +1484,7 @@
'large_object': 8*WORD,
'large_object_gcptrs': 10*WORD,
'card_page_indices': 4,
+ 'translated_to_c': False,
}
root_stack_depth = 200
@@ -1585,7 +1593,8 @@
gcname = "marksweep"
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
- GC_PARAMS = {'start_heap_size': 1024*WORD }
+ GC_PARAMS = {'start_heap_size': 1024*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestHybridTaggedPointerGC(TaggedPointerGCTests):
@@ -1596,7 +1605,8 @@
from pypy.rpython.memory.gc.generation import GenerationGC as \
GCClass
GC_PARAMS = {'space_size': 512*WORD,
- 'nursery_size': 32*WORD}
+ 'nursery_size': 32*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestMarkCompactTaggedpointerGC(TaggedPointerGCTests):
@@ -1605,5 +1615,6 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.markcompact import MarkCompactGC as GCClass
- GC_PARAMS = {'space_size': 4096*WORD}
+ GC_PARAMS = {'space_size': 4096*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
Modified: pypy/branch/fast-forward/pypy/translator/driver.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/driver.py (original)
+++ pypy/branch/fast-forward/pypy/translator/driver.py Fri Oct 1 22:53:02 2010
@@ -426,6 +426,22 @@
[OOTYPE],
"JIT compiler generation")
+ def task_jittest_lltype(self):
+ """ Run with the JIT on top of the llgraph backend
+ """
+ # parent process loop: spawn a child, wait for the child to finish,
+ # print a message, and restart
+ from pypy.translator.goal import unixcheckpoint
+ unixcheckpoint.restartable_point(auto='run')
+ # load the module pypy/jit/tl/jittest.py, which you can hack at
+ # and restart without needing to restart the whole translation process
+ from pypy.jit.tl import jittest
+ jittest.jittest(self)
+ #
+ task_jittest_lltype = taskdef(task_jittest_lltype,
+ [RTYPE],
+ "test of the JIT on the llgraph backend")
+
def task_backendopt_lltype(self):
""" Run all backend optimizations - lltype version
"""
@@ -433,7 +449,8 @@
backend_optimizations(self.translator)
#
task_backendopt_lltype = taskdef(task_backendopt_lltype,
- [RTYPE, '??pyjitpl_lltype'],
+ [RTYPE, '??pyjitpl_lltype',
+ '??jittest_lltype'],
"lltype back-end optimisations")
BACKENDOPT = 'backendopt_lltype'
Modified: pypy/branch/fast-forward/pypy/translator/goal/translate.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/goal/translate.py (original)
+++ pypy/branch/fast-forward/pypy/translator/goal/translate.py Fri Oct 1 22:53:02 2010
@@ -27,6 +27,7 @@
("annotate", "do type inference", "-a --annotate", ""),
("rtype", "do rtyping", "-t --rtype", ""),
("pyjitpl", "JIT generation step", "--pyjitpl", ""),
+ ("jittest", "JIT test with llgraph backend", "--jittest", ""),
("backendopt", "do backend optimizations", "--backendopt", ""),
("source", "create source", "-s --source", ""),
("compile", "compile", "-c --compile", " (default goal)"),
From hakanardo at codespeak.net Sat Oct 2 10:31:31 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 2 Oct 2010 10:31:31 +0200 (CEST)
Subject: [pypy-svn] r77545 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test
Message-ID: <20101002083131.A8FFD282BDB@codespeak.net>
Author: hakanardo
Date: Sat Oct 2 10:31:29 2010
New Revision: 77545
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py
Log:
Look ahead on future operations using optimizer.operations no longer suported. test_loop_invariant_mul_ovf passing.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py Sat Oct 2 10:31:29 2010
@@ -9,14 +9,28 @@
"""Keeps track of the bounds placed on integers by the guards and
remove redundant guards"""
+ def setup(self, virtuals):
+ self.posponedop = None
+ self.nextop = None
+
def propagate_forward(self, op):
+ if op.is_ovf():
+ self.posponedop = op
+ return
+ if self.posponedop:
+ self.nextop = op
+ op = self.posponedop
+ self.posponedop = None
+
opnum = op.getopnum()
for value, func in optimize_ops:
if opnum == value:
func(self, op)
break
else:
+ assert not op.is_ovf()
self.emit_operation(op)
+
def propagate_bounds_backward(self, box):
# FIXME: This takes care of the instruction where box is the reuslt
@@ -85,45 +99,45 @@
v2 = self.getvalue(op.getarg(1))
resbound = v1.intbound.add_bound(v2.intbound)
if resbound.has_lower and resbound.has_upper and \
- self.nextop().getopnum() == rop.GUARD_NO_OVERFLOW:
+ self.nextop.getopnum() == rop.GUARD_NO_OVERFLOW:
# Transform into INT_ADD and remove guard
op = op.copy_and_change(rop.INT_ADD)
- self.skip_nextop()
self.optimize_INT_ADD(op) # emit the op
else:
self.emit_operation(op)
r = self.getvalue(op.result)
r.intbound.intersect(resbound)
+ self.emit_operation(self.nextop)
def optimize_INT_SUB_OVF(self, op):
v1 = self.getvalue(op.getarg(0))
v2 = self.getvalue(op.getarg(1))
resbound = v1.intbound.sub_bound(v2.intbound)
if resbound.has_lower and resbound.has_upper and \
- self.nextop().getopnum() == rop.GUARD_NO_OVERFLOW:
+ self.nextop.getopnum() == rop.GUARD_NO_OVERFLOW:
# Transform into INT_SUB and remove guard
op = op.copy_and_change(rop.INT_SUB)
- self.skip_nextop()
self.optimize_INT_SUB(op) # emit the op
else:
self.emit_operation(op)
r = self.getvalue(op.result)
r.intbound.intersect(resbound)
-
+ self.emit_operation(self.nextop)
+
def optimize_INT_MUL_OVF(self, op):
v1 = self.getvalue(op.getarg(0))
v2 = self.getvalue(op.getarg(1))
resbound = v1.intbound.mul_bound(v2.intbound)
if resbound.has_lower and resbound.has_upper and \
- self.nextop().getopnum() == rop.GUARD_NO_OVERFLOW:
+ self.nextop.getopnum() == rop.GUARD_NO_OVERFLOW:
# Transform into INT_MUL and remove guard
op = op.copy_and_change(rop.INT_MUL)
- self.skip_nextop()
self.optimize_INT_MUL(op) # emit the op
else:
self.emit_operation(op)
r = self.getvalue(op.result)
r.intbound.intersect(resbound)
+ self.emit_operation(self.nextop)
def optimize_INT_LT(self, op):
v1 = self.getvalue(op.getarg(0))
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Sat Oct 2 10:31:29 2010
@@ -182,12 +182,6 @@
op = ResOperation(opnum, args, result)
self.optimizer.pure_operations[self.optimizer.make_args_key(op)] = op
- def nextop(self):
- return self.optimizer.loop.operations[self.optimizer.i + 1]
-
- def skip_nextop(self):
- self.optimizer.i += 1
-
def setup(self, virtuals):
pass
@@ -205,6 +199,7 @@
self.pure_operations = args_dict()
self.producer = {}
self.pendingfields = []
+ self.posponedop = None
if optimizations:
self.first_optimization = optimizations[0]
@@ -254,9 +249,9 @@
return constbox
return None
- def make_equal_to(self, box, value):
+ def make_equal_to(self, box, value, replace=False):
assert isinstance(value, OptValue)
- assert box not in self.values
+ assert replace or box not in self.values
self.values[box] = value
def make_constant(self, box, constbox):
@@ -392,10 +387,17 @@
def optimize_default(self, op):
canfold = op.is_always_pure()
- is_ovf = op.is_ovf()
- if is_ovf:
- nextop = self.loop.operations[self.i + 1]
+ if op.is_ovf():
+ self.posponedop = op
+ return
+ if self.posponedop:
+ nextop = op
+ op = self.posponedop
+ self.posponedop = None
canfold = nextop.getopnum() == rop.GUARD_NO_OVERFLOW
+ else:
+ nextop = None
+
if canfold:
for i in range(op.numargs()):
if self.get_constant_box(op.getarg(i)) is None:
@@ -406,9 +408,8 @@
for i in range(op.numargs())]
resbox = execute_nonspec(self.cpu, None,
op.getopnum(), argboxes, op.getdescr())
+ # FIXME: Don't we need to check for an overflow here?
self.make_constant(op.result, resbox.constbox())
- if is_ovf:
- self.i += 1 # skip next operation, it is the unneeded guard
return
# did we do the exact same operation already?
@@ -416,20 +417,22 @@
oldop = self.pure_operations.get(args, None)
if oldop is not None and oldop.getdescr() is op.getdescr():
assert oldop.getopnum() == op.getopnum()
- self.make_equal_to(op.result, self.getvalue(oldop.result))
- if is_ovf:
- self.i += 1 # skip next operation, it is the unneeded guard
+ self.make_equal_to(op.result, self.getvalue(oldop.result),
+ True)
return
else:
self.pure_operations[args] = op
# otherwise, the operation remains
self.emit_operation(op)
+ if nextop:
+ self.emit_operation(nextop)
- def optimize_GUARD_NO_OVERFLOW(self, op):
- # otherwise the default optimizer will clear fields, which is unwanted
- # in this case
- self.emit_operation(op)
+ #def optimize_GUARD_NO_OVERFLOW(self, op):
+ # # otherwise the default optimizer will clear fields, which is unwanted
+ # # in this case
+ # self.emit_operation(op)
+ # FIXME: Is this still needed?
def optimize_DEBUG_MERGE_POINT(self, op):
self.emit_operation(op)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py Sat Oct 2 10:31:29 2010
@@ -334,11 +334,11 @@
return res
res = self.meta_interp(f, [6, 7])
assert res == 308
- self.check_loop_count(1)
- self.check_loops({'guard_true': 1, 'guard_no_overflow': 1,
- 'int_add': 2, 'int_sub': 1, 'int_gt': 1,
- 'int_mul': 1, 'int_mul_ovf': 1,
- 'jump': 1})
+ self.check_loop_count(2)
+ self.check_loops({'guard_true': 2, 'guard_no_overflow': 1,
+ 'int_add': 4, 'int_sub': 2, 'int_gt': 2,
+ 'int_mul': 2, 'int_mul_ovf': 1,
+ 'jump': 2})
def test_loop_invariant_mul_bridge1(self):
myjitdriver = JitDriver(greens = [], reds = ['y', 'res', 'x'])
@@ -364,12 +364,12 @@
myjitdriver.can_enter_jit(x=x, y=y, res=res)
myjitdriver.jit_merge_point(x=x, y=y, res=res)
res += x * x
- if y<8:
+ if y<16:
res += 1
y -= 1
return res
- res = self.meta_interp(f, [6, 16])
- assert res == 583
+ res = self.meta_interp(f, [6, 32])
+ assert res == 1167
self.check_loop_count(3)
self.check_loops({'int_lt': 1, 'int_gt': 1,
'guard_false': 1, 'guard_true': 1,
From afa at codespeak.net Sat Oct 2 12:15:33 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Sat, 2 Oct 2010 12:15:33 +0200 (CEST)
Subject: [pypy-svn] r77546 - pypy/branch/fast-forward/lib-python
Message-ID: <20101002101533.8B0CB282BAD@codespeak.net>
Author: afa
Date: Sat Oct 2 12:15:32 2010
New Revision: 77546
Modified:
pypy/branch/fast-forward/lib-python/TODO
Log:
More TODO tasks
Modified: pypy/branch/fast-forward/lib-python/TODO
==============================================================================
--- pypy/branch/fast-forward/lib-python/TODO (original)
+++ pypy/branch/fast-forward/lib-python/TODO Sat Oct 2 12:15:32 2010
@@ -28,6 +28,8 @@
- (list|str|unicode|bytearray).(index|find) should accept None as indices
+- Fix fcntl.fcntl(fd, fcntl.F_NOTIFY, fcntl.DN_MULTISHOT) on 32bit platform.
+
Medium tasks
------------
@@ -37,6 +39,8 @@
- "exceptions must be old-style classes or derived from BaseException, not str"
in the 'raise' statement and generator.throw()
+- missing builtin: memoryview
+
Longer tasks
------------
From getxsick at codespeak.net Sun Oct 3 16:45:23 2010
From: getxsick at codespeak.net (getxsick at codespeak.net)
Date: Sun, 3 Oct 2010 16:45:23 +0200 (CEST)
Subject: [pypy-svn] r77550 - pypy/extradoc/sprintinfo/ddorf2010
Message-ID: <20101003144523.12FEA282BAD@codespeak.net>
Author: getxsick
Date: Sun Oct 3 16:45:21 2010
New Revision: 77550
Modified:
pypy/extradoc/sprintinfo/ddorf2010/announce.txt
Log:
fix url
Modified: pypy/extradoc/sprintinfo/ddorf2010/announce.txt
==============================================================================
--- pypy/extradoc/sprintinfo/ddorf2010/announce.txt (original)
+++ pypy/extradoc/sprintinfo/ddorf2010/announce.txt Sun Oct 3 16:45:21 2010
@@ -30,4 +30,4 @@
you have codespeak commit rights).
.. _`pypy-sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint
-.. _`people`: http://codespeak.net/pypy/extradoc/sprintinfo/ddorf2010/people.txt
+.. _`people`: http://codespeak.net/svn/pypy/extradoc/sprintinfo/ddorf2010/people.txt
From afa at codespeak.net Sun Oct 3 22:28:01 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Sun, 3 Oct 2010 22:28:01 +0200 (CEST)
Subject: [pypy-svn] r77551 - pypy/branch/fast-forward/pypy/interpreter/test
Message-ID: <20101003202801.3D7F2282BAD@codespeak.net>
Author: afa
Date: Sun Oct 3 22:27:59 2010
New Revision: 77551
Modified:
pypy/branch/fast-forward/pypy/interpreter/test/test_argument.py
Log:
Add methods to the DummySpace to let the tests pass
Modified: pypy/branch/fast-forward/pypy/interpreter/test/test_argument.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/test/test_argument.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/test/test_argument.py Sun Oct 3 22:27:59 2010
@@ -98,12 +98,27 @@
def isinstance(self, obj, cls):
return isinstance(obj, cls)
+ isinstance_w = isinstance
def exception_match(self, w_type1, w_type2):
return issubclass(w_type1, w_type2)
+ def call_method(self, obj, name, *args):
+ try:
+ method = getattr(obj, name)
+ except AttributeError:
+ raise OperationError(AttributeError, name)
+ return method(*args)
+
+ def type(self, obj):
+ class Type:
+ def getname(self, space, default):
+ return type(obj).__name__
+ return Type()
+
w_TypeError = TypeError
+ w_AttributeError = AttributeError
w_dict = dict
class TestArgumentsNormal(object):
From afa at codespeak.net Sun Oct 3 22:42:56 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Sun, 3 Oct 2010 22:42:56 +0200 (CEST)
Subject: [pypy-svn] r77552 - pypy/branch/fast-forward/pypy/module/_rawffi
Message-ID: <20101003204256.754A3282BAD@codespeak.net>
Author: afa
Date: Sun Oct 3 22:42:55 2010
New Revision: 77552
Modified:
pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py
Log:
Typecode '?' is now used for the Bool type.
Change the default to '\0'
Modified: pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py (original)
+++ pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py Sun Oct 3 22:42:55 2010
@@ -243,7 +243,7 @@
_array_shapes = None
size = 0
alignment = 0
- itemcode = '?'
+ itemcode = '\0'
def allocate(self, space, length, autofree=False):
raise NotImplementedError
From afa at codespeak.net Sun Oct 3 23:04:23 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Sun, 3 Oct 2010 23:04:23 +0200 (CEST)
Subject: [pypy-svn] r77553 -
pypy/branch/fast-forward/pypy/interpreter/astcompiler/test
Message-ID: <20101003210423.2A783282BAD@codespeak.net>
Author: afa
Date: Sun Oct 3 23:04:21 2010
New Revision: 77553
Modified:
pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_compiler.py
Log:
Reverse the test, now that pypy uses 2.7 bytecodes
Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_compiler.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_compiler.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_compiler.py Sun Oct 3 23:04:21 2010
@@ -27,8 +27,8 @@
space = self.space
code = compile_with_astcompiler(source, 'exec', space)
# 2.7 bytecode is too different, the standard `dis` module crashes
- # when trying to display pypy (2.5-like) bytecode.
- if sys.version_info < (2, 7):
+ # on older cpython versions
+ if sys.version_info >= (2, 7):
print
code.dump()
w_dict = space.newdict()
From afa at codespeak.net Mon Oct 4 08:24:07 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 4 Oct 2010 08:24:07 +0200 (CEST)
Subject: [pypy-svn] r77554 -
pypy/branch/fast-forward/pypy/module/_rawffi/test
Message-ID: <20101004062407.6D4F1282BEA@codespeak.net>
Author: afa
Date: Mon Oct 4 08:24:04 2010
New Revision: 77554
Modified:
pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py
Log:
Fix test of repr()
Modified: pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py (original)
+++ pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py Mon Oct 4 08:24:04 2010
@@ -710,7 +710,7 @@
# fragile
S = _rawffi.Structure([('x', 'c'), ('y', 'l')])
assert (repr(_rawffi.Array((S, 2))) ==
- "<_rawffi.Array '?' (%d, %d)>" % (4*lsize, lsize))
+ "<_rawffi.Array '\0' (%d, %d)>" % (4*lsize, lsize))
assert (repr(_rawffi.Structure([('x', 'i'), ('yz', 'i')])) ==
"<_rawffi.Structure 'x' 'yz' (%d, %d)>" % (2*isize, isize))
From antocuni at codespeak.net Mon Oct 4 10:10:37 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 4 Oct 2010 10:10:37 +0200 (CEST)
Subject: [pypy-svn] r77555 - in pypy/branch/jitffi/pypy: jit/metainterp/test
rlib rlib/test
Message-ID: <20101004081037.23D95282BAD@codespeak.net>
Author: antocuni
Date: Mon Oct 4 10:10:35 2010
New Revision: 77555
Added:
pypy/branch/jitffi/pypy/rlib/test/test_libffi.py (contents, props changed)
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py
pypy/branch/jitffi/pypy/rlib/libffi.py
Log:
add a more user-friendly interface to build argchains. Moreover, checkin test_libffi, which I forgot to svn add earlier :-(
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py Mon Oct 4 10:10:35 2010
@@ -2,7 +2,7 @@
import py
from pypy.rlib.jit import JitDriver, hint
from pypy.jit.metainterp.test.test_basic import LLJitMixin
-from pypy.rlib.libffi import CDLL, ffi_type_sint, IntArg, Func
+from pypy.rlib.libffi import CDLL, ffi_type_sint, ArgChain, Func
from pypy.tool.udir import udir
from pypy.translator.tool.cbuild import ExternalCompilationInfo
from pypy.translator.platform import platform
@@ -34,10 +34,9 @@
driver.jit_merge_point(n=n, func=func)
driver.can_enter_jit(n=n, func=func)
func = hint(func, promote=True)
- arg0 = IntArg(n)
- arg1 = IntArg(1)
- arg0.next = arg1
- n = func.call(arg0, lltype.Signed)
+ argchain = ArgChain()
+ argchain.int(n).int(1)
+ n = func.call(argchain, lltype.Signed)
return n
res = self.meta_interp(f, [0])
Modified: pypy/branch/jitffi/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/libffi.py Mon Oct 4 10:10:35 2010
@@ -18,6 +18,28 @@
# ----------------------------------------------------------------------
+class ArgChain(object):
+ first = None
+ last = None
+ numargs = 0
+
+ def int(self, intval):
+ self._append(IntArg(intval))
+ return self
+
+ def float(self, floatval):
+ self._append(FloatArg(floatval))
+ return self
+
+ def _append(self, arg):
+ if self.first is None:
+ self.first = self.last = arg
+ else:
+ self.last.next = arg
+ self.last = arg
+ self.numargs += 1
+
+
class AbstractArg(object):
next = None
@@ -116,7 +138,7 @@
# assuming that archain is completely virtual.
ll_args = self._prepare()
i = 0
- arg = argchain
+ arg = argchain.first
while arg:
arg.push(self, ll_args, i)
i += 1
Added: pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
==============================================================================
--- (empty file)
+++ pypy/branch/jitffi/pypy/rlib/test/test_libffi.py Mon Oct 4 10:10:35 2010
@@ -0,0 +1,57 @@
+import py
+import sys
+from pypy.rpython.lltypesystem import rffi, lltype
+from pypy.rpython.lltypesystem.ll2ctypes import ALLOCATED
+from pypy.rlib.test.test_clibffi import BaseFfiTest, get_libm_name
+from pypy.rlib.libffi import CDLL, Func, get_libc_name, ArgChain
+from pypy.rlib.libffi import ffi_type_double, ffi_type_void
+
+
+class TestLibffi(BaseFfiTest):
+ """
+ Test the new JIT-friendly interface to libffi
+ """
+
+ def get_libc(self):
+ return CDLL(get_libc_name())
+
+ def get_libm(self):
+ return CDLL(get_libm_name(sys.platform))
+
+ def test_argchain(self):
+ chain = ArgChain()
+ assert chain.numargs == 0
+ chain2 = chain.int(42)
+ assert chain2 is chain
+ assert chain.numargs == 1
+ intarg = chain.first
+ assert chain.last is intarg
+ assert intarg.intval == 42
+ chain.float(123.45)
+ assert chain.numargs == 2
+ assert chain.first is intarg
+ assert intarg.next is chain.last
+ floatarg = intarg.next
+ assert floatarg.floatval == 123.45
+
+ def test_library_open(self):
+ lib = self.get_libc()
+ del lib
+ assert not ALLOCATED
+
+ def test_library_get_func(self):
+ lib = self.get_libc()
+ ptr = lib.getpointer('fopen', [], ffi_type_void)
+ py.test.raises(KeyError, lib.getpointer, 'xxxxxxxxxxxxxxx', [], ffi_type_void)
+ del ptr
+ del lib
+ assert not ALLOCATED
+
+ def test_call_argchain(self):
+ libm = self.get_libm()
+ pow = libm.getpointer('pow', [ffi_type_double, ffi_type_double],
+ ffi_type_double)
+ argchain = ArgChain()
+ argchain.float(2.0).float(3.0)
+ res = pow.call(argchain, rffi.DOUBLE)
+ assert res == 8.0
From antocuni at codespeak.net Mon Oct 4 10:14:56 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 4 Oct 2010 10:14:56 +0200 (CEST)
Subject: [pypy-svn] r77556 - pypy/branch/jitffi/pypy/rlib
Message-ID: <20101004081456.84117282BAD@codespeak.net>
Author: antocuni
Date: Mon Oct 4 10:14:55 2010
New Revision: 77556
Modified:
pypy/branch/jitffi/pypy/rlib/libffi.py
Log:
move the public interface (well, just one method) to the top of the class
Modified: pypy/branch/jitffi/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/libffi.py Mon Oct 4 10:14:55 2010
@@ -74,6 +74,37 @@
self.keepalive = keepalive
self.funcsym = funcsym
+ @jit.unroll_safe
+ @specialize.arg(2)
+ def call(self, argchain, RESULT):
+ # WARNING! This code is written carefully in a way that the JIT
+ # optimizer will see a sequence of calls like the following:
+ #
+ # libffi_prepare_call
+ # libffi_push_arg
+ # libffi_push_arg
+ # ...
+ # libffi_call
+ #
+ # It is important that there is no other operation in the middle, else
+ # the optimizer will fail to recognize the pattern and won't turn it
+ # into a fast CALL. Note that "arg = arg.next" is optimized away,
+ # assuming that archain is completely virtual.
+ ll_args = self._prepare()
+ i = 0
+ arg = argchain.first
+ while arg:
+ arg.push(self, ll_args, i)
+ i += 1
+ arg = arg.next
+ result = self._do_call(self.funcsym, ll_args, RESULT)
+ return result
+
+ # END OF THE PUBLIC INTERFACE
+ # -------------------------------------------------------------------------
+ # the following methods are supposed to be seen opaquely by the JIT
+ # optimizer. Don't call them
+
def _prepare(self):
ll_args = lltype.malloc(rffi.VOIDPP.TO, len(self.argtypes), flavor='raw')
return ll_args
@@ -120,31 +151,6 @@
lltype.free(ll_args[i], flavor='raw')
lltype.free(ll_args, flavor='raw')
- @jit.unroll_safe
- @specialize.arg(2)
- def call(self, argchain, RESULT):
- # WARNING! This code is written carefully in a way that the JIT
- # optimizer will see a sequence of calls like the following:
- #
- # libffi_prepare_call
- # libffi_push_arg
- # libffi_push_arg
- # ...
- # libffi_call
- #
- # It is important that there is no other operation in the middle, else
- # the optimizer will fail to recognize the pattern and won't turn it
- # into a fast CALL. Note that "arg = arg.next" is optimized away,
- # assuming that archain is completely virtual.
- ll_args = self._prepare()
- i = 0
- arg = argchain.first
- while arg:
- arg.push(self, ll_args, i)
- i += 1
- arg = arg.next
- result = self._do_call(self.funcsym, ll_args, RESULT)
- return result
# ----------------------------------------------------------------------
From afa at codespeak.net Mon Oct 4 10:21:55 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 4 Oct 2010 10:21:55 +0200 (CEST)
Subject: [pypy-svn] r77557 -
pypy/branch/fast-forward/lib-python/modified-2.7.0
Message-ID: <20101004082155.66493282BAD@codespeak.net>
Author: afa
Date: Mon Oct 4 10:21:54 2010
New Revision: 77557
Added:
pypy/branch/fast-forward/lib-python/modified-2.7.0/pickle.py
- copied, changed from r77554, pypy/branch/fast-forward/lib-python/2.7.0/pickle.py
Log:
Allow pickling of functions, this fixes greenlet tests.
Same change as in modified-2.5.2
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/pickle.py (from r77554, pypy/branch/fast-forward/lib-python/2.7.0/pickle.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/pickle.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/pickle.py Mon Oct 4 10:21:54 2010
@@ -727,6 +727,29 @@
dispatch[InstanceType] = save_inst
+ def save_function(self, obj):
+ try:
+ return self.save_global(obj)
+ except PicklingError, e:
+ pass
+ # Check copy_reg.dispatch_table
+ reduce = dispatch_table.get(type(obj))
+ if reduce:
+ rv = reduce(obj)
+ else:
+ # Check for a __reduce_ex__ method, fall back to __reduce__
+ reduce = getattr(obj, "__reduce_ex__", None)
+ if reduce:
+ rv = reduce(self.proto)
+ else:
+ reduce = getattr(obj, "__reduce__", None)
+ if reduce:
+ rv = reduce()
+ else:
+ raise e
+ return self.save_reduce(obj=obj, *rv)
+ dispatch[FunctionType] = save_function
+
def save_global(self, obj, name=None, pack=struct.pack):
write = self.write
memo = self.memo
@@ -768,7 +791,6 @@
self.memoize(obj)
dispatch[ClassType] = save_global
- dispatch[FunctionType] = save_global
dispatch[BuiltinFunctionType] = save_global
dispatch[TypeType] = save_global
From afa at codespeak.net Mon Oct 4 10:40:39 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 4 Oct 2010 10:40:39 +0200 (CEST)
Subject: [pypy-svn] r77558 - pypy/trunk/pypy/module/parser/test
Message-ID: <20101004084039.82D3B282BAD@codespeak.net>
Author: afa
Date: Mon Oct 4 10:40:38 2010
New Revision: 77558
Modified:
pypy/trunk/pypy/module/parser/test/test_parser.py
Log:
Avoid hard-coded constant in test
Modified: pypy/trunk/pypy/module/parser/test/test_parser.py
==============================================================================
--- pypy/trunk/pypy/module/parser/test/test_parser.py (original)
+++ pypy/trunk/pypy/module/parser/test/test_parser.py Mon Oct 4 10:40:38 2010
@@ -10,6 +10,9 @@
cls.w_m = space.appexec([], """():
import parser
return parser""")
+ cls.w_symbol = space.appexec([], """():
+ import symbol
+ return symbol""")
class AppTestParser(ParserModuleTest):
@@ -36,7 +39,7 @@
seq = getattr(s, meth)()
assert isinstance(seq, tp)
assert len(seq) == 4
- assert seq[0] == 286
+ assert seq[0] == self.symbol.file_input
assert len(seq[2]) == 2
assert len(seq[3]) == 2
assert seq[2][0] == 4
From afa at codespeak.net Mon Oct 4 10:42:02 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 4 Oct 2010 10:42:02 +0200 (CEST)
Subject: [pypy-svn] r77559 - pypy/branch/fast-forward/lib_pypy
Message-ID: <20101004084202.DACBC282BAD@codespeak.net>
Author: afa
Date: Mon Oct 4 10:42:01 2010
New Revision: 77559
Modified:
pypy/branch/fast-forward/lib_pypy/_subprocess.py
Log:
Add constant needed by the win32 implementation of subprocess.
Modified: pypy/branch/fast-forward/lib_pypy/_subprocess.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/_subprocess.py (original)
+++ pypy/branch/fast-forward/lib_pypy/_subprocess.py Mon Oct 4 10:42:01 2010
@@ -162,7 +162,8 @@
STARTF_USESHOWWINDOW = 0x001
STARTF_USESTDHANDLES = 0x100
SW_HIDE = 0
-CREATE_NEW_CONSOLE = 0x010
+CREATE_NEW_CONSOLE = 0x010
+CREATE_NEW_PROCESS_GROUP = 0x200
def WaitForSingleObject(handle, milliseconds):
res = _WaitForSingleObject(handle.handle, milliseconds)
From antocuni at codespeak.net Mon Oct 4 11:24:15 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 4 Oct 2010 11:24:15 +0200 (CEST)
Subject: [pypy-svn] r77560 - in pypy/branch/jitffi/pypy: jit/codewriter
jit/metainterp/optimizeopt rlib
Message-ID: <20101004092415.5A163282BAD@codespeak.net>
Author: antocuni
Date: Mon Oct 4 11:24:13 2010
New Revision: 77560
Modified:
pypy/branch/jitffi/pypy/jit/codewriter/support.py
pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
pypy/branch/jitffi/pypy/rlib/libffi.py
Log:
use separate methods for pushing ints and floats. This is needed because
oopspec and specialize don't really play well togheter
Modified: pypy/branch/jitffi/pypy/jit/codewriter/support.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/codewriter/support.py (original)
+++ pypy/branch/jitffi/pypy/jit/codewriter/support.py Mon Oct 4 11:24:13 2010
@@ -222,20 +222,21 @@
# libffi support
# --------------
-def _ll_1_libffi_prepare_call(llfunc):
+def func(llfunc):
from pypy.rlib.libffi import Func
- func = cast_base_ptr_to_instance(Func, llfunc)
- return func._prepare()
+ return cast_base_ptr_to_instance(Func, llfunc)
-def _ll_4_libffi_push_arg(llfunc, value, ll_args, i):
- from pypy.rlib.libffi import Func
- func = cast_base_ptr_to_instance(Func, llfunc)
- return func._push_arg(value, ll_args, i)
+def _ll_1_libffi_prepare_call(llfunc):
+ return func(llfunc)._prepare()
+
+def _ll_4_libffi_push_int(llfunc, value, ll_args, i):
+ return func(llfunc)._push_int(value, ll_args, i)
+
+def _ll_4_libffi_push_float(llfunc, value, ll_args, i):
+ return func(llfunc)._push_float(value, ll_args, i)
def _ll_4_libffi_call(llfunc, funcsym, ll_args, RESULT):
- from pypy.rlib.libffi import Func
- func = cast_base_ptr_to_instance(Func, llfunc)
- return func._do_call(funcsym, ll_args, lltype.Signed)
+ return func(llfunc)._do_call(funcsym, ll_args, lltype.Signed)
# XXX: should be RESULT, but it doesn't work
Modified: pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py Mon Oct 4 11:24:13 2010
@@ -63,7 +63,7 @@
funcname = str(funcval.box)
if '_libffi_prepare_call' in funcname:
return 'prepare_call'
- elif '_libffi_push_arg' in funcname:
+ elif '_libffi_push_' in funcname:
return 'push_arg'
elif '_libffi_call' in funcname:
return 'call'
Modified: pypy/branch/jitffi/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/libffi.py Mon Oct 4 11:24:13 2010
@@ -51,7 +51,7 @@
self.intval = intval
def push(self, func, ll_args, i):
- func._push_arg(self.intval, ll_args, i)
+ func._push_int(self.intval, ll_args, i)
class FloatArg(AbstractArg):
""" An argument holding a float
@@ -61,7 +61,7 @@
self.floatval = floatval
def push(self, func, ll_args, i):
- func._push_arg(self.floatval, ll_args, i)
+ func._push_float(self.floatval, ll_args, i)
class Func(AbstractFuncPtr):
@@ -110,18 +110,23 @@
return ll_args
_prepare.oopspec = 'libffi_prepare_call(self)'
- def _push_arg(self, value, ll_args, i):
+ @specialize.arg(1)
+ def _push_arg(self, TYPE, value, ll_args, i):
# XXX: check the type is not translated?
argtype = self.argtypes[i]
c_size = intmask(argtype.c_size)
ll_buf = lltype.malloc(rffi.CCHARP.TO, c_size, flavor='raw')
push_arg_as_ffiptr(argtype, value, ll_buf)
ll_args[i] = ll_buf
- # XXX this is bad, fix it somehow in the future, but specialize:argtype
- # doesn't work correctly with mixing non-negative and normal integers
- _push_arg._annenforceargs_ = [None, int, None, int]
- #_push_arg._annspecialcase_ = 'specialize:argtype(1)'
- _push_arg.oopspec = 'libffi_push_arg(self, value, ll_args, i)'
+
+ def _push_int(self, value, ll_args, i):
+ self._push_arg(lltype.Signed, value, ll_args, i)
+ _push_int.oopspec = 'libffi_push_int(self, value, ll_args, i)'
+ _push_int._annenforceargs_ = [None, int, None, int]
+
+ def _push_float(self, value, ll_args, i):
+ self._push_arg(lltype.Float, value, ll_args, i)
+ _push_float.oopspec = 'libffi_push_float(self, value, ll_args, i)'
def _do_call(self, funcsym, ll_args, RESULT):
# XXX: check len(args)?
From antocuni at codespeak.net Mon Oct 4 11:29:01 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 4 Oct 2010 11:29:01 +0200 (CEST)
Subject: [pypy-svn] r77561 - in pypy/branch/jitffi/pypy/rlib: . test
Message-ID: <20101004092901.D1C07282BAD@codespeak.net>
Author: antocuni
Date: Mon Oct 4 11:29:00 2010
New Revision: 77561
Modified:
pypy/branch/jitffi/pypy/rlib/jit.py
pypy/branch/jitffi/pypy/rlib/libffi.py
pypy/branch/jitffi/pypy/rlib/test/test_jit.py
Log:
introduce a new decorator to set oopspec, and use it in libffi.py
Modified: pypy/branch/jitffi/pypy/rlib/jit.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/jit.py (original)
+++ pypy/branch/jitffi/pypy/rlib/jit.py Mon Oct 4 11:29:00 2010
@@ -77,6 +77,12 @@
return result
return decorator
+def oopspec(spec):
+ def decorator(func):
+ func.oopspec = spec
+ return func
+ return decorator
+
class Entry(ExtRegistryEntry):
_about_ = hint
Modified: pypy/branch/jitffi/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/libffi.py Mon Oct 4 11:29:00 2010
@@ -105,10 +105,10 @@
# the following methods are supposed to be seen opaquely by the JIT
# optimizer. Don't call them
+ @jit.oopspec('libffi_prepare_call(self)')
def _prepare(self):
ll_args = lltype.malloc(rffi.VOIDPP.TO, len(self.argtypes), flavor='raw')
return ll_args
- _prepare.oopspec = 'libffi_prepare_call(self)'
@specialize.arg(1)
def _push_arg(self, TYPE, value, ll_args, i):
@@ -119,15 +119,17 @@
push_arg_as_ffiptr(argtype, value, ll_buf)
ll_args[i] = ll_buf
+ @jit.oopspec('libffi_push_int(self, value, ll_args, i)')
def _push_int(self, value, ll_args, i):
self._push_arg(lltype.Signed, value, ll_args, i)
- _push_int.oopspec = 'libffi_push_int(self, value, ll_args, i)'
_push_int._annenforceargs_ = [None, int, None, int]
+ @jit.oopspec('libffi_push_float(self, value, ll_args, i)')
def _push_float(self, value, ll_args, i):
self._push_arg(lltype.Float, value, ll_args, i)
- _push_float.oopspec = 'libffi_push_float(self, value, ll_args, i)'
+ @specialize.arg(3)
+ @jit.oopspec('libffi_call(self, funcsym, ll_args, RESULT)')
def _do_call(self, funcsym, ll_args, RESULT):
# XXX: check len(args)?
ll_result = lltype.nullptr(rffi.CCHARP.TO)
@@ -147,8 +149,6 @@
self._free_buffers(ll_result, ll_args)
#check_fficall_result(ffires, self.flags)
return res
- _do_call._annspecialcase_ = 'specialize:arg(3)'
- _do_call.oopspec = 'libffi_call(self, funcsym, ll_args, RESULT)'
def _free_buffers(self, ll_result, ll_args):
lltype.free(ll_result, flavor='raw')
Modified: pypy/branch/jitffi/pypy/rlib/test/test_jit.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_jit.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_jit.py Mon Oct 4 11:29:00 2010
@@ -1,10 +1,16 @@
import py
from pypy.rlib.jit import hint, we_are_jitted, JitDriver, purefunction_promote
-from pypy.rlib.jit import JitHintError
+from pypy.rlib.jit import JitHintError, oopspec
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.rpython.lltypesystem import lltype
+def test_oopspec():
+ @oopspec('foobar')
+ def fn():
+ pass
+ assert fn.oopspec == 'foobar'
+
class BaseTestJIT(BaseRtypingTest):
def test_hint(self):
def f():
From antocuni at codespeak.net Mon Oct 4 11:38:57 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 4 Oct 2010 11:38:57 +0200 (CEST)
Subject: [pypy-svn] r77562 - in pypy/branch/jitffi/pypy: jit/metainterp/test
rlib
Message-ID: <20101004093857.C9F20282BAD@codespeak.net>
Author: antocuni
Date: Mon Oct 4 11:38:56 2010
New Revision: 77562
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py
pypy/branch/jitffi/pypy/rlib/libffi.py
Log:
actually test float args
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py Mon Oct 4 11:38:56 2010
@@ -2,7 +2,7 @@
import py
from pypy.rlib.jit import JitDriver, hint
from pypy.jit.metainterp.test.test_basic import LLJitMixin
-from pypy.rlib.libffi import CDLL, ffi_type_sint, ArgChain, Func
+from pypy.rlib.libffi import CDLL, ffi_type_sint, ffi_type_double, ArgChain, Func
from pypy.tool.udir import udir
from pypy.translator.tool.cbuild import ExternalCompilationInfo
from pypy.translator.platform import platform
@@ -14,9 +14,9 @@
# it via rlib.libffi
c_file = udir.ensure("test_jit_direct_call", dir=1).join("xlib.c")
c_file.write(py.code.Source('''
- int sum_xy(int x, int y)
+ int sum_xy(int x, double y)
{
- return (x + y);
+ return (x + (int)y);
}
'''))
eci = ExternalCompilationInfo(export_symbols=['sum_xy'])
@@ -28,14 +28,14 @@
def f(n):
cdll = CDLL(self.lib_name)
- func = cdll.getpointer('sum_xy', [ffi_type_sint, ffi_type_sint],
+ func = cdll.getpointer('sum_xy', [ffi_type_sint, ffi_type_double],
ffi_type_sint)
while n < 10:
driver.jit_merge_point(n=n, func=func)
driver.can_enter_jit(n=n, func=func)
func = hint(func, promote=True)
argchain = ArgChain()
- argchain.int(n).int(1)
+ argchain.int(n).float(1.2)
n = func.call(argchain, lltype.Signed)
return n
Modified: pypy/branch/jitffi/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/libffi.py Mon Oct 4 11:38:56 2010
@@ -1,5 +1,5 @@
from pypy.rpython.lltypesystem import rffi, lltype
-from pypy.rlib.objectmodel import specialize
+from pypy.rlib.objectmodel import specialize, enforceargs
from pypy.rlib.rarithmetic import intmask, r_uint
from pypy.rlib import jit
from pypy.rlib import clibffi
@@ -120,11 +120,12 @@
ll_args[i] = ll_buf
@jit.oopspec('libffi_push_int(self, value, ll_args, i)')
+ @enforceargs( None, int, None, int) # fix the annotation for tests
def _push_int(self, value, ll_args, i):
self._push_arg(lltype.Signed, value, ll_args, i)
- _push_int._annenforceargs_ = [None, int, None, int]
@jit.oopspec('libffi_push_float(self, value, ll_args, i)')
+ @enforceargs( None, float, None, int) # fix the annotation for tests
def _push_float(self, value, ll_args, i):
self._push_arg(lltype.Float, value, ll_args, i)
From afa at codespeak.net Mon Oct 4 11:40:31 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 4 Oct 2010 11:40:31 +0200 (CEST)
Subject: [pypy-svn] r77564 - pypy/branch/fast-forward/pypy/module/parser/test
Message-ID: <20101004094031.38AF0282BAD@codespeak.net>
Author: afa
Date: Mon Oct 4 11:40:29 2010
New Revision: 77564
Modified:
pypy/branch/fast-forward/pypy/module/parser/test/test_parser.py
Log:
Merge r77558 from trunk
Modified: pypy/branch/fast-forward/pypy/module/parser/test/test_parser.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/parser/test/test_parser.py (original)
+++ pypy/branch/fast-forward/pypy/module/parser/test/test_parser.py Mon Oct 4 11:40:29 2010
@@ -10,6 +10,9 @@
cls.w_m = space.appexec([], """():
import parser
return parser""")
+ cls.w_symbol = space.appexec([], """():
+ import symbol
+ return symbol""")
class AppTestParser(ParserModuleTest):
@@ -36,7 +39,7 @@
seq = getattr(s, meth)()
assert isinstance(seq, tp)
assert len(seq) == 4
- assert seq[0] == 286
+ assert seq[0] == self.symbol.file_input
assert len(seq[2]) == 2
assert len(seq[3]) == 2
assert seq[2][0] == 4
From antocuni at codespeak.net Mon Oct 4 13:48:30 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 4 Oct 2010 13:48:30 +0200 (CEST)
Subject: [pypy-svn] r77566 - in pypy/branch/jitffi/pypy: jit/codewriter rlib
Message-ID: <20101004114830.A1DD0282BAD@codespeak.net>
Author: antocuni
Date: Mon Oct 4 13:48:29 2010
New Revision: 77566
Modified:
pypy/branch/jitffi/pypy/jit/codewriter/support.py
pypy/branch/jitffi/pypy/rlib/libffi.py
Log:
manually specialize _do_call into _do_call_{int,float}. This is needed because oopspec does not play well with specialize()
Modified: pypy/branch/jitffi/pypy/jit/codewriter/support.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/codewriter/support.py (original)
+++ pypy/branch/jitffi/pypy/jit/codewriter/support.py Mon Oct 4 13:48:29 2010
@@ -235,9 +235,11 @@
def _ll_4_libffi_push_float(llfunc, value, ll_args, i):
return func(llfunc)._push_float(value, ll_args, i)
-def _ll_4_libffi_call(llfunc, funcsym, ll_args, RESULT):
+def _ll_3_libffi_call_int(llfunc, funcsym, ll_args):
return func(llfunc)._do_call(funcsym, ll_args, lltype.Signed)
-# XXX: should be RESULT, but it doesn't work
+
+def _ll_3_libffi_call_float(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, lltype.Float)
Modified: pypy/branch/jitffi/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/libffi.py Mon Oct 4 13:48:29 2010
@@ -74,6 +74,10 @@
self.keepalive = keepalive
self.funcsym = funcsym
+ # ========================================================================
+ # PUBLIC INTERFACE
+ # ========================================================================
+
@jit.unroll_safe
@specialize.arg(2)
def call(self, argchain, RESULT):
@@ -97,40 +101,59 @@
arg.push(self, ll_args, i)
i += 1
arg = arg.next
- result = self._do_call(self.funcsym, ll_args, RESULT)
- return result
+ if RESULT is lltype.Signed:
+ return self._do_call_int(self.funcsym, ll_args)
+ elif RESULT is lltype.Float:
+ return self._do_call_float(self.funcsym, ll_args)
+ else:
+ raise TypeError, 'Unsupported result type: %s' % RESULT
# END OF THE PUBLIC INTERFACE
- # -------------------------------------------------------------------------
- # the following methods are supposed to be seen opaquely by the JIT
- # optimizer. Don't call them
+ # ------------------------------------------------------------------------
+
+ # JIT friendly interface
+ # the following methods are supposed to be seen opaquely by the optimizer
@jit.oopspec('libffi_prepare_call(self)')
def _prepare(self):
ll_args = lltype.malloc(rffi.VOIDPP.TO, len(self.argtypes), flavor='raw')
return ll_args
- @specialize.arg(1)
- def _push_arg(self, TYPE, value, ll_args, i):
- # XXX: check the type is not translated?
- argtype = self.argtypes[i]
- c_size = intmask(argtype.c_size)
- ll_buf = lltype.malloc(rffi.CCHARP.TO, c_size, flavor='raw')
- push_arg_as_ffiptr(argtype, value, ll_buf)
- ll_args[i] = ll_buf
+ # _push_* and _do_call_* in theory could be automatically specialize()d by
+ # the annotator. However, specialization doesn't work well with oopspec,
+ # so we specialize them by hand
@jit.oopspec('libffi_push_int(self, value, ll_args, i)')
@enforceargs( None, int, None, int) # fix the annotation for tests
def _push_int(self, value, ll_args, i):
- self._push_arg(lltype.Signed, value, ll_args, i)
+ self._push_arg(value, ll_args, i)
@jit.oopspec('libffi_push_float(self, value, ll_args, i)')
@enforceargs( None, float, None, int) # fix the annotation for tests
def _push_float(self, value, ll_args, i):
- self._push_arg(lltype.Float, value, ll_args, i)
+ self._push_arg(value, ll_args, i)
+
+ @jit.oopspec('libffi_call_int(self, funcsym, ll_args)')
+ def _do_call_int(self, funcsym, ll_args):
+ return self._do_call(funcsym, ll_args, lltype.Signed)
+
+ @jit.oopspec('libffi_call_float(self, funcsym, ll_args)')
+ def _do_call_float(self, funcsym, ll_args):
+ return self._do_call(funcsym, ll_args, lltype.Float)
+
+ # ------------------------------------------------------------------------
+ # private methods
+
+ @specialize.argtype(1)
+ def _push_arg(self, value, ll_args, i):
+ # XXX: check the type is not translated?
+ argtype = self.argtypes[i]
+ c_size = intmask(argtype.c_size)
+ ll_buf = lltype.malloc(rffi.CCHARP.TO, c_size, flavor='raw')
+ push_arg_as_ffiptr(argtype, value, ll_buf)
+ ll_args[i] = ll_buf
@specialize.arg(3)
- @jit.oopspec('libffi_call(self, funcsym, ll_args, RESULT)')
def _do_call(self, funcsym, ll_args, RESULT):
# XXX: check len(args)?
ll_result = lltype.nullptr(rffi.CCHARP.TO)
From antocuni at codespeak.net Mon Oct 4 13:56:49 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 4 Oct 2010 13:56:49 +0200 (CEST)
Subject: [pypy-svn] r77567 - pypy/branch/jitffi/pypy/jit/metainterp/test
Message-ID: <20101004115649.00BCD282BAD@codespeak.net>
Author: antocuni
Date: Mon Oct 4 13:56:47 2010
New Revision: 77567
Added:
pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py
- copied, changed from r77562, pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py
Removed:
pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py
Log:
rename test_direct_call into test_fficall, and add a test for returning floats
Copied: pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py (from r77562, pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py)
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_direct_call.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py Mon Oct 4 13:56:47 2010
@@ -8,7 +8,7 @@
from pypy.translator.platform import platform
from pypy.rpython.lltypesystem import lltype, rffi
-class TestDirectCall(LLJitMixin):
+class TestFfiCall(LLJitMixin):
def setup_class(cls):
# prepare C code as an example, so we can load it and call
# it via rlib.libffi
@@ -16,14 +16,21 @@
c_file.write(py.code.Source('''
int sum_xy(int x, double y)
{
- return (x + (int)y);
+ return (x + (int)y);
+ }
+
+ float abs(double x)
+ {
+ if (x<0)
+ return -x;
+ return x;
}
'''))
eci = ExternalCompilationInfo(export_symbols=['sum_xy'])
cls.lib_name = str(platform.compile([c_file], eci, 'x',
standalone=False))
- def test_one(self):
+ def test_simple(self):
driver = JitDriver(reds = ['n', 'func'], greens = [])
def f(n):
@@ -48,3 +55,25 @@
'guard_true': 1,
'jump': 1})
+
+ def test_float_result(self):
+ driver = JitDriver(reds = ['n', 'func', 'res'], greens = [])
+
+ def f(n):
+ cdll = CDLL(self.lib_name)
+ func = cdll.getpointer('abs', [ffi_type_double], ffi_type_double)
+ res = 0.0
+ while n < 10:
+ driver.jit_merge_point(n=n, func=func, res=res)
+ driver.can_enter_jit(n=n, func=func, res=res)
+ func = hint(func, promote=True)
+ argchain = ArgChain()
+ argchain.float(float(-n))
+ res = func.call(argchain, lltype.Float)
+ n += 1
+ return res
+
+ res = self.meta_interp(f, [0])
+ assert res == 9
+ self.check_loops(call=1)
+
From antocuni at codespeak.net Mon Oct 4 14:20:51 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 4 Oct 2010 14:20:51 +0200 (CEST)
Subject: [pypy-svn] r77568 - in pypy/branch/jitffi/pypy: jit/codewriter rlib
Message-ID: <20101004122051.F2650282BAD@codespeak.net>
Author: antocuni
Date: Mon Oct 4 14:20:50 2010
New Revision: 77568
Modified:
pypy/branch/jitffi/pypy/jit/codewriter/support.py
pypy/branch/jitffi/pypy/rlib/libffi.py
Log:
use rffi.{LONG,DOUBLE} instead of lltype.{Signed,Float} (just for aesthetic reasons)
Modified: pypy/branch/jitffi/pypy/jit/codewriter/support.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/codewriter/support.py (original)
+++ pypy/branch/jitffi/pypy/jit/codewriter/support.py Mon Oct 4 14:20:50 2010
@@ -1,5 +1,5 @@
import sys
-from pypy.rpython.lltypesystem import lltype, rclass
+from pypy.rpython.lltypesystem import lltype, rclass, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython import rlist
from pypy.rpython.lltypesystem import rstr as ll_rstr, rdict as ll_rdict
@@ -236,10 +236,10 @@
return func(llfunc)._push_float(value, ll_args, i)
def _ll_3_libffi_call_int(llfunc, funcsym, ll_args):
- return func(llfunc)._do_call(funcsym, ll_args, lltype.Signed)
+ return func(llfunc)._do_call(funcsym, ll_args, rffi.LONG)
def _ll_3_libffi_call_float(llfunc, funcsym, ll_args):
- return func(llfunc)._do_call(funcsym, ll_args, lltype.Float)
+ return func(llfunc)._do_call(funcsym, ll_args, rffi.DOUBLE)
Modified: pypy/branch/jitffi/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/libffi.py Mon Oct 4 14:20:50 2010
@@ -101,9 +101,9 @@
arg.push(self, ll_args, i)
i += 1
arg = arg.next
- if RESULT is lltype.Signed:
+ if RESULT is rffi.LONG:
return self._do_call_int(self.funcsym, ll_args)
- elif RESULT is lltype.Float:
+ elif RESULT is rffi.DOUBLE:
return self._do_call_float(self.funcsym, ll_args)
else:
raise TypeError, 'Unsupported result type: %s' % RESULT
@@ -135,11 +135,11 @@
@jit.oopspec('libffi_call_int(self, funcsym, ll_args)')
def _do_call_int(self, funcsym, ll_args):
- return self._do_call(funcsym, ll_args, lltype.Signed)
+ return self._do_call(funcsym, ll_args, rffi.LONG)
@jit.oopspec('libffi_call_float(self, funcsym, ll_args)')
def _do_call_float(self, funcsym, ll_args):
- return self._do_call(funcsym, ll_args, lltype.Float)
+ return self._do_call(funcsym, ll_args, rffi.DOUBLE)
# ------------------------------------------------------------------------
# private methods
From antocuni at codespeak.net Mon Oct 4 15:24:11 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 4 Oct 2010 15:24:11 +0200 (CEST)
Subject: [pypy-svn] r77569 - in pypy/branch/jitffi/pypy: jit/metainterp/test
rlib
Message-ID: <20101004132411.B8462282BAD@codespeak.net>
Author: antocuni
Date: Mon Oct 4 15:24:09 2010
New Revision: 77569
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py
pypy/branch/jitffi/pypy/rlib/libffi.py
Log:
cast the result to the correct type when calling a function
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py Mon Oct 4 15:24:09 2010
@@ -2,7 +2,7 @@
import py
from pypy.rlib.jit import JitDriver, hint
from pypy.jit.metainterp.test.test_basic import LLJitMixin
-from pypy.rlib.libffi import CDLL, ffi_type_sint, ffi_type_double, ArgChain, Func
+from pypy.rlib.libffi import CDLL, ffi_type_sint, ffi_type_double, ffi_type_uchar, ArgChain, Func
from pypy.tool.udir import udir
from pypy.translator.tool.cbuild import ExternalCompilationInfo
from pypy.translator.platform import platform
@@ -12,7 +12,7 @@
def setup_class(cls):
# prepare C code as an example, so we can load it and call
# it via rlib.libffi
- c_file = udir.ensure("test_jit_direct_call", dir=1).join("xlib.c")
+ c_file = udir.ensure("test_jit_fficall", dir=1).join("xlib.c")
c_file.write(py.code.Source('''
int sum_xy(int x, double y)
{
@@ -25,8 +25,13 @@
return -x;
return x;
}
+
+ unsigned char cast_to_uchar(int x)
+ {
+ return 200+(unsigned char)x;
+ }
'''))
- eci = ExternalCompilationInfo(export_symbols=['sum_xy'])
+ eci = ExternalCompilationInfo(export_symbols=[])
cls.lib_name = str(platform.compile([c_file], eci, 'x',
standalone=False))
@@ -77,3 +82,23 @@
assert res == 9
self.check_loops(call=1)
+ def test_cast_result(self):
+ driver = JitDriver(reds = ['n', 'res', 'func'], greens = [])
+
+ def f(n):
+ cdll = CDLL(self.lib_name)
+ func = cdll.getpointer('cast_to_uchar', [ffi_type_sint],
+ ffi_type_uchar)
+ res = 0
+ while n < 10:
+ driver.jit_merge_point(n=n, func=func, res=res)
+ driver.can_enter_jit(n=n, func=func, res=res)
+ func = hint(func, promote=True)
+ argchain = ArgChain()
+ argchain.int(0)
+ res = func.call(argchain, rffi.UCHAR)
+ n += 1
+ return res
+
+ res = self.meta_interp(f, [0])
+ assert res == 200
Modified: pypy/branch/jitffi/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/libffi.py Mon Oct 4 15:24:09 2010
@@ -15,6 +15,13 @@
import_types()
del import_types
+def _fits_into_long(TYPE):
+ if not isinstance(TYPE, lltype.Number):
+ return False
+ if TYPE is rffi.FLOAT or TYPE is rffi.DOUBLE:
+ return False
+ sz = rffi.sizeof(TYPE)
+ return sz <= rffi.sizeof(rffi.LONG)
# ----------------------------------------------------------------------
@@ -101,12 +108,15 @@
arg.push(self, ll_args, i)
i += 1
arg = arg.next
- if RESULT is rffi.LONG:
- return self._do_call_int(self.funcsym, ll_args)
+ #
+ if _fits_into_long(RESULT):
+ res = self._do_call_int(self.funcsym, ll_args)
elif RESULT is rffi.DOUBLE:
return self._do_call_float(self.funcsym, ll_args)
else:
raise TypeError, 'Unsupported result type: %s' % RESULT
+ #
+ return rffi.cast(RESULT, res)
# END OF THE PUBLIC INTERFACE
# ------------------------------------------------------------------------
@@ -119,6 +129,7 @@
ll_args = lltype.malloc(rffi.VOIDPP.TO, len(self.argtypes), flavor='raw')
return ll_args
+
# _push_* and _do_call_* in theory could be automatically specialize()d by
# the annotator. However, specialization doesn't work well with oopspec,
# so we specialize them by hand
From antocuni at codespeak.net Mon Oct 4 15:28:44 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 4 Oct 2010 15:28:44 +0200 (CEST)
Subject: [pypy-svn] r77570 - pypy/branch/jitffi/pypy/jit/metainterp/test
Message-ID: <20101004132844.416B4282BAD@codespeak.net>
Author: antocuni
Date: Mon Oct 4 15:28:42 2010
New Revision: 77570
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py
Log:
use rffi.{LONG,DOUBLE} also here
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py Mon Oct 4 15:28:42 2010
@@ -48,7 +48,7 @@
func = hint(func, promote=True)
argchain = ArgChain()
argchain.int(n).float(1.2)
- n = func.call(argchain, lltype.Signed)
+ n = func.call(argchain, rffi.LONG)
return n
res = self.meta_interp(f, [0])
@@ -74,7 +74,7 @@
func = hint(func, promote=True)
argchain = ArgChain()
argchain.float(float(-n))
- res = func.call(argchain, lltype.Float)
+ res = func.call(argchain, rffi.DOUBLE)
n += 1
return res
From arigo at codespeak.net Mon Oct 4 15:29:38 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 4 Oct 2010 15:29:38 +0200 (CEST)
Subject: [pypy-svn] r77571 - pypy/branch/32ptr-on-64bit
Message-ID: <20101004132938.6BD27282BAD@codespeak.net>
Author: arigo
Date: Mon Oct 4 15:29:36 2010
New Revision: 77571
Added:
pypy/branch/32ptr-on-64bit/
- copied from r77569, pypy/trunk/
Log:
A branch in which I play around with the idea of compressing
64-bit pointers into 32-bit integers. The "compression" is
simply taking the pointer value and shifting it 3 bits, so it
is limited to a 32GB heap.
From arigo at codespeak.net Mon Oct 4 15:30:25 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 4 Oct 2010 15:30:25 +0200 (CEST)
Subject: [pypy-svn] r77572 - in pypy/branch/32ptr-on-64bit/pypy: annotation
config rpython rpython/lltypesystem rpython/memory
rpython/memory/gc rpython/memory/test rpython/test
translator/c translator/c/src translator/c/test
Message-ID: <20101004133025.07E1A282BAD@codespeak.net>
Author: arigo
Date: Mon Oct 4 15:30:22 2010
New Revision: 77572
Added:
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py
pypy/branch/32ptr-on-64bit/pypy/rpython/test/test_rcompressed.py
Modified:
pypy/branch/32ptr-on-64bit/pypy/annotation/model.py
pypy/branch/32ptr-on-64bit/pypy/config/translationoption.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/opimpl.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rclass.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rffi.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gcwrapper.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_gc.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_gctypelayout.py
pypy/branch/32ptr-on-64bit/pypy/rpython/raddress.py
pypy/branch/32ptr-on-64bit/pypy/rpython/rmodel.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/database.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/src/commondefs.h
pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_lladdresses.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_newgc.py
Log:
In-progress.
Modified: pypy/branch/32ptr-on-64bit/pypy/annotation/model.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/annotation/model.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/annotation/model.py Mon Oct 4 15:30:22 2010
@@ -518,6 +518,14 @@
def can_be_none(self):
return False
+# for compressed 32-bit "pointers" on 64-bit platforms
+
+class SomeHiddenGcRef32(SomeObject):
+ immutable = True
+
+ def can_be_none(self):
+ return False
+
#____________________________________________________________
# annotation of low-level types
@@ -582,6 +590,7 @@
(SomeChar(), lltype.Char),
(SomeUnicodeCodePoint(), lltype.UniChar),
(SomeAddress(), llmemory.Address),
+ (SomeHiddenGcRef32(), llmemory.HiddenGcRef32),
]
def annotation_to_lltype(s_val, info=None):
Modified: pypy/branch/32ptr-on-64bit/pypy/config/translationoption.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/config/translationoption.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/config/translationoption.py Mon Oct 4 15:30:22 2010
@@ -193,6 +193,11 @@
"When true, enable the use of tagged pointers. "
"If false, use normal boxing",
default=False),
+ BoolOption("compressptr", "Compress pointers; limits the program to 32GB",
+ default=False, cmdline="--compressptr",
+ requires=[("translation.type_system", "lltype"),
+ ("translation.taggedpointers", False)]
+ + [("compressptr (64-bit only)", True)]*(not IS_64_BITS)),
# options for ootype
OptionDescription("ootype", "Object Oriented Typesystem options", [
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py Mon Oct 4 15:30:22 2010
@@ -8,6 +8,7 @@
from pypy.rlib.objectmodel import Symbolic
from pypy.rpython.lltypesystem import lltype
from pypy.tool.uid import uid
+from pypy.rlib.objectmodel import we_are_translated
class AddressOffset(Symbolic):
@@ -562,6 +563,61 @@
def array_item_type_match(T1, T2):
return T1 == T2 or (T2 == GCREF and isinstance(T1, lltype.Ptr))
+# ____________________________________________________________
+
+class _hiddengcref32(object):
+ """A 'hidden' compressed 32-bit value that represents a
+ full, 64-bit GC pointer."""
+
+ def __init__(self, adr64):
+ self.adr64 = adr64
+
+ def __repr__(self):
+ return '<_hiddengcref32>'
+
+ def __str__(self):
+ return '<_hiddengcref32 %s>' % (self.adr64,)
+
+compressed_null_addr = _hiddengcref32(NULL)
+HiddenGcRef32 = lltype.Primitive("HiddenGcRef32", compressed_null_addr)
+_hiddengcref32._TYPE = HiddenGcRef32
+
+class OddValueMarker(AddressOffset):
+ """This is the value '1'. Used as an odd-valued marker by the GC."""
+ def __repr__(self):
+ return '< OddValueMarker 1 >'
+ def ref(self, ptr):
+ raise AssertionError("should not try to directly get the address"
+ " from a HiddenGcRef32")
+
+def has_odd_value_marker(addressofs):
+ if we_are_translated():
+ return bool(addressofs & 1)
+ return _has_odd_value_marker(addressofs)
+
+def _has_odd_value_marker(addressofs):
+ while isinstance(addressofs, CompositeOffset):
+ addressofs = addressofs.offsets[-1]
+ return isinstance(addressofs, OddValueMarker)
+
+def remove_odd_value_marker(addressofs):
+ if we_are_translated():
+ return addressofs - 1
+ return _remove_odd_value_marker(addressofs)
+
+def _remove_odd_value_marker(addressofs):
+ if isinstance(addressofs, CompositeOffset):
+ offsets = list(addressofs.offsets)
+ offsets[-1] = _remove_odd_value_marker(offsets[-1])
+ if offsets[-1] == 0:
+ del offsets[-1]
+ if len(offsets) == 1:
+ return offsets[0]
+ return CompositeOffset(*offsets)
+ assert isinstance(addressofs, OddValueMarker)
+ return 0
+
+# ____________________________________________________________
class _fakeaccessor(object):
def __init__(self, addr):
@@ -597,6 +653,9 @@
class _char_fakeaccessor(_fakeaccessor):
TYPE = lltype.Char
+class _hiddengcref32_fakeaccessor(_fakeaccessor):
+ TYPE = HiddenGcRef32
+
class _address_fakeaccessor(_fakeaccessor):
TYPE = Address
@@ -624,12 +683,14 @@
"char": lltype.Char,
"address": Address,
"float": lltype.Float,
+ "hiddengcref32": HiddenGcRef32,
}
fakeaddress.signed = property(_signed_fakeaccessor)
fakeaddress.float = property(_float_fakeaccessor)
fakeaddress.char = property(_char_fakeaccessor)
fakeaddress.address = property(_address_fakeaccessor)
+fakeaddress.hiddengcref32 = property(_hiddengcref32_fakeaccessor)
fakeaddress._TYPE = Address
# the obtained address will not keep the object alive. e.g. if the object is
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py Mon Oct 4 15:30:22 2010
@@ -426,6 +426,9 @@
'gc_gettypeptr_group': LLOp(canfold=True),
'get_member_index': LLOp(canfold=True),
+ 'hide_into_adr32': LLOp(canrun=True),
+ 'show_from_adr32': LLOp(canrun=True),
+
# __________ used by the JIT ________
'jit_marker': LLOp(),
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/opimpl.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/opimpl.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/opimpl.py Mon Oct 4 15:30:22 2010
@@ -526,6 +526,18 @@
def op_shrink_array(array, smallersize):
return False
+def op_hide_into_adr32(adr):
+ if lltype.typeOf(adr) != llmemory.Address:
+ adr = llmemory.cast_ptr_to_adr(adr)
+ return llmemory._hiddengcref32(adr)
+
+def op_show_from_adr32(RESTYPE, adr32):
+ if RESTYPE == llmemory.Address:
+ return adr32.adr64
+ else:
+ return llmemory.cast_adr_to_ptr(adr32.adr64, RESTYPE)
+op_show_from_adr32.need_result_type = True
+
# ____________________________________________________________
def get_op_impl(opname):
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rclass.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rclass.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rclass.py Mon Oct 4 15:30:22 2010
@@ -4,6 +4,7 @@
from pypy.objspace.flow.model import Constant
from pypy.rpython.error import TyperError
from pypy.rpython.rmodel import Repr, inputconst, warning, mangle
+from pypy.rpython.rmodel import externalvsinternalfield
from pypy.rpython.rclass import AbstractClassRepr,\
AbstractInstanceRepr,\
MissingRTypeAttribute,\
@@ -326,7 +327,8 @@
fields = {}
allinstancefields = {}
if self.classdef is None:
- fields['__class__'] = 'typeptr', get_type_repr(self.rtyper)
+ r = get_type_repr(self.rtyper)
+ fields['__class__'] = 'typeptr', r, r
else:
# instance attributes
if llfields is None:
@@ -337,8 +339,12 @@
if not attrdef.readonly:
r = self.rtyper.getrepr(attrdef.s_value)
mangled_name = 'inst_' + name
- fields[name] = mangled_name, r
- llfields.append((mangled_name, r.lowleveltype))
+ if self.gcflavor == 'gc':
+ r, internal_r = externalvsinternalfield(self.rtyper, r)
+ else:
+ internal_r = r
+ fields[name] = mangled_name, r, internal_r
+ llfields.append((mangled_name, internal_r.lowleveltype))
self.rbase = getinstancerepr(self.rtyper, self.classdef.basedef,
self.gcflavor)
@@ -394,7 +400,7 @@
return getinstancerepr(self.rtyper, None, self.gcflavor)
def _get_field(self, attr):
- return self.fields[attr]
+ return self.fields[attr][:2]
def null_instance(self):
return nullptr(self.object_type)
@@ -410,7 +416,7 @@
# recursively build the parent part of the instance
self.rbase.initialize_prebuilt_data(value, classdef, result.super)
# then add instance attributes from this level
- for name, (mangled_name, r) in self.fields.items():
+ for name, (mangled_name, _, r) in self.fields.items():
if r.lowleveltype is Void:
llattrvalue = None
else:
@@ -440,7 +446,7 @@
def getfieldrepr(self, attr):
"""Return the repr used for the given attribute."""
if attr in self.fields:
- mangled_name, r = self.fields[attr]
+ mangled_name, r, internal_r = self.fields[attr]
return r
else:
if self.classdef is None:
@@ -450,12 +456,13 @@
def getfield(self, vinst, attr, llops, force_cast=False, flags={}):
"""Read the given attribute (or __class__ for the type) of 'vinst'."""
if attr in self.fields:
- mangled_name, r = self.fields[attr]
+ mangled_name, r, internal_r = self.fields[attr]
cname = inputconst(Void, mangled_name)
if force_cast:
vinst = llops.genop('cast_pointer', [vinst], resulttype=self)
self.hook_access_field(vinst, cname, llops, flags)
- return llops.genop('getfield', [vinst, cname], resulttype=r)
+ v = llops.genop('getfield', [vinst, cname], resulttype=internal_r)
+ return llops.convertvar(v, internal_r, r)
else:
if self.classdef is None:
raise MissingRTypeAttribute(attr)
@@ -466,7 +473,8 @@
flags={}):
"""Write the given attribute (or __class__ for the type) of 'vinst'."""
if attr in self.fields:
- mangled_name, r = self.fields[attr]
+ mangled_name, r, internal_r = self.fields[attr]
+ vvalue = llops.convertvar(vvalue, r, internal_r)
cname = inputconst(Void, mangled_name)
if force_cast:
vinst = llops.genop('cast_pointer', [vinst], resulttype=self)
@@ -499,7 +507,7 @@
for fldname in flds:
if fldname == '__class__':
continue
- mangled_name, r = self.allinstancefields[fldname]
+ mangled_name, r, internal_r = self.allinstancefields[fldname]
if r.lowleveltype is Void:
continue
value = self.classdef.classdesc.read_attribute(fldname, None)
Added: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py
==============================================================================
--- (empty file)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py Mon Oct 4 15:30:22 2010
@@ -0,0 +1,57 @@
+from pypy.tool.pairtype import pairtype
+from pypy.rlib.objectmodel import we_are_translated
+from pypy.config.translationoption import IS_64_BITS
+from pypy.rpython.rmodel import Repr, inputconst
+from pypy.rpython.lltypesystem import lltype, llmemory, rffi
+
+
+
+def get_compressed_gcref_repr(rtyper, baserepr):
+ try:
+ comprmgr = rtyper.compressed_gcref_manager
+ except AttributeError:
+ comprmgr = rtyper.compressed_gcref_manager = ComprGcRefManager(rtyper)
+ return comprmgr.get_compressed_gcref_repr(baserepr)
+
+
+class ComprGcRefManager(object):
+ def __init__(self, rtyper):
+ assert IS_64_BITS # this is only for 64-bits
+ self.rtyper = rtyper
+ self.comprgcreprs = {}
+
+ def get_compressed_gcref_repr(self, baserepr):
+ try:
+ return self.comprgcreprs[baserepr]
+ except KeyError:
+ comprgcrepr = CompressedGcRefRepr(self, baserepr)
+ self.comprgcreprs[baserepr] = comprgcrepr
+ return comprgcrepr
+
+
+class CompressedGcRefRepr(Repr):
+ lowleveltype = llmemory.HiddenGcRef32
+
+ def __init__(self, mgr, baserepr):
+ self.mgr = mgr
+ self.baserepr = baserepr
+ self.BASETYPE = self.baserepr.lowleveltype
+
+ def convert_const(self, value):
+ ptr = self.baserepr.convert_const(value)
+ T = lltype.typeOf(ptr)
+ assert T == self.BASETYPE
+ return llmemory._hiddengcref32(llmemory.cast_ptr_to_adr(ptr))
+
+
+class __extend__(pairtype(Repr, CompressedGcRefRepr)):
+ def convert_from_to((r_from, r_to), v, llops):
+ assert r_from.lowleveltype.TO._gckind == 'gc'
+ return llops.genop('hide_into_adr32', [v],
+ resulttype=llmemory.HiddenGcRef32)
+
+class __extend__(pairtype(CompressedGcRefRepr, Repr)):
+ def convert_from_to((r_from, r_to), v, llops):
+ assert r_to.lowleveltype.TO._gckind == 'gc'
+ return llops.genop('show_from_adr32', [v],
+ resulttype=r_to.lowleveltype)
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rffi.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rffi.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rffi.py Mon Oct 4 15:30:22 2010
@@ -805,6 +805,8 @@
return 8
if tp is lltype.SingleFloat:
return 4
+ if tp is llmemory.HiddenGcRef32:
+ return 4
assert isinstance(tp, lltype.Number)
if tp is lltype.Signed:
return ULONG._type.BITS/8
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py Mon Oct 4 15:30:22 2010
@@ -1,4 +1,5 @@
from pypy.rpython.lltypesystem import lltype, llmemory, llarena
+from pypy.rpython.lltypesystem.lloperation import llop
from pypy.rlib.debug import ll_assert
from pypy.rpython.memory.gcheader import GCHeaderBuilder
from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
@@ -168,8 +169,10 @@
def trace(self, obj, callback, arg):
"""Enumerate the locations inside the given obj that can contain
- GC pointers. For each such location, callback(pointer, arg) is
- called, where 'pointer' is an address inside the object.
+ GC pointers. For each such GC pointer, callback(object, arg) is
+ called, where 'object' is the address of the stored object.
+ 'callback' can return a new address that replaces the old one in
+ 'obj', or it can return None.
Typically, 'callback' is a bound method and 'arg' can be None.
"""
typeid = self.get_type_id(obj)
@@ -179,16 +182,16 @@
item = obj + llmemory.gcarrayofptr_itemsoffset
while length > 0:
if self.points_to_valid_gc_object(item):
- callback(item, arg)
+ newaddr = callback(item.address[0], arg)
+ if newaddr is not None:
+ item.address[0] = newaddr
item += llmemory.gcarrayofptr_singleitemoffset
length -= 1
return
offsets = self.offsets_to_gc_pointers(typeid)
i = 0
while i < len(offsets):
- item = obj + offsets[i]
- if self.points_to_valid_gc_object(item):
- callback(item, arg)
+ self._trace_see(obj, offsets[i], callback, arg)
i += 1
if self.has_gcptr_in_varsize(typeid):
item = obj + self.varsize_offset_to_variable_part(typeid)
@@ -198,14 +201,35 @@
while length > 0:
j = 0
while j < len(offsets):
- itemobj = item + offsets[j]
- if self.points_to_valid_gc_object(itemobj):
- callback(itemobj, arg)
+ self._trace_see(item, offsets[j], callback, arg)
j += 1
item += itemlength
length -= 1
trace._annspecialcase_ = 'specialize:arg(2)'
+ def _trace_see(self, obj, ofs, callback, arg):
+ if self.config.compressptr and llmemory.has_odd_value_marker(ofs):
+ # special handling of HiddenGcRef32 on 64-bit platforms
+ ofs = llmemory.remove_odd_value_marker(ofs)
+ item = obj + ofs
+ address = llop.show_from_adr32(llmemory.Address,
+ item.hiddengcref32[0])
+ if self.is_valid_gc_object(address):
+ newaddr = callback(address, arg)
+ if newaddr is not None:
+ newaddr32 = llop.hide_into_adr32(llmemory.HiddenGcRef32,
+ newaddr)
+ item.hiddengcref32[0] = newaddr32
+ else:
+ # common case
+ item = obj + ofs
+ if self.points_to_valid_gc_object(item):
+ newaddr = callback(item.address[0], arg)
+ if newaddr is not None:
+ item.address[0] = newaddr
+ _trace_see._annspecialcase_ = 'specialize:arg(3)'
+ _trace_see._always_inline_ = True
+
def trace_partial(self, obj, start, stop, callback, arg):
"""Like trace(), but only walk the array part, for indices in
range(start, stop). Must only be called if has_gcptr_in_varsize().
@@ -218,7 +242,9 @@
item += llmemory.gcarrayofptr_singleitemoffset * start
while length > 0:
if self.points_to_valid_gc_object(item):
- callback(item, arg)
+ newaddr = callback(item.address[0], arg)
+ if newaddr is not None:
+ item.address[0] = newaddr
item += llmemory.gcarrayofptr_singleitemoffset
length -= 1
return
@@ -231,9 +257,7 @@
while length > 0:
j = 0
while j < len(offsets):
- itemobj = item + offsets[j]
- if self.points_to_valid_gc_object(itemobj):
- callback(itemobj, arg)
+ self._trace_see(item, offsets[j], callback, arg)
j += 1
item += itemlength
length -= 1
@@ -279,8 +303,7 @@
obj = root.address[0]
ll_assert(bool(obj), "NULL address from walk_roots()")
self._debug_record(obj)
- def _debug_callback2(self, pointer, ignored):
- obj = pointer.address[0]
+ def _debug_callback2(self, obj, ignored):
ll_assert(bool(obj), "NULL address from self.trace()")
self._debug_record(obj)
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py Mon Oct 4 15:30:22 2010
@@ -1025,19 +1025,17 @@
def _trace_drag_out1(self, root):
- self._trace_drag_out(root, None)
+ root.address[0] = self._trace_drag_out(root.address[0], None)
- def _trace_drag_out(self, root, ignored):
- obj = root.address[0]
+ def _trace_drag_out(self, obj, ignored):
#
# If 'obj' is not in the nursery, nothing to change.
if not self.is_in_nursery(obj):
- return
+ return obj
#
# If 'obj' was already forwarded, change it to its forwarding address.
if self.is_forwarded(obj):
- root.address[0] = self.get_forwarding_address(obj)
- return
+ return self.get_forwarding_address(obj)
#
# First visit to 'obj': we must move it out of the nursery.
size_gc_header = self.gcheaderbuilder.size_gc_header
@@ -1075,14 +1073,14 @@
newobj = newhdr + size_gc_header
llmemory.cast_adr_to_ptr(obj, FORWARDSTUBPTR).forw = newobj
#
- # Change the original pointer to this object.
- root.address[0] = newobj
- #
# Add the newobj to the list 'old_objects_pointing_to_young',
# because it can contain further pointers to other young objects.
# We will fix such references to point to the copy of the young
# objects when we walk 'old_objects_pointing_to_young'.
self.old_objects_pointing_to_young.append(newobj)
+ #
+ # Change the original pointer to this newly built object.
+ return newobj
def _malloc_out_of_nursery(self, totalsize):
@@ -1279,8 +1277,8 @@
def _collect_ref(self, root):
self.objects_to_trace.append(root.address[0])
- def _collect_ref_rec(self, root, ignored):
- self.objects_to_trace.append(root.address[0])
+ def _collect_ref_rec(self, obj, ignored):
+ self.objects_to_trace.append(obj)
def visit_all_objects(self):
pending = self.objects_to_trace
@@ -1421,8 +1419,8 @@
self.objects_with_finalizers.delete()
self.objects_with_finalizers = new_with_finalizer
- def _append_if_nonnull(pointer, stack):
- stack.append(pointer.address[0])
+ def _append_if_nonnull(obj, stack):
+ stack.append(obj)
_append_if_nonnull = staticmethod(_append_if_nonnull)
def _finalization_state(self, obj):
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py Mon Oct 4 15:30:22 2010
@@ -16,6 +16,8 @@
"""
_alloc_flavor_ = 'raw'
+ # if config.translation.compressptr, the OFFSETS_TO_GC_PTR lists
+ # as odd integers the fields that are compressed pointers (UINT).
OFFSETS_TO_GC_PTR = lltype.Array(lltype.Signed)
ADDRESS_VOID_FUNC = lltype.FuncType([llmemory.Address], lltype.Void)
FINALIZERTYPE = lltype.Ptr(ADDRESS_VOID_FUNC)
@@ -389,6 +391,8 @@
# (adr + off)
elif isinstance(TYPE, lltype.Ptr) and TYPE.TO._gckind == 'gc':
offsets.append(0)
+ elif TYPE == llmemory.HiddenGcRef32:
+ offsets.append(llmemory.OddValueMarker())
return offsets
def gc_pointers_inside(v, adr, mutable_only=False):
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gcwrapper.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gcwrapper.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gcwrapper.py Mon Oct 4 15:30:22 2010
@@ -88,7 +88,9 @@
def setinterior(self, toplevelcontainer, inneraddr, INNERTYPE, newvalue,
offsets=()):
if (lltype.typeOf(toplevelcontainer).TO._gckind == 'gc' and
- isinstance(INNERTYPE, lltype.Ptr) and INNERTYPE.TO._gckind == 'gc'):
+ (isinstance(INNERTYPE, lltype.Ptr) and
+ INNERTYPE.TO._gckind == 'gc')
+ or INNERTYPE == llmemory.HiddenGcRef32):
#
wb = True
if self.has_write_barrier_from_array:
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_gc.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_gc.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_gc.py Mon Oct 4 15:30:22 2010
@@ -77,6 +77,23 @@
#assert simulator.current_size - curr < 16000 * INT_SIZE / 4
#print "size before: %s, size after %s" % (curr, simulator.current_size)
+ def test_llinterp_instances(self):
+ class Cons(object):
+ def __init__(self, car, cdr):
+ self.car = car
+ self.cdr = cdr
+ def malloc_a_lot():
+ i = 0
+ while i < 10:
+ i += 1
+ a = Cons(1, Cons(2, Cons(i, None)))
+ b = a
+ j = 0
+ while j < 20:
+ j += 1
+ b = Cons(j, b)
+ res = self.interpret(malloc_a_lot, [])
+
def test_global_list(self):
lst = []
def append_to_list(i, j):
@@ -776,3 +793,20 @@
class TestMiniMarkGCCardMarking(TestMiniMarkGC):
GC_PARAMS = {'card_page_indices': 4}
+
+class TestMiniMarkGCCompressPtr(TestMiniMarkGC):
+ def setup_class(cls):
+ TestMiniMarkGC.setup_class.im_func(cls)
+ #
+ from pypy.config.translationoption import IS_64_BITS
+ if not IS_64_BITS:
+ py.test.skip("only for 64-bits")
+ from pypy.config.pypyoption import get_pypy_config
+ cls._config = get_pypy_config(translating=True)
+ cls._config.translation.compressptr = True
+
+ def interpret(self, *args, **kwds):
+ if kwds.get('taggedpointers'):
+ py.test.skip("cannot have both taggedpointers and compressptr")
+ kwds['config'] = self._config
+ return super(TestMiniMarkGCCompressPtr, self).interpret(*args, **kwds)
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_gctypelayout.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_gctypelayout.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_gctypelayout.py Mon Oct 4 15:30:22 2010
@@ -37,6 +37,15 @@
for T, c in [(GC_S, 0), (GC_S2, 2), (GC_A, 0), (GC_A2, 0), (GC_S3, 2)]:
assert len(offsets_to_gc_pointers(T)) == c
+def test_hiddenptr32():
+ from pypy.rpython.lltypesystem.llmemory import HIDDENPTR32
+ from pypy.rpython.lltypesystem.llmemory import has_odd_value_marker
+ T = lltype.GcStruct('T', ('foo', lltype.Ptr(GC_S3)), ('bar', HIDDENPTR32))
+ ofs = offsets_to_gc_pointers(T)
+ assert len(ofs) == 2
+ assert not has_odd_value_marker(ofs[0])
+ assert has_odd_value_marker(ofs[1])
+
def test_layout_builder(lltype2vtable=None):
# XXX a very minimal test
layoutbuilder = TypeLayoutBuilder(FakeGC, lltype2vtable)
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/raddress.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/raddress.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/raddress.py Mon Oct 4 15:30:22 2010
@@ -2,7 +2,7 @@
from pypy.tool.pairtype import pairtype
from pypy.annotation import model as annmodel
from pypy.rpython.lltypesystem.llmemory import NULL, Address, \
- cast_adr_to_int, fakeaddress
+ cast_adr_to_int, fakeaddress, HiddenGcRef32, _hiddengcref32
from pypy.rpython.rmodel import Repr, IntegerRepr
from pypy.rpython.rptr import PtrRepr
from pypy.rpython.lltypesystem import lltype
@@ -24,6 +24,13 @@
def rtyper_makekey(self):
return self.__class__, self.type
+class __extend__(annmodel.SomeHiddenGcRef32):
+ def rtyper_makerepr(self, rtyper):
+ return hiddengcref32_repr
+
+ def rtyper_makekey(self):
+ return self.__class__,
+
class AddressRepr(Repr):
lowleveltype = Address
@@ -139,4 +146,13 @@
def convert_from_to((r_ptr, r_addr), v, llops):
return llops.genop('cast_ptr_to_adr', [v], resulttype=Address)
+# ____________________________________________________________
+
+class HiddenGcRef32Repr(Repr):
+ lowleveltype = HiddenGcRef32
+
+ def convert_const(self, value):
+ assert isinstance(value, _hiddengcref32)
+ return value
+hiddengcref32_repr = HiddenGcRef32Repr()
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/rmodel.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/rmodel.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/rmodel.py Mon Oct 4 15:30:22 2010
@@ -438,9 +438,21 @@
from pypy.rpython import rclass
if (isinstance(item_repr, rclass.AbstractInstanceRepr) and
getattr(item_repr, 'gcflavor', 'gc') == 'gc'):
+ #if rtyper.annotator.translator.config.translation.compressptr:
+ # return externalvsinternalfield(rtyper, item_repr)
return item_repr, rclass.getinstancerepr(rtyper, None)
- else:
- return item_repr, item_repr
+ return item_repr, item_repr
+
+def externalvsinternalfield(rtyper, field_repr):
+ # usually a no-op, except if config.translation.compressptr, in which case
+ # it tries hard to return a wrapping compressed_gcref_repr
+ if rtyper.annotator.translator.config.translation.compressptr:
+ if (isinstance(field_repr.lowleveltype, Ptr) and
+ field_repr.lowleveltype.TO._gckind == 'gc'):
+ from pypy.rpython.lltypesystem import rcompressed
+ return (field_repr,
+ rcompressed.get_compressed_gcref_repr(rtyper, field_repr))
+ return field_repr, field_repr
class DummyValueBuilder(object):
Added: pypy/branch/32ptr-on-64bit/pypy/rpython/test/test_rcompressed.py
==============================================================================
--- (empty file)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/test/test_rcompressed.py Mon Oct 4 15:30:22 2010
@@ -0,0 +1,28 @@
+import py
+from pypy.config.translationoption import IS_64_BITS
+from pypy.rpython.test import test_rclass
+
+
+def setup_module(mod):
+ if not IS_64_BITS:
+ py.test.skip("for 64-bits only")
+
+
+class MixinCompressed64(object):
+ def _get_config(self):
+ from pypy.config.pypyoption import get_pypy_config
+ config = get_pypy_config(translating=True)
+ config.translation.compressptr = True
+ return config
+
+ def interpret(self, *args, **kwds):
+ kwds['config'] = self._get_config()
+ return super(MixinCompressed64, self).interpret(*args, **kwds)
+
+ def interpret_raises(self, *args, **kwds):
+ kwds['config'] = self._get_config()
+ return super(MixinCompressed64, self).interpret_raises(*args, **kwds)
+
+
+class TestLLtype64(MixinCompressed64, test_rclass.TestLLtype):
+ pass
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/database.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/database.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/database.py Mon Oct 4 15:30:22 2010
@@ -55,6 +55,7 @@
# assign to a constant object is something C doesn't think is
# constant
self.late_initializations = []
+ self.late_initializations_hiddengcref32 = []
self.namespace = CNameManager()
if translator is None or translator.rtyper is None:
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py Mon Oct 4 15:30:22 2010
@@ -771,25 +771,33 @@
yield '}'
def generic_initializationexpr(db, value, access_expr, decoration):
- if isinstance(typeOf(value), ContainerType):
+ TYPE = typeOf(value)
+ if isinstance(TYPE, ContainerType):
node = db.getcontainernode(value)
lines = list(node.initializationexpr(decoration+'.'))
lines[-1] += ','
return lines
else:
comma = ','
- if typeOf(value) == Ptr(PyObject) and value:
+ if TYPE == Ptr(PyObject) and value:
# cannot just write 'gxxx' as a constant in a structure :-(
node = db.getcontainernode(value._obj)
expr = 'NULL /*%s*/' % node.name
node.where_to_copy_me.append('&%s' % access_expr)
- elif typeOf(value) == Float and (isinf(value) or isnan(value)):
- db.late_initializations.append(('%s' % access_expr, db.get(value)))
+ elif TYPE == Float and (isinf(value) or isnan(value)):
+ db.late_initializations.append((access_expr, db.get(value)))
expr = '0.0 /* patched later by %sinfinity */' % (
'-+'[value > 0])
+ elif TYPE == llmemory.HiddenGcRef32:
+ if value.adr64:
+ db.late_initializations_hiddengcref32.append((access_expr,
+ value))
+ expr = '0 /*HIDE_INTO_ADR32%s*/' % db.get(value.adr64.ptr)
+ else:
+ expr = '0'
else:
expr = db.get(value)
- if typeOf(value) is Void:
+ if TYPE is Void:
comma = ''
expr += comma
i = expr.find('\n')
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py Mon Oct 4 15:30:22 2010
@@ -7,7 +7,7 @@
from pypy.rpython.lltypesystem.llmemory import Address, \
AddressOffset, ItemOffset, ArrayItemsOffset, FieldOffset, \
CompositeOffset, ArrayLengthOffset, \
- GCHeaderOffset, GCREF, AddressAsInt
+ GCHeaderOffset, GCREF, AddressAsInt, HiddenGcRef32, OddValueMarker
from pypy.rpython.lltypesystem.llarena import RoundedUpForAllocation
from pypy.translator.c.support import cdecl, barebonearray
@@ -63,6 +63,8 @@
return '(%s+%dL)' % (name, value.rest)
elif isinstance(value, AddressAsInt):
return '((long)%s)' % name_address(value.adr, db)
+ elif isinstance(value, OddValueMarker):
+ return '1 /*OddValueMarker*/'
else:
raise Exception("unimplemented symbolic %r"%value)
if value is None:
@@ -145,6 +147,12 @@
else:
return 'NULL'
+def name_hiddengcref32(value, db):
+ # The only prebuilt HiddenGcRef32 that should occur in a translated C
+ # program occur as fields or items of a GcStruct or GcArray.
+ db.get(value.adr64)
+ return 'HIDE_INTO_ADR32(???) /* see primitive.py, name_hiddengcref32() */'
+
def name_small_integer(value, db):
"""Works for integers of size at most INT or UINT."""
if isinstance(value, Symbolic):
@@ -173,6 +181,7 @@
Void: name_void,
Address: name_address,
GCREF: name_gcref,
+ HiddenGcRef32: name_hiddengcref32,
}
PrimitiveType = {
@@ -188,6 +197,7 @@
Void: 'void @',
Address: 'void* @',
GCREF: 'void* @',
+ HiddenGcRef32: 'hiddengcref32_t @',
}
def define_c_primitive(ll_type, c_name, suffix=''):
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/src/commondefs.h
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/src/commondefs.h (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/src/commondefs.h Mon Oct 4 15:30:22 2010
@@ -65,6 +65,15 @@
# define SIZEOF_LONG 8
# define SIZEOF_LONG_LONG 8
+ /* HiddenGcRef32 support (on 64-bits only) */
+ typedef unsigned int hiddengcref32_t;
+# define uint32_t hiddengcref32_t
+# define OP_SHOW_FROM_ADR32(x, r) r = (void*)(((unsigned long)(x)) << 3)
+# define OP_HIDE_INTO_ADR32(x, r) \
+ r = (hiddengcref32_t)(((unsigned long)(x)) >> 3); \
+ RPyAssert((void*)(((unsigned long)(r)) << 3) == (x), \
+ "pointer too big or misaligned!")
+
#endif
/********************************************************/
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_lladdresses.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_lladdresses.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_lladdresses.py Mon Oct 4 15:30:22 2010
@@ -232,3 +232,43 @@
assert res == 456
res = fc(77)
assert res == 123
+
+
+##class TestHiddenGcRef32(StandaloneTests):
+
+## def setup_class(cls):
+## from pypy.config.translationoption import IS_64_BITS
+## if 0:# not IS_64_BITS:
+## py.test.skip("only for 64-bits")
+
+## def test_hiddengcref32(self):
+## from pypy.rpython.lltypesystem.lloperation import llop
+## S = lltype.GcStruct('S', ('x', lltype.Signed),
+## ('y', HiddenGcRef32))
+## prebuilt = lltype.malloc(S, immortal=True)
+## prebuilt2 = lltype.malloc(S, immortal=True)
+## prebuilt.x = 42
+## prebuilt2.x = 53
+## prebuilt.y = llop.hide_into_adr32(HiddenGcRef32, prebuilt2)
+## prebuilt2.y = llop.hide_into_adr32(HiddenGcRef32, prebuilt)
+
+## def check(a, b):
+## p = llop.show_from_adr32(lltype.Ptr(S), a.y)
+## assert p == b
+
+## def f(argv):
+## assert prebuilt.x == 42
+## assert prebuilt2.x == 53
+## check(prebuilt, prebuilt2)
+## check(prebuilt2, prebuilt)
+## p = lltype.malloc(S)
+## p.y = llop.hide_into_adr32(HiddenGcRef32, prebuilt)
+## prebuilt2.y = llop.hide_into_adr32(HiddenGcRef32, p)
+## check(p, prebuilt)
+## check(prebuilt2, p)
+## check(prebuilt, prebuilt2)
+## return 0
+
+## fc = self.compile(f)
+## res = fc([])
+## assert res == 0
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_newgc.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_newgc.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_newgc.py Mon Oct 4 15:30:22 2010
@@ -18,6 +18,7 @@
should_be_moving = False
removetypeptr = False
taggedpointers = False
+ compressptr = False
GC_CAN_MOVE = False
GC_CAN_MALLOC_NONMOVABLE = True
GC_CAN_SHRINK_ARRAY = False
@@ -41,7 +42,8 @@
t = Translation(main, standalone=True, gc=cls.gcpolicy,
policy=annpolicy.StrictAnnotatorPolicy(),
taggedpointers=cls.taggedpointers,
- gcremovetypeptr=cls.removetypeptr)
+ gcremovetypeptr=cls.removetypeptr,
+ compressptr=cls.compressptr)
t.disable(['backendopt'])
t.set_backend_extra_options(c_debug_defines=True)
t.rtype()
@@ -623,7 +625,7 @@
assert open(self.filename, 'r').read() == "hello world\n"
os.unlink(self.filename)
- def define_callback_with_collect(cls):
+ def XXXXXXXXXdefine_callback_with_collect(cls):
from pypy.rlib.libffi import ffi_type_pointer, cast_type_to_ffitype,\
CDLL, ffi_type_void, CallbackFuncPtr, ffi_type_sint
from pypy.rpython.lltypesystem import rffi, ll2ctypes
@@ -1333,6 +1335,15 @@
def test_gc_heap_stats(self):
py.test.skip("not implemented")
+class TestMiniMarkGCCompressPtr(TestMiniMarkGC):
+ compressptr = True
+
+ def setup_class(cls):
+ from pypy.config.translationoption import IS_64_BITS
+ if not IS_64_BITS:
+ py.test.skip("only for 64-bits")
+ TestMiniMarkGC.setup_class.im_func(cls)
+
# ____________________________________________________________________
class TaggedPointersTest(object):
From antocuni at codespeak.net Mon Oct 4 15:52:29 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 4 Oct 2010 15:52:29 +0200 (CEST)
Subject: [pypy-svn] r77573 - in pypy/branch/jitffi/pypy:
jit/backend/llsupport jit/backend/llsupport/test
jit/backend/test jit/metainterp/optimizeopt
jit/metainterp/test rlib rlib/test
Message-ID: <20101004135229.EB15F282BE8@codespeak.net>
Author: antocuni
Date: Mon Oct 4 15:52:28 2010
New Revision: 77573
Modified:
pypy/branch/jitffi/pypy/jit/backend/llsupport/ffisupport.py
pypy/branch/jitffi/pypy/jit/backend/llsupport/test/test_ffisupport.py
pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py
pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py
pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py
pypy/branch/jitffi/pypy/rlib/libffi.py
pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
Log:
introduce a new namespace to contain all the ffi types
Modified: pypy/branch/jitffi/pypy/jit/backend/llsupport/ffisupport.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/backend/llsupport/ffisupport.py (original)
+++ pypy/branch/jitffi/pypy/jit/backend/llsupport/ffisupport.py Mon Oct 4 15:52:28 2010
@@ -4,7 +4,7 @@
def get_call_descr_dynamic(ffi_args, ffi_result, extrainfo=None):
"""Get a call descr: the types of result and args are represented by
- rlib.libffi.ffi_type_*"""
+ rlib.libffi.types.*"""
try:
reskind = get_ffi_type_kind(ffi_result)
argkinds = [get_ffi_type_kind(arg) for arg in ffi_args]
@@ -23,41 +23,41 @@
# XXX: maybe we can turn this into a dictionary, but we need to do it at
-# runtime as libffi.ffi_type_* are pointers
+# runtime as libffi.types.* pointers
def get_ffi_type_kind(ffi_type):
- from pypy.rlib import libffi
- if ffi_type is libffi.ffi_type_void:
+ from pypy.rlib.libffi import types
+ if ffi_type is types.void:
return history.VOID
- elif ffi_type is libffi.ffi_type_pointer:
+ elif ffi_type is types.pointer:
return history.REF
- elif ffi_type is libffi.ffi_type_double:
+ elif ffi_type is types.double:
return history.FLOAT
- elif ffi_type is libffi.ffi_type_uchar:
+ elif ffi_type is types.uchar:
return history.INT
- elif ffi_type is libffi.ffi_type_uint8:
+ elif ffi_type is types.uint8:
return history.INT
- elif ffi_type is libffi.ffi_type_schar:
+ elif ffi_type is types.schar:
return history.INT
- elif ffi_type is libffi.ffi_type_sint8:
+ elif ffi_type is types.sint8:
return history.INT
- elif ffi_type is libffi.ffi_type_uint16:
+ elif ffi_type is types.uint16:
return history.INT
- elif ffi_type is libffi.ffi_type_ushort:
+ elif ffi_type is types.ushort:
return history.INT
- elif ffi_type is libffi.ffi_type_sint16:
+ elif ffi_type is types.sint16:
return history.INT
- elif ffi_type is libffi.ffi_type_sshort:
+ elif ffi_type is types.sshort:
return history.INT
- elif ffi_type is libffi.ffi_type_uint:
+ elif ffi_type is types.uint:
return history.INT
- elif ffi_type is libffi.ffi_type_uint32:
+ elif ffi_type is types.uint32:
return history.INT
- elif ffi_type is libffi.ffi_type_sint:
+ elif ffi_type is types.sint:
return history.INT
- elif ffi_type is libffi.ffi_type_sint32:
+ elif ffi_type is types.sint32:
return history.INT
- ## elif ffi_type is libffi.ffi_type_uint64:
+ ## elif ffi_type is types.uint64:
## return history.INT
- ## elif ffi_type is libffi.ffi_type_sint64:
+ ## elif ffi_type is types.sint64:
## return history.INT
raise KeyError
Modified: pypy/branch/jitffi/pypy/jit/backend/llsupport/test/test_ffisupport.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/backend/llsupport/test/test_ffisupport.py (original)
+++ pypy/branch/jitffi/pypy/jit/backend/llsupport/test/test_ffisupport.py Mon Oct 4 15:52:28 2010
@@ -1,18 +1,18 @@
-from pypy.rlib import libffi
+from pypy.rlib.libffi import types
from pypy.jit.backend.llsupport.ffisupport import get_call_descr_dynamic, \
VoidCallDescr, DynamicIntCallDescr
def test_call_descr_dynamic():
- args = [libffi.ffi_type_sint, libffi.ffi_type_double, libffi.ffi_type_pointer]
- descr = get_call_descr_dynamic(args, libffi.ffi_type_void)
+ args = [types.sint, types.double, types.pointer]
+ descr = get_call_descr_dynamic(args, types.void)
assert isinstance(descr, VoidCallDescr)
assert descr.arg_classes == 'ifr'
- descr = get_call_descr_dynamic([], libffi.ffi_type_sint8)
+ descr = get_call_descr_dynamic([], types.sint8)
assert isinstance(descr, DynamicIntCallDescr)
assert descr.get_result_size(False) == 1
- descr = get_call_descr_dynamic([], libffi.ffi_type_float)
+ descr = get_call_descr_dynamic([], types.float)
assert descr is None # single floats are not supported so far
Modified: pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py Mon Oct 4 15:52:28 2010
@@ -421,7 +421,7 @@
assert x == 3.5 - 42
def test_call(self):
- from pypy.rlib.libffi import ffi_type_sint, ffi_type_uchar, ffi_type_sint16
+ from pypy.rlib.libffi import types
def func_int(a, b):
return a + b
@@ -429,9 +429,9 @@
return chr(ord(c) + ord(c1))
functions = [
- (func_int, lltype.Signed, ffi_type_sint, 655360),
- (func_int, rffi.SHORT, ffi_type_sint16, 1213),
- (func_char, lltype.Char, ffi_type_uchar, 12)
+ (func_int, lltype.Signed, types.sint, 655360),
+ (func_int, rffi.SHORT, types.sint16, 1213),
+ (func_char, lltype.Char, types.uchar, 12)
]
for func, TP, ffi_type, num in functions:
Modified: pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py Mon Oct 4 15:52:28 2010
@@ -15,7 +15,7 @@
def _get_signature(self, funcval):
"""
given the funcval, return a tuple (argtypes, restype), where the
- actuall types are libffi.ffi_type_*
+ actuall types are libffi.types.*
The implementation is tricky because we have three possible cases:
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py Mon Oct 4 15:52:28 2010
@@ -2,7 +2,7 @@
import py
from pypy.rlib.jit import JitDriver, hint
from pypy.jit.metainterp.test.test_basic import LLJitMixin
-from pypy.rlib.libffi import CDLL, ffi_type_sint, ffi_type_double, ffi_type_uchar, ArgChain, Func
+from pypy.rlib.libffi import CDLL, types, ArgChain, Func
from pypy.tool.udir import udir
from pypy.translator.tool.cbuild import ExternalCompilationInfo
from pypy.translator.platform import platform
@@ -40,8 +40,8 @@
def f(n):
cdll = CDLL(self.lib_name)
- func = cdll.getpointer('sum_xy', [ffi_type_sint, ffi_type_double],
- ffi_type_sint)
+ func = cdll.getpointer('sum_xy', [types.sint, types.double],
+ types.sint)
while n < 10:
driver.jit_merge_point(n=n, func=func)
driver.can_enter_jit(n=n, func=func)
@@ -66,7 +66,7 @@
def f(n):
cdll = CDLL(self.lib_name)
- func = cdll.getpointer('abs', [ffi_type_double], ffi_type_double)
+ func = cdll.getpointer('abs', [types.double], types.double)
res = 0.0
while n < 10:
driver.jit_merge_point(n=n, func=func, res=res)
@@ -87,8 +87,8 @@
def f(n):
cdll = CDLL(self.lib_name)
- func = cdll.getpointer('cast_to_uchar', [ffi_type_sint],
- ffi_type_uchar)
+ func = cdll.getpointer('cast_to_uchar', [types.sint],
+ types.uchar)
res = 0
while n < 10:
driver.jit_merge_point(n=n, func=func, res=res)
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py Mon Oct 4 15:52:28 2010
@@ -3900,7 +3900,7 @@
# ------------------------------------------------
from pypy.rpython.lltypesystem import llmemory
-from pypy.rlib.libffi import Func, ffi_type_sint, ffi_type_double
+from pypy.rlib.libffi import Func, types
from pypy.jit.metainterp.history import AbstractDescr
class MyCallDescr(AbstractDescr):
@@ -3928,7 +3928,7 @@
def _identityhash(self):
return id(self)
-class TestFfiCall(OptimizeOptTest, LLtypeMixin):
+class TestFfiCall(BaseTestOptimizeOpt, LLtypeMixin):
class namespace:
cpu = LLtypeMixin.cpu
@@ -3936,8 +3936,8 @@
int_float__int = MyCallDescr('if', 'i')
funcptr = FakeLLObject()
func = FakeLLObject(_fake_class=Func,
- argtypes=[ffi_type_sint, ffi_type_double],
- restype=ffi_type_sint)
+ argtypes=[types.sint, types.double],
+ restype=types.sint)
namespace = namespace.__dict__
Modified: pypy/branch/jitffi/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/libffi.py Mon Oct 4 15:52:28 2010
@@ -7,13 +7,31 @@
push_arg_as_ffiptr, c_ffi_call
from pypy.rlib.rdynload import dlopen, dlclose, dlsym, dlsym_byordinal
-def import_types():
- g = globals()
- for key, value in clibffi.__dict__.iteritems():
- if key.startswith('ffi_type_'):
- g[key] = value
-import_types()
-del import_types
+class types(object):
+ """
+ This namespace contains the primitive types you can use to declare the
+ signatures of the ffi functions.
+
+ In general, the name of the types are closely related to the ones of the
+ C-level ffi_type_*: e.g, instead of ffi_type_sint you should use
+ libffi.types.sint.
+
+ However, you should not rely on a perfect correspondence: in particular,
+ the exact meaning of ffi_type_{slong,ulong} changes a lot between libffi
+ versions, so types.slong could be different than ffi_type_slong.
+ """
+
+ @classmethod
+ def _import(cls):
+ prefix = 'ffi_type_'
+ for key, value in clibffi.__dict__.iteritems():
+ if key.startswith(prefix):
+ name = key[len(prefix):]
+ setattr(cls, name, value)
+ cls.slong = clibffi.cast_type_to_ffitype(rffi.LONG)
+ cls.ulong = clibffi.cast_type_to_ffitype(rffi.ULONG)
+
+types._import()
def _fits_into_long(TYPE):
if not isinstance(TYPE, lltype.Number):
@@ -23,7 +41,7 @@
sz = rffi.sizeof(TYPE)
return sz <= rffi.sizeof(rffi.LONG)
-# ----------------------------------------------------------------------
+# ======================================================================
class ArgChain(object):
first = None
@@ -71,6 +89,9 @@
func._push_float(self.floatval, ll_args, i)
+# ======================================================================
+
+
class Func(AbstractFuncPtr):
_immutable_fields_ = ['funcsym', 'argtypes', 'restype']
@@ -168,7 +189,7 @@
def _do_call(self, funcsym, ll_args, RESULT):
# XXX: check len(args)?
ll_result = lltype.nullptr(rffi.CCHARP.TO)
- if self.restype != ffi_type_void:
+ if self.restype != types.void:
ll_result = lltype.malloc(rffi.CCHARP.TO,
intmask(self.restype.c_size),
flavor='raw')
@@ -192,8 +213,8 @@
lltype.free(ll_args, flavor='raw')
-# ----------------------------------------------------------------------
-
+# ======================================================================
+
# XXX: it partially duplicate the code in clibffi.py
class CDLL(object):
Modified: pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_libffi.py Mon Oct 4 15:52:28 2010
@@ -3,8 +3,7 @@
from pypy.rpython.lltypesystem import rffi, lltype
from pypy.rpython.lltypesystem.ll2ctypes import ALLOCATED
from pypy.rlib.test.test_clibffi import BaseFfiTest, get_libm_name
-from pypy.rlib.libffi import CDLL, Func, get_libc_name, ArgChain
-from pypy.rlib.libffi import ffi_type_double, ffi_type_void
+from pypy.rlib.libffi import CDLL, Func, get_libc_name, ArgChain, types
class TestLibffi(BaseFfiTest):
@@ -41,16 +40,16 @@
def test_library_get_func(self):
lib = self.get_libc()
- ptr = lib.getpointer('fopen', [], ffi_type_void)
- py.test.raises(KeyError, lib.getpointer, 'xxxxxxxxxxxxxxx', [], ffi_type_void)
+ ptr = lib.getpointer('fopen', [], types.void)
+ py.test.raises(KeyError, lib.getpointer, 'xxxxxxxxxxxxxxx', [], types.void)
del ptr
del lib
assert not ALLOCATED
def test_call_argchain(self):
libm = self.get_libm()
- pow = libm.getpointer('pow', [ffi_type_double, ffi_type_double],
- ffi_type_double)
+ pow = libm.getpointer('pow', [types.double, types.double],
+ types.double)
argchain = ArgChain()
argchain.float(2.0).float(3.0)
res = pow.call(argchain, rffi.DOUBLE)
From afa at codespeak.net Mon Oct 4 16:19:43 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 4 Oct 2010 16:19:43 +0200 (CEST)
Subject: [pypy-svn] r77574 - pypy/trunk/pypy/jit/tl/spli
Message-ID: <20101004141943.C601B282BE8@codespeak.net>
Author: afa
Date: Mon Oct 4 16:19:42 2010
New Revision: 77574
Modified:
pypy/trunk/pypy/jit/tl/spli/interpreter.py
Log:
The "spli" interpreter uses the host CPython to create compiled files,
be sure to use the same opcodes to interpret it.
This fixes the tests when run on top of CPython 2.7.
Modified: pypy/trunk/pypy/jit/tl/spli/interpreter.py
==============================================================================
--- pypy/trunk/pypy/jit/tl/spli/interpreter.py (original)
+++ pypy/trunk/pypy/jit/tl/spli/interpreter.py Mon Oct 4 16:19:42 2010
@@ -1,12 +1,14 @@
import os
-from pypy.tool import stdlib_opcode as opcode
+from pypy.tool import stdlib_opcode
from pypy.jit.tl.spli import objects, pycode
-from pypy.tool.stdlib_opcode import unrolling_opcode_descs
-from pypy.tool.stdlib_opcode import opcode_method_names
from pypy.rlib.unroll import unrolling_iterable
from pypy.rlib.jit import JitDriver, hint, dont_look_inside
from pypy.rlib.objectmodel import we_are_translated
+opcode_method_names = stdlib_opcode.host_bytecode_spec.method_names
+unrolling_opcode_descs = unrolling_iterable(
+ stdlib_opcode.host_bytecode_spec.ordered_opdescs)
+HAVE_ARGUMENT = stdlib_opcode.host_HAVE_ARGUMENT
compare_ops = [
"cmp_lt", # "<"
@@ -79,7 +81,7 @@
self.stack_depth = hint(self.stack_depth, promote=True)
op = ord(code[instr_index])
instr_index += 1
- if op >= opcode.HAVE_ARGUMENT:
+ if op >= HAVE_ARGUMENT:
low = ord(code[instr_index])
hi = ord(code[instr_index + 1])
oparg = (hi << 8) | low
@@ -183,6 +185,12 @@
next_instr += arg
return next_instr
+ def POP_JUMP_IF_FALSE(self, arg, next_instr, code):
+ w_cond = self.pop()
+ if not w_cond.is_true():
+ next_instr = arg
+ return next_instr
+
def JUMP_FORWARD(self, arg, next_instr, code):
return next_instr + arg
From afa at codespeak.net Mon Oct 4 16:23:11 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 4 Oct 2010 16:23:11 +0200 (CEST)
Subject: [pypy-svn] r77575 - pypy/branch/fast-forward/pypy/jit/tl/spli
Message-ID: <20101004142311.9C8FD282BE8@codespeak.net>
Author: afa
Date: Mon Oct 4 16:23:10 2010
New Revision: 77575
Modified:
pypy/branch/fast-forward/pypy/jit/tl/spli/interpreter.py
Log:
Merge r77574 from trunk
Modified: pypy/branch/fast-forward/pypy/jit/tl/spli/interpreter.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/tl/spli/interpreter.py (original)
+++ pypy/branch/fast-forward/pypy/jit/tl/spli/interpreter.py Mon Oct 4 16:23:10 2010
@@ -1,12 +1,14 @@
import os
-from pypy.tool import stdlib_opcode as opcode
+from pypy.tool import stdlib_opcode
from pypy.jit.tl.spli import objects, pycode
-from pypy.tool.stdlib_opcode import unrolling_opcode_descs
-from pypy.tool.stdlib_opcode import opcode_method_names
from pypy.rlib.unroll import unrolling_iterable
from pypy.rlib.jit import JitDriver, hint, dont_look_inside
from pypy.rlib.objectmodel import we_are_translated
+opcode_method_names = stdlib_opcode.host_bytecode_spec.method_names
+unrolling_opcode_descs = unrolling_iterable(
+ stdlib_opcode.host_bytecode_spec.ordered_opdescs)
+HAVE_ARGUMENT = stdlib_opcode.host_HAVE_ARGUMENT
compare_ops = [
"cmp_lt", # "<"
@@ -79,7 +81,7 @@
self.stack_depth = hint(self.stack_depth, promote=True)
op = ord(code[instr_index])
instr_index += 1
- if op >= opcode.HAVE_ARGUMENT:
+ if op >= HAVE_ARGUMENT:
low = ord(code[instr_index])
hi = ord(code[instr_index + 1])
oparg = (hi << 8) | low
@@ -183,6 +185,12 @@
next_instr += arg
return next_instr
+ def POP_JUMP_IF_FALSE(self, arg, next_instr, code):
+ w_cond = self.pop()
+ if not w_cond.is_true():
+ next_instr = arg
+ return next_instr
+
def JUMP_FORWARD(self, arg, next_instr, code):
return next_instr + arg
From antocuni at codespeak.net Mon Oct 4 16:50:07 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 4 Oct 2010 16:50:07 +0200 (CEST)
Subject: [pypy-svn] r77576 - pypy/branch/jitffi/pypy/jit/backend/test
Message-ID: <20101004145007.932BB282B90@codespeak.net>
Author: antocuni
Date: Mon Oct 4 16:50:05 2010
New Revision: 77576
Modified:
pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py
Log:
fix the test to use the new libffi interface
Modified: pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py Mon Oct 4 16:50:05 2010
@@ -517,16 +517,15 @@
assert res.value == func_ints(*args)
def test_call_to_c_function(self):
- # XXX: fix this to use libffi instead of clibffi
- from pypy.rlib.clibffi import CDLL, ffi_type_uchar, ffi_type_sint
+ from pypy.rlib.libffi import CDLL, types, ArgChain
libc = CDLL('libc.so.6')
- c_tolower = libc.getpointer('tolower', [ffi_type_uchar], ffi_type_sint)
- c_tolower.push_arg('A')
- assert c_tolower.call(lltype.Signed) == ord('a')
+ c_tolower = libc.getpointer('tolower', [types.uchar], types.sint)
+ argchain = ArgChain().int(ord('A'))
+ assert c_tolower.call(argchain, rffi.INT) == ord('a')
func_adr = llmemory.cast_ptr_to_adr(c_tolower.funcsym)
funcbox = ConstInt(heaptracker.adr2int(func_adr))
- calldescr = self.cpu.calldescrof_dynamic([ffi_type_uchar], ffi_type_sint)
+ calldescr = self.cpu.calldescrof_dynamic([types.uchar], types.sint)
res = self.execute_operation(rop.CALL,
[funcbox, BoxInt(ord('A'))],
'int',
From arigo at codespeak.net Mon Oct 4 16:51:18 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 4 Oct 2010 16:51:18 +0200 (CEST)
Subject: [pypy-svn] r77577 - pypy/branch/minor-cleanup
Message-ID: <20101004145118.E9B91282B90@codespeak.net>
Author: arigo
Date: Mon Oct 4 16:51:17 2010
New Revision: 77577
Added:
pypy/branch/minor-cleanup/
- copied from r77576, pypy/trunk/
Log:
Temporary branch.
From arigo at codespeak.net Mon Oct 4 16:51:59 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 4 Oct 2010 16:51:59 +0200 (CEST)
Subject: [pypy-svn] r77578 - in pypy/branch/minor-cleanup/pypy: annotation
annotation/test module/sys objspace/std rpython rpython/test
translator/goal
Message-ID: <20101004145159.9584C282B90@codespeak.net>
Author: arigo
Date: Mon Oct 4 16:51:57 2010
New Revision: 77578
Removed:
pypy/branch/minor-cleanup/pypy/rpython/rspecialcase.py
pypy/branch/minor-cleanup/pypy/rpython/test/test_rspecialcase.py
Modified:
pypy/branch/minor-cleanup/pypy/annotation/policy.py
pypy/branch/minor-cleanup/pypy/annotation/test/test_annrpython.py
pypy/branch/minor-cleanup/pypy/module/sys/__init__.py
pypy/branch/minor-cleanup/pypy/module/sys/state.py
pypy/branch/minor-cleanup/pypy/objspace/std/fake.py
pypy/branch/minor-cleanup/pypy/objspace/std/objspace.py
pypy/branch/minor-cleanup/pypy/rpython/rtyper.py
pypy/branch/minor-cleanup/pypy/translator/goal/ann_override.py
Log:
Kill an old way to do specialization at the annotator level.
Modified: pypy/branch/minor-cleanup/pypy/annotation/policy.py
==============================================================================
--- pypy/branch/minor-cleanup/pypy/annotation/policy.py (original)
+++ pypy/branch/minor-cleanup/pypy/annotation/policy.py Mon Oct 4 16:51:57 2010
@@ -1,4 +1,4 @@
-# base annotation policy for overrides and specialization
+# base annotation policy for specialization
from pypy.annotation.specialize import default_specialize as default
from pypy.annotation.specialize import specialize_argvalue, specialize_argtype, specialize_arglistitemtype
from pypy.annotation.specialize import memo
@@ -41,7 +41,7 @@
if directive is None:
return pol.default_specialize
- # specialize|override:name[(args)]
+ # specialize[(args)]
directive_parts = directive.split('(', 1)
if len(directive_parts) == 1:
[name] = directive_parts
@@ -60,14 +60,6 @@
except AttributeError:
raise AttributeError("%r specialize tag not defined in annotation"
"policy %s" % (name, pol))
- if directive.startswith('override:'):
- # different signature: override__xyz(*args_s)
- if parms:
- raise Exception, "override:* specialisations don't support parameters"
- def specialize_override(funcdesc, args_s):
- funcdesc.overridden = True
- return specializer(*args_s)
- return specialize_override
else:
if not parms:
return specializer
@@ -92,9 +84,5 @@
from pypy.rpython.annlowlevel import LowLevelAnnotatorPolicy
return LowLevelAnnotatorPolicy.specialize__ll_and_arg(*args)
- def override__ignore(pol, *args):
- bk = getbookkeeper()
- return bk.immutablevalue(None)
-
class StrictAnnotatorPolicy(AnnotatorPolicy):
allow_someobjects = False
Modified: pypy/branch/minor-cleanup/pypy/annotation/test/test_annrpython.py
==============================================================================
--- pypy/branch/minor-cleanup/pypy/annotation/test/test_annrpython.py (original)
+++ pypy/branch/minor-cleanup/pypy/annotation/test/test_annrpython.py Mon Oct 4 16:51:57 2010
@@ -766,28 +766,6 @@
s = a.build_types(f, [list])
assert s.classdef is a.bookkeeper.getuniqueclassdef(IndexError) # KeyError ignored because l is a list
- def test_overrides(self):
- excs = []
- def record_exc(e):
- """NOT_RPYTHON"""
- excs.append(sys.exc_info)
- record_exc._annspecialcase_ = "override:record_exc"
- def g():
- pass
- def f():
- try:
- g()
- except Exception, e:
- record_exc(e)
- class MyAnnotatorPolicy(policy.AnnotatorPolicy):
-
- def override__record_exc(pol, s_e):
- return a.bookkeeper.immutablevalue(None)
-
- a = self.RPythonAnnotator(policy=MyAnnotatorPolicy())
- s = a.build_types(f, [])
- assert s.const is None
-
def test_freeze_protocol(self):
class Stuff:
def __init__(self, flag):
Modified: pypy/branch/minor-cleanup/pypy/module/sys/__init__.py
==============================================================================
--- pypy/branch/minor-cleanup/pypy/module/sys/__init__.py (original)
+++ pypy/branch/minor-cleanup/pypy/module/sys/__init__.py Mon Oct 4 16:51:57 2010
@@ -7,13 +7,15 @@
"""Sys Builtin Module. """
def __init__(self, space, w_name):
"""NOT_RPYTHON""" # because parent __init__ isn't
+ if space.config.translating:
+ del self.__class__.interpleveldefs['pypy_getudir']
super(Module, self).__init__(space, w_name)
self.checkinterval = 100
self.recursionlimit = 100
self.w_default_encoder = None
self.defaultencoding = "ascii"
self.filesystemencoding = None
-
+
interpleveldefs = {
'__name__' : '(space.wrap("sys"))',
'__doc__' : '(space.wrap("PyPy sys module"))',
@@ -37,7 +39,7 @@
'argv' : 'state.get(space).w_argv',
'warnoptions' : 'state.get(space).w_warnoptions',
'builtin_module_names' : 'state.w_None',
- 'pypy_getudir' : 'state.pypy_getudir',
+ 'pypy_getudir' : 'state.pypy_getudir', # not translated
'pypy_initial_path' : 'state.pypy_initial_path',
'_getframe' : 'vm._getframe',
Modified: pypy/branch/minor-cleanup/pypy/module/sys/state.py
==============================================================================
--- pypy/branch/minor-cleanup/pypy/module/sys/state.py (original)
+++ pypy/branch/minor-cleanup/pypy/module/sys/state.py Mon Oct 4 16:51:57 2010
@@ -95,15 +95,8 @@
def getio(space):
return space.fromcache(IOState)
-def _pypy_getudir(space):
- """NOT_RPYTHON"""
+def pypy_getudir(space):
+ """NOT_RPYTHON
+ (should be removed from interpleveldefs before translation)"""
from pypy.tool.udir import udir
return space.wrap(str(udir))
-_pypy_getudir._annspecialcase_ = "override:ignore"
-
-# we need the indirection because this function will live in a dictionary with other
-# RPYTHON functions and share call sites with them. Better it not be a special-case
-# directly.
-def pypy_getudir(space):
- return _pypy_getudir(space)
-
Modified: pypy/branch/minor-cleanup/pypy/objspace/std/fake.py
==============================================================================
--- pypy/branch/minor-cleanup/pypy/objspace/std/fake.py (original)
+++ pypy/branch/minor-cleanup/pypy/objspace/std/fake.py Mon Oct 4 16:51:57 2010
@@ -21,7 +21,6 @@
#debug_print("faking obj %s" % x)
ft = fake_type(type(x))
return ft(space, x)
-fake_object._annspecialcase_ = "override:fake_object"
import sys
@@ -47,7 +46,6 @@
w_exc = space.wrap(exc)
w_value = space.wrap(value)
raise OperationError, OperationError(w_exc, w_value), tb
-wrap_exception._annspecialcase_ = "override:ignore"
def fake_type(cpy_type):
assert type(cpy_type) is type
Modified: pypy/branch/minor-cleanup/pypy/objspace/std/objspace.py
==============================================================================
--- pypy/branch/minor-cleanup/pypy/objspace/std/objspace.py (original)
+++ pypy/branch/minor-cleanup/pypy/objspace/std/objspace.py Mon Oct 4 16:51:57 2010
@@ -242,7 +242,6 @@
w_result = getattr(self, 'w_' + x.__name__)
return w_result
return None
- wrap_exception_cls._annspecialcase_ = "override:wrap_exception_cls"
def unwrap(self, w_obj):
if isinstance(w_obj, Wrappable):
Modified: pypy/branch/minor-cleanup/pypy/rpython/rtyper.py
==============================================================================
--- pypy/branch/minor-cleanup/pypy/rpython/rtyper.py (original)
+++ pypy/branch/minor-cleanup/pypy/rpython/rtyper.py Mon Oct 4 16:51:57 2010
@@ -421,7 +421,7 @@
assert noexclink.exitcase is None
if pos == "removed":
# the exception cannot actually occur at all.
- # See for example rspecialcase.rtype_call_specialcase().
+ # This is set by calling exception_cannot_occur().
# We just remove all exception links.
block.exitswitch = None
block.exits = block.exits[:1]
@@ -1019,7 +1019,7 @@
from pypy.rpython import rint, rbool, rfloat
from pypy.rpython import rrange
from pypy.rpython import rstr, rdict, rlist
-from pypy.rpython import rclass, rbuiltin, rpbc, rspecialcase
+from pypy.rpython import rclass, rbuiltin, rpbc
from pypy.rpython import rexternalobj
from pypy.rpython import rptr
from pypy.rpython import rgeneric
Modified: pypy/branch/minor-cleanup/pypy/translator/goal/ann_override.py
==============================================================================
--- pypy/branch/minor-cleanup/pypy/translator/goal/ann_override.py (original)
+++ pypy/branch/minor-cleanup/pypy/translator/goal/ann_override.py Mon Oct 4 16:51:57 2010
@@ -27,21 +27,6 @@
pol.pypytypes = {}
pol.single_space = single_space
- #def override__wrap_exception_cls(pol, space, x):
- # import pypy.objspace.std.typeobject as typeobject
- # clsdef = getbookkeeper().getuniqueclassdef(typeobject.W_TypeObject)
- # return annmodel.SomeInstance(clsdef, can_be_None=True)
- #
- #def override__fake_object(pol, space, x):
- # from pypy.interpreter import typedef
- # clsdef = getbookkeeper().getuniqueclassdef(typedef.W_Root)
- # return annmodel.SomeInstance(clsdef)
- #
- #def override__cpy_compile(pol, self, source, filename, mode, flags):
- # from pypy.interpreter import pycode
- # clsdef = getbookkeeper().getuniqueclassdef(pycode.PyCode)
- # return annmodel.SomeInstance(clsdef)
-
def specialize__wrap(pol, funcdesc, args_s):
from pypy.interpreter.baseobjspace import Wrappable
from pypy.annotation.classdef import ClassDef
From arigo at codespeak.net Mon Oct 4 16:54:04 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 4 Oct 2010 16:54:04 +0200 (CEST)
Subject: [pypy-svn] r77579 - pypy/branch/larger-writebarrier
Message-ID: <20101004145404.DB0E9282B90@codespeak.net>
Author: arigo
Date: Mon Oct 4 16:54:03 2010
New Revision: 77579
Removed:
pypy/branch/larger-writebarrier/
Log:
Remove merged branch.
From afa at codespeak.net Mon Oct 4 19:01:36 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 4 Oct 2010 19:01:36 +0200 (CEST)
Subject: [pypy-svn] r77580 - pypy/branch/fast-forward/pypy/doc/config
Message-ID: <20101004170136.28210282B90@codespeak.net>
Author: afa
Date: Mon Oct 4 19:01:35 2010
New Revision: 77580
Added:
pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._multiprocessing.txt (contents, props changed)
Log:
Add missing doc file
Added: pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._multiprocessing.txt
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._multiprocessing.txt Mon Oct 4 19:01:35 2010
@@ -0,0 +1,2 @@
+Use the '_multiprocessing' module.
+Used by the 'multiprocessing standard lib module. This module is expected to be working and is included by default.
From afa at codespeak.net Mon Oct 4 19:24:47 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 4 Oct 2010 19:24:47 +0200 (CEST)
Subject: [pypy-svn] r77581 - in pypy/branch/fast-forward/pypy: doc/config
module/_io module/_io/test
Message-ID: <20101004172447.4EAA9282B90@codespeak.net>
Author: afa
Date: Mon Oct 4 19:24:45 2010
New Revision: 77581
Added:
pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._io.txt (contents, props changed)
pypy/branch/fast-forward/pypy/module/_io/ (props changed)
pypy/branch/fast-forward/pypy/module/_io/__init__.py (contents, props changed)
pypy/branch/fast-forward/pypy/module/_io/interp_io.py (contents, props changed)
pypy/branch/fast-forward/pypy/module/_io/test/ (props changed)
pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (contents, props changed)
Log:
Add minimal stub of a _io module,
enough to "import io" though.
Added: pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._io.txt
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._io.txt Mon Oct 4 19:24:45 2010
@@ -0,0 +1,2 @@
+Use the '_io module.
+Used by the 'io' standard lib module. This module is expected to be working and is included by default.
Added: pypy/branch/fast-forward/pypy/module/_io/__init__.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/module/_io/__init__.py Mon Oct 4 19:24:45 2010
@@ -0,0 +1,30 @@
+from pypy.interpreter.mixedmodule import MixedModule
+import sys
+
+class Module(MixedModule):
+
+ appleveldefs = {
+ }
+
+ interpleveldefs = {
+ 'DEFAULT_BUFFER_SIZE': 'space.wrap(interp_io.DEFAULT_BUFFER_SIZE)',
+ 'BlockingIOError': 'interp_io.W_BlockingIOError',
+ '_IOBase': 'interp_io.W_IOBase',
+ '_RawIOBase': 'interp_io.W_RawIOBase',
+ '_BufferedIOBase': 'interp_io.W_BufferedIOBase',
+ '_TextIOBase': 'interp_io.W_TextIOBase',
+
+ 'FileIO': 'interp_io.W_FileIO',
+ 'BytesIO': 'interp_io.W_BytesIO',
+ 'StringIO': 'interp_io.W_StringIO',
+ 'BufferedReader': 'interp_io.W_BufferedReader',
+ 'BufferedWriter': 'interp_io.W_BufferedWriter',
+ 'BufferedRWPair': 'interp_io.W_BufferedRWPair',
+ 'BufferedRandom': 'interp_io.W_BufferedRandom',
+ 'TextIOWrapper': 'interp_io.W_TextIOWrapper',
+ }
+
+ def startup(self, space):
+ for name in """UnsupportedOperation open IncrementalNewlineDecoder
+ """.split():
+ space.setattr(self, space.wrap(name), space.w_None)
Added: pypy/branch/fast-forward/pypy/module/_io/interp_io.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_io.py Mon Oct 4 19:24:45 2010
@@ -0,0 +1,94 @@
+from pypy.interpreter.baseobjspace import ObjSpace, Wrappable
+from pypy.interpreter.typedef import TypeDef, interp_attrproperty
+from pypy.module.exceptions.interp_exceptions import W_IOError
+
+DEFAULT_BUFFER_SIZE = 8192
+
+
+class W_BlockingIOError(W_IOError):
+ def __init__(self, space):
+ W_IOError.__init__(self, space)
+ self.written = 0
+
+ def descr_init(self, space, w_errno, w_strerror, written=0):
+ W_IOError.descr_init(self, space, [w_errno, w_strerror])
+ self.written = written
+
+W_BlockingIOError.typedef = TypeDef(
+ 'BlockingIOError',
+ __doc__ = ("Exception raised when I/O would block "
+ "on a non-blocking I/O stream"),
+ characters_written = interp_attrproperty('written', W_BlockingIOError),
+ )
+
+class W_IOBase(Wrappable):
+ pass
+W_IOBase.typedef = TypeDef(
+ '_IOBase',
+ )
+
+class W_RawIOBase(W_IOBase):
+ pass
+W_RawIOBase.typedef = TypeDef(
+ '_RawIOBase', W_IOBase.typedef,
+ )
+
+class W_BufferedIOBase(W_IOBase):
+ pass
+W_BufferedIOBase.typedef = TypeDef(
+ '_BufferedIOBase', W_IOBase.typedef,
+ )
+
+class W_TextIOBase(W_IOBase):
+ pass
+W_TextIOBase.typedef = TypeDef(
+ '_TextIOBase', W_IOBase.typedef,
+ )
+
+class W_FileIO(W_RawIOBase):
+ pass
+W_FileIO.typedef = TypeDef(
+ 'FileIO', W_RawIOBase.typedef,
+ )
+
+class W_BytesIO(W_BufferedIOBase):
+ pass
+W_BytesIO.typedef = TypeDef(
+ 'BytesIO', W_BufferedIOBase.typedef,
+ )
+
+class W_StringIO(W_TextIOBase):
+ pass
+W_StringIO.typedef = TypeDef(
+ 'StringIO', W_TextIOBase.typedef,
+ )
+
+class W_BufferedReader(W_BufferedIOBase):
+ pass
+W_BufferedReader.typedef = TypeDef(
+ 'BufferedReader', W_BufferedIOBase.typedef,
+ )
+
+class W_BufferedWriter(W_BufferedIOBase):
+ pass
+W_BufferedWriter.typedef = TypeDef(
+ 'BufferedWriter', W_BufferedIOBase.typedef,
+ )
+
+class W_BufferedRWPair(W_BufferedIOBase):
+ pass
+W_BufferedRWPair.typedef = TypeDef(
+ 'BufferedRWPair', W_BufferedIOBase.typedef,
+ )
+
+class W_BufferedRandom(W_BufferedIOBase):
+ pass
+W_BufferedRandom.typedef = TypeDef(
+ 'BufferedRandom', W_BufferedIOBase.typedef,
+ )
+
+class W_TextIOWrapper(W_TextIOBase):
+ pass
+W_TextIOWrapper.typedef = TypeDef(
+ 'TextIOWrapper', W_TextIOBase.typedef,
+ )
Added: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Mon Oct 4 19:24:45 2010
@@ -0,0 +1,8 @@
+from pypy.conftest import gettestobjspace
+
+class AppTestIoModule:
+ def setup_class(cls):
+ cls.space = gettestobjspace(usemodules=['_io'])
+
+ def test_import(self):
+ import io
From afa at codespeak.net Mon Oct 4 19:27:35 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 4 Oct 2010 19:27:35 +0200 (CEST)
Subject: [pypy-svn] r77582 - pypy/branch/fast-forward/pypy/config
Message-ID: <20101004172735.DE62136C225@codespeak.net>
Author: afa
Date: Mon Oct 4 19:27:34 2010
New Revision: 77582
Modified:
pypy/branch/fast-forward/pypy/config/pypyoption.py
Log:
Translation should use the _io module by default
Modified: pypy/branch/fast-forward/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/fast-forward/pypy/config/pypyoption.py (original)
+++ pypy/branch/fast-forward/pypy/config/pypyoption.py Mon Oct 4 19:27:34 2010
@@ -19,7 +19,7 @@
default_modules.update(dict.fromkeys(
["_codecs", "gc", "_weakref", "marshal", "errno", "imp",
"math", "_sre", "_pickle_support", "operator",
- "parser", "symbol", "token", "_ast", "_random", "__pypy__",
+ "parser", "symbol", "token", "_ast", "_io", "_random", "__pypy__",
"_testing"]))
From afa at codespeak.net Mon Oct 4 19:34:22 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 4 Oct 2010 19:34:22 +0200 (CEST)
Subject: [pypy-svn] r77583 - pypy/branch/fast-forward/pypy/module/_io
Message-ID: <20101004173422.0B35B36C225@codespeak.net>
Author: afa
Date: Mon Oct 4 19:34:21 2010
New Revision: 77583
Modified:
pypy/branch/fast-forward/pypy/module/_io/__init__.py
Log:
Try to fix translation
Modified: pypy/branch/fast-forward/pypy/module/_io/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/__init__.py Mon Oct 4 19:34:21 2010
@@ -22,9 +22,8 @@
'BufferedRWPair': 'interp_io.W_BufferedRWPair',
'BufferedRandom': 'interp_io.W_BufferedRandom',
'TextIOWrapper': 'interp_io.W_TextIOWrapper',
- }
- def startup(self, space):
- for name in """UnsupportedOperation open IncrementalNewlineDecoder
- """.split():
- space.setattr(self, space.wrap(name), space.w_None)
+ 'open': 'space.w_None',
+ 'UnsupportedOperation': 'space.w_None',
+ 'IncrementalNewlineDecoder': 'space.w_None',
+ }
From afa at codespeak.net Mon Oct 4 20:02:10 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 4 Oct 2010 20:02:10 +0200 (CEST)
Subject: [pypy-svn] r77584 - pypy/branch/fast-forward/pypy/module/_io
Message-ID: <20101004180210.B6552282B90@codespeak.net>
Author: afa
Date: Mon Oct 4 20:02:09 2010
New Revision: 77584
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_io.py
Log:
more translation fixes
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_io.py Mon Oct 4 20:02:09 2010
@@ -1,5 +1,6 @@
-from pypy.interpreter.baseobjspace import ObjSpace, Wrappable
+from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root
from pypy.interpreter.typedef import TypeDef, interp_attrproperty
+from pypy.interpreter.gateway import interp2app, Arguments
from pypy.module.exceptions.interp_exceptions import W_IOError
DEFAULT_BUFFER_SIZE = 8192
@@ -10,14 +11,23 @@
W_IOError.__init__(self, space)
self.written = 0
+ def descr_new(space, w_subtype, __args__):
+ self = space.allocate_instance(W_BlockingIOError, w_subtype)
+ W_BlockingIOError.__init__(self, space)
+ return space.wrap(self)
+ descr_new.unwrap_spec = [ObjSpace, W_Root, Arguments]
+
def descr_init(self, space, w_errno, w_strerror, written=0):
W_IOError.descr_init(self, space, [w_errno, w_strerror])
self.written = written
+ descr_init.unwrap_spec = ['self', ObjSpace, W_Root, W_Root, int]
W_BlockingIOError.typedef = TypeDef(
'BlockingIOError',
__doc__ = ("Exception raised when I/O would block "
"on a non-blocking I/O stream"),
+ __new__ = interp2app(W_BlockingIOError.descr_new.im_func),
+ __init__ = interp2app(W_BlockingIOError.descr_init),
characters_written = interp_attrproperty('written', W_BlockingIOError),
)
From afa at codespeak.net Mon Oct 4 20:51:56 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 4 Oct 2010 20:51:56 +0200 (CEST)
Subject: [pypy-svn] r77585 - pypy/branch/fast-forward/pypy/rlib/test
Message-ID: <20101004185156.DD046282B90@codespeak.net>
Author: afa
Date: Mon Oct 4 20:51:55 2010
New Revision: 77585
Modified:
pypy/branch/fast-forward/pypy/rlib/test/test_rstring.py
Log:
Fix test now that UnicodeBuilder has been made stricter
Modified: pypy/branch/fast-forward/pypy/rlib/test/test_rstring.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/test/test_rstring.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/test/test_rstring.py Mon Oct 4 20:51:55 2010
@@ -39,7 +39,7 @@
s.append(u'a')
s.append(u'abc')
s.append_slice(u'abcdef', 1, 2)
- s.append_multiple_char('d', 4)
+ s.append_multiple_char(u'd', 4)
assert s.build() == 'aabcbdddd'
assert isinstance(s.build(), unicode)
From dan at codespeak.net Mon Oct 4 22:07:47 2010
From: dan at codespeak.net (dan at codespeak.net)
Date: Mon, 4 Oct 2010 22:07:47 +0200 (CEST)
Subject: [pypy-svn] r77586 - pypy/trunk/pypy/doc
Message-ID: <20101004200747.BD1FD282B90@codespeak.net>
Author: dan
Date: Mon Oct 4 22:07:44 2010
New Revision: 77586
Modified:
pypy/trunk/pypy/doc/docindex.txt
Log:
Fixed typo in docindex (Thanks ArneBab)
Modified: pypy/trunk/pypy/doc/docindex.txt
==============================================================================
--- pypy/trunk/pypy/doc/docindex.txt (original)
+++ pypy/trunk/pypy/doc/docindex.txt Mon Oct 4 22:07:44 2010
@@ -84,7 +84,7 @@
PyPy's own tests `summary`_, daily updated, run through BuildBot infrastructure.
You can also find CPython's compliance tests run with compiled ``pypy-c``
-exeuctables there.
+executables there.
information dating from early 2007:
From afa at codespeak.net Mon Oct 4 23:11:00 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 4 Oct 2010 23:11:00 +0200 (CEST)
Subject: [pypy-svn] r77587 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101004211100.C95DD282B90@codespeak.net>
Author: afa
Date: Mon Oct 4 23:10:58 2010
New Revision: 77587
Added:
pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_stringio.py
Modified:
pypy/branch/fast-forward/pypy/module/_io/__init__.py
pypy/branch/fast-forward/pypy/module/_io/interp_io.py
Log:
Start to implement io.StringIO,
enough for py/_plugin/pytest_helpconfig.py to use it.
Modified: pypy/branch/fast-forward/pypy/module/_io/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/__init__.py Mon Oct 4 23:10:58 2010
@@ -16,7 +16,7 @@
'FileIO': 'interp_io.W_FileIO',
'BytesIO': 'interp_io.W_BytesIO',
- 'StringIO': 'interp_io.W_StringIO',
+ 'StringIO': 'interp_stringio.W_StringIO',
'BufferedReader': 'interp_io.W_BufferedReader',
'BufferedWriter': 'interp_io.W_BufferedWriter',
'BufferedRWPair': 'interp_io.W_BufferedRWPair',
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_io.py Mon Oct 4 23:10:58 2010
@@ -67,12 +67,6 @@
'BytesIO', W_BufferedIOBase.typedef,
)
-class W_StringIO(W_TextIOBase):
- pass
-W_StringIO.typedef = TypeDef(
- 'StringIO', W_TextIOBase.typedef,
- )
-
class W_BufferedReader(W_BufferedIOBase):
pass
W_BufferedReader.typedef = TypeDef(
Added: pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py Mon Oct 4 23:10:58 2010
@@ -0,0 +1,68 @@
+from pypy.module._io.interp_io import W_TextIOBase
+from pypy.interpreter.typedef import TypeDef
+from pypy.interpreter.gateway import interp2app, unwrap_spec
+from pypy.interpreter.error import operationerrfmt
+from pypy.interpreter.baseobjspace import ObjSpace, W_Root
+
+class W_StringIO(W_TextIOBase):
+ def __init__(self):
+ self.buf = []
+ self.pos = 0
+
+ def _check_closed(self):
+ pass
+ def _check_initialized(self):
+ pass
+
+ @unwrap_spec(ObjSpace, W_Root)
+ def descr_new(space, w_subtype):
+ self = space.allocate_instance(W_StringIO, w_subtype)
+ W_StringIO.__init__(self)
+ return space.wrap(self)
+
+ def resize_buffer(self, newlength):
+ if len(self.buf) > newlength:
+ self.buf = self.buf[:newlength]
+ if len(self.buf) < newlength:
+ self.buf.extend([u'\0'] * (newlength - len(self.buf)))
+
+ def write(self, string):
+ # XXX self.decoder
+ decoded = string
+ # XXX writenl
+
+ length = len(decoded)
+ if self.pos + length > len(self.buf):
+ self.resize_buffer(self.pos + length)
+
+ for i in range(length):
+ self.buf[self.pos + i] = string[i]
+ self.pos += length
+
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def write_w(self, space, w_obj):
+ self._check_initialized()
+ if not space.isinstance_w(w_obj, space.w_unicode):
+ raise operationerrfmt(space.w_TypeError,
+ "string argument expected, got '%s'",
+ space.type(self).getname(space, '?'))
+ self._check_closed()
+ string = space.unicode_w(w_obj)
+ size = len(string)
+ if size:
+ self.write(string)
+ return space.wrap(size)
+
+ @unwrap_spec('self', ObjSpace)
+ def getvalue_w(self, space):
+ self._check_initialized()
+ self._check_closed()
+ return space.wrap(u''.join(self.buf))
+
+W_StringIO.typedef = TypeDef(
+ 'StringIO', W_TextIOBase.typedef,
+ __new__ = interp2app(W_StringIO.descr_new.im_func),
+ write=interp2app(W_StringIO.write_w),
+ getvalue=interp2app(W_StringIO.getvalue_w),
+ )
+
Added: pypy/branch/fast-forward/pypy/module/_io/test/test_stringio.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_stringio.py Mon Oct 4 23:10:58 2010
@@ -0,0 +1,7 @@
+class AppTestStringIO:
+ def test_stringio(self):
+ import io
+ sio = io.StringIO()
+ sio.write(u'Hello ')
+ sio.write(u'world')
+ assert sio.getvalue() == u'Hello world'
From afa at codespeak.net Mon Oct 4 23:11:50 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 4 Oct 2010 23:11:50 +0200 (CEST)
Subject: [pypy-svn] r77588 - in pypy/branch/fast-forward/pypy/module: _io
_io/test _multiprocessing/test
Message-ID: <20101004211150.4813C282B90@codespeak.net>
Author: afa
Date: Mon Oct 4 23:11:48 2010
New Revision: 77588
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py (contents, props changed)
pypy/branch/fast-forward/pypy/module/_io/test/test_stringio.py (contents, props changed)
pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py (contents, props changed)
Log:
fixeol
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py Mon Oct 4 23:11:48 2010
@@ -1,68 +1,68 @@
-from pypy.module._io.interp_io import W_TextIOBase
-from pypy.interpreter.typedef import TypeDef
-from pypy.interpreter.gateway import interp2app, unwrap_spec
-from pypy.interpreter.error import operationerrfmt
-from pypy.interpreter.baseobjspace import ObjSpace, W_Root
-
-class W_StringIO(W_TextIOBase):
- def __init__(self):
- self.buf = []
- self.pos = 0
-
- def _check_closed(self):
- pass
- def _check_initialized(self):
- pass
-
- @unwrap_spec(ObjSpace, W_Root)
- def descr_new(space, w_subtype):
- self = space.allocate_instance(W_StringIO, w_subtype)
- W_StringIO.__init__(self)
- return space.wrap(self)
-
- def resize_buffer(self, newlength):
- if len(self.buf) > newlength:
- self.buf = self.buf[:newlength]
- if len(self.buf) < newlength:
- self.buf.extend([u'\0'] * (newlength - len(self.buf)))
-
- def write(self, string):
- # XXX self.decoder
- decoded = string
- # XXX writenl
-
- length = len(decoded)
- if self.pos + length > len(self.buf):
- self.resize_buffer(self.pos + length)
-
- for i in range(length):
- self.buf[self.pos + i] = string[i]
- self.pos += length
-
- @unwrap_spec('self', ObjSpace, W_Root)
- def write_w(self, space, w_obj):
- self._check_initialized()
- if not space.isinstance_w(w_obj, space.w_unicode):
- raise operationerrfmt(space.w_TypeError,
- "string argument expected, got '%s'",
- space.type(self).getname(space, '?'))
- self._check_closed()
- string = space.unicode_w(w_obj)
- size = len(string)
- if size:
- self.write(string)
- return space.wrap(size)
-
- @unwrap_spec('self', ObjSpace)
- def getvalue_w(self, space):
- self._check_initialized()
- self._check_closed()
- return space.wrap(u''.join(self.buf))
-
-W_StringIO.typedef = TypeDef(
- 'StringIO', W_TextIOBase.typedef,
- __new__ = interp2app(W_StringIO.descr_new.im_func),
- write=interp2app(W_StringIO.write_w),
- getvalue=interp2app(W_StringIO.getvalue_w),
- )
-
+from pypy.module._io.interp_io import W_TextIOBase
+from pypy.interpreter.typedef import TypeDef
+from pypy.interpreter.gateway import interp2app, unwrap_spec
+from pypy.interpreter.error import operationerrfmt
+from pypy.interpreter.baseobjspace import ObjSpace, W_Root
+
+class W_StringIO(W_TextIOBase):
+ def __init__(self):
+ self.buf = []
+ self.pos = 0
+
+ def _check_closed(self):
+ pass
+ def _check_initialized(self):
+ pass
+
+ @unwrap_spec(ObjSpace, W_Root)
+ def descr_new(space, w_subtype):
+ self = space.allocate_instance(W_StringIO, w_subtype)
+ W_StringIO.__init__(self)
+ return space.wrap(self)
+
+ def resize_buffer(self, newlength):
+ if len(self.buf) > newlength:
+ self.buf = self.buf[:newlength]
+ if len(self.buf) < newlength:
+ self.buf.extend([u'\0'] * (newlength - len(self.buf)))
+
+ def write(self, string):
+ # XXX self.decoder
+ decoded = string
+ # XXX writenl
+
+ length = len(decoded)
+ if self.pos + length > len(self.buf):
+ self.resize_buffer(self.pos + length)
+
+ for i in range(length):
+ self.buf[self.pos + i] = string[i]
+ self.pos += length
+
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def write_w(self, space, w_obj):
+ self._check_initialized()
+ if not space.isinstance_w(w_obj, space.w_unicode):
+ raise operationerrfmt(space.w_TypeError,
+ "string argument expected, got '%s'",
+ space.type(self).getname(space, '?'))
+ self._check_closed()
+ string = space.unicode_w(w_obj)
+ size = len(string)
+ if size:
+ self.write(string)
+ return space.wrap(size)
+
+ @unwrap_spec('self', ObjSpace)
+ def getvalue_w(self, space):
+ self._check_initialized()
+ self._check_closed()
+ return space.wrap(u''.join(self.buf))
+
+W_StringIO.typedef = TypeDef(
+ 'StringIO', W_TextIOBase.typedef,
+ __new__ = interp2app(W_StringIO.descr_new.im_func),
+ write=interp2app(W_StringIO.write_w),
+ getvalue=interp2app(W_StringIO.getvalue_w),
+ )
+
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_stringio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_stringio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_stringio.py Mon Oct 4 23:11:48 2010
@@ -1,7 +1,7 @@
-class AppTestStringIO:
- def test_stringio(self):
- import io
- sio = io.StringIO()
- sio.write(u'Hello ')
- sio.write(u'world')
- assert sio.getvalue() == u'Hello world'
+class AppTestStringIO:
+ def test_stringio(self):
+ import io
+ sio = io.StringIO()
+ sio.write(u'Hello ')
+ sio.write(u'world')
+ assert sio.getvalue() == u'Hello world'
Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py (original)
+++ pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py Mon Oct 4 23:11:48 2010
@@ -1,48 +1,48 @@
-import py
-import sys
-from pypy.conftest import gettestobjspace
-
-class TestConnection:
- def test_simple(self):
- from pypy.module._multiprocessing import interp_connection
-
-class AppTestConnection:
- def setup_class(cls):
- space = gettestobjspace(usemodules=('_multiprocessing', 'thread'))
- cls.space = space
- if sys.platform == "win32":
- # stubs for some modules,
- # just for multiprocessing to import correctly.
- w_modules = space.sys.get('modules')
- space.setitem(w_modules, space.wrap('msvcrt'), space.sys)
- space.setitem(w_modules, space.wrap('_subprocess'), space.sys)
-
- # import multiprocessing once
- space.appexec([], """(): import multiprocessing""")
-
- def test_winpipe_connection(self):
- import sys
- if sys.platform != "win32":
- skip("win32 only")
-
- import multiprocessing
- rhandle, whandle = multiprocessing.Pipe()
-
- obj = [1, 2.0, "hello"]
- whandle.send(obj)
- obj2 = rhandle.recv()
- assert obj == obj2
-
- def test_ospipe_connection(self):
- import _multiprocessing
- import os
- fd1, fd2 = os.pipe()
- rhandle = _multiprocessing.Connection(fd1, writable=False)
- whandle = _multiprocessing.Connection(fd2, readable=False)
-
- obj = [1, 2.0, "hello"]
- whandle.send(obj)
- obj2 = rhandle.recv()
- assert obj == obj2
-
-
+import py
+import sys
+from pypy.conftest import gettestobjspace
+
+class TestConnection:
+ def test_simple(self):
+ from pypy.module._multiprocessing import interp_connection
+
+class AppTestConnection:
+ def setup_class(cls):
+ space = gettestobjspace(usemodules=('_multiprocessing', 'thread'))
+ cls.space = space
+ if sys.platform == "win32":
+ # stubs for some modules,
+ # just for multiprocessing to import correctly.
+ w_modules = space.sys.get('modules')
+ space.setitem(w_modules, space.wrap('msvcrt'), space.sys)
+ space.setitem(w_modules, space.wrap('_subprocess'), space.sys)
+
+ # import multiprocessing once
+ space.appexec([], """(): import multiprocessing""")
+
+ def test_winpipe_connection(self):
+ import sys
+ if sys.platform != "win32":
+ skip("win32 only")
+
+ import multiprocessing
+ rhandle, whandle = multiprocessing.Pipe()
+
+ obj = [1, 2.0, "hello"]
+ whandle.send(obj)
+ obj2 = rhandle.recv()
+ assert obj == obj2
+
+ def test_ospipe_connection(self):
+ import _multiprocessing
+ import os
+ fd1, fd2 = os.pipe()
+ rhandle = _multiprocessing.Connection(fd1, writable=False)
+ whandle = _multiprocessing.Connection(fd2, readable=False)
+
+ obj = [1, 2.0, "hello"]
+ whandle.send(obj)
+ obj2 = rhandle.recv()
+ assert obj == obj2
+
+
From fijal at codespeak.net Mon Oct 4 23:32:32 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Mon, 4 Oct 2010 23:32:32 +0200 (CEST)
Subject: [pypy-svn] r77589 - pypy/trunk/pypy/jit/metainterp
Message-ID: <20101004213232.2F5A1282B90@codespeak.net>
Author: fijal
Date: Mon Oct 4 23:32:30 2010
New Revision: 77589
Modified:
pypy/trunk/pypy/jit/metainterp/simple_optimize.py
Log:
CALL_LOOPINVARIANT is just a CALL
Modified: pypy/trunk/pypy/jit/metainterp/simple_optimize.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/simple_optimize.py (original)
+++ pypy/trunk/pypy/jit/metainterp/simple_optimize.py Mon Oct 4 23:32:30 2010
@@ -9,9 +9,10 @@
def transform(op):
from pypy.jit.metainterp.history import AbstractDescr
- # Rename CALL_PURE to CALL.
+ # Rename CALL_PURE and CALL_INVARIANT to CALL.
# Simplify the VIRTUAL_REF_* so that they don't show up in the backend.
- if op.getopnum() == rop.CALL_PURE:
+ if (op.getopnum() == rop.CALL_PURE or
+ op.getopnum() == rop.CALL_LOOPINVARIANT):
op = ResOperation(rop.CALL, op.getarglist()[1:], op.result,
op.getdescr())
elif op.getopnum() == rop.VIRTUAL_REF:
From afa at codespeak.net Tue Oct 5 00:35:39 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 5 Oct 2010 00:35:39 +0200 (CEST)
Subject: [pypy-svn] r77590 - pypy/branch/fast-forward/pypy/module/_ssl
Message-ID: <20101004223539.958B5282B90@codespeak.net>
Author: afa
Date: Tue Oct 5 00:35:38 2010
New Revision: 77590
Modified:
pypy/branch/fast-forward/pypy/module/_ssl/__init__.py
pypy/branch/fast-forward/pypy/module/_ssl/interp_ssl.py
Log:
Probably a very incomplete fix,
but at least the tests in module/_ssl pass
Modified: pypy/branch/fast-forward/pypy/module/_ssl/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_ssl/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/_ssl/__init__.py Tue Oct 5 00:35:38 2010
@@ -2,7 +2,7 @@
class Module(MixedModule):
interpleveldefs = {
- 'ssl': 'interp_ssl.ssl',
+ 'sslwrap': 'interp_ssl.sslwrap',
}
appleveldefs = {
Modified: pypy/branch/fast-forward/pypy/module/_ssl/interp_ssl.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_ssl/interp_ssl.py (original)
+++ pypy/branch/fast-forward/pypy/module/_ssl/interp_ssl.py Tue Oct 5 00:35:38 2010
@@ -359,6 +359,10 @@
return self.space.wrap(result)
read.unwrap_spec = ['self', int]
+ def do_handshake(self):
+ # XXX
+ pass
+
SSLObject.typedef = TypeDef("SSLObject",
server = interp2app(SSLObject.server,
@@ -367,11 +371,12 @@
unwrap_spec=SSLObject.issuer.unwrap_spec),
write = interp2app(SSLObject.write,
unwrap_spec=SSLObject.write.unwrap_spec),
- read = interp2app(SSLObject.read, unwrap_spec=SSLObject.read.unwrap_spec)
+ read = interp2app(SSLObject.read, unwrap_spec=SSLObject.read.unwrap_spec),
+ do_handshake=interp2app(SSLObject.do_handshake, unwrap_spec=['self']),
)
-def new_sslobject(space, w_sock, w_key_file, w_cert_file):
+def new_sslobject(space, w_sock, side, w_key_file, w_cert_file):
ss = SSLObject(space)
sock_fd = space.int_w(space.call_method(w_sock, "fileno"))
@@ -562,8 +567,12 @@
return errstr, errval
-def ssl(space, w_socket, w_key_file=None, w_cert_file=None):
- """ssl(socket, [keyfile, certfile]) -> sslobject"""
- return space.wrap(new_sslobject(space, w_socket, w_key_file, w_cert_file))
-ssl.unwrap_spec = [ObjSpace, W_Root, W_Root, W_Root]
+def sslwrap(space, w_socket, side, w_key_file=None, w_cert_file=None,
+ cert_mode=PY_SSL_CERT_NONE, protocol=PY_SSL_VERSION_SSL23,
+ w_cacerts_file=None, w_cipher=None):
+ """sslwrap(socket, side, [keyfile, certfile]) -> sslobject"""
+ return space.wrap(new_sslobject(
+ space, w_socket, side, w_key_file, w_cert_file))
+sslwrap.unwrap_spec = [ObjSpace, W_Root, int, W_Root, W_Root,
+ int, int, W_Root, W_Root]
From afa at codespeak.net Tue Oct 5 01:34:44 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 5 Oct 2010 01:34:44 +0200 (CEST)
Subject: [pypy-svn] r77591 - pypy/branch/fast-forward/lib-python
Message-ID: <20101004233444.70EEF282B90@codespeak.net>
Author: afa
Date: Tue Oct 5 01:34:42 2010
New Revision: 77591
Modified:
pypy/branch/fast-forward/lib-python/TODO
Log:
One more task, in the "easy" level.
Modified: pypy/branch/fast-forward/lib-python/TODO
==============================================================================
--- pypy/branch/fast-forward/lib-python/TODO (original)
+++ pypy/branch/fast-forward/lib-python/TODO Tue Oct 5 01:34:42 2010
@@ -30,6 +30,8 @@
- Fix fcntl.fcntl(fd, fcntl.F_NOTIFY, fcntl.DN_MULTISHOT) on 32bit platform.
+- Missing module future_builtins
+
Medium tasks
------------
From afa at codespeak.net Tue Oct 5 08:58:01 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 5 Oct 2010 08:58:01 +0200 (CEST)
Subject: [pypy-svn] r77592 -
pypy/branch/fast-forward/lib-python/modified-2.7.0
Message-ID: <20101005065801.80F3A282B90@codespeak.net>
Author: afa
Date: Tue Oct 5 08:57:59 2010
New Revision: 77592
Modified:
pypy/branch/fast-forward/lib-python/modified-2.7.0/pickle.py
Log:
PyPy can save references to module dictionaries.
Same code as in modified-2.5.2, except for a hack
needed by previous versions of the py library.
This fixes a test in pypy.interpreter.test.test_zzpickle_and_slow
Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/pickle.py
==============================================================================
--- pypy/branch/fast-forward/lib-python/modified-2.7.0/pickle.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/pickle.py Tue Oct 5 08:57:59 2010
@@ -638,6 +638,10 @@
# else tmp is empty, and we're done
def save_dict(self, obj):
+ modict_saver = self._pickle_moduledict(obj)
+ if modict_saver is not None:
+ return self.save_reduce(*modict_saver)
+
write = self.write
if self.bin:
@@ -687,6 +691,29 @@
write(SETITEM)
# else tmp is empty, and we're done
+ def _pickle_moduledict(self, obj):
+ # save module dictionary as "getattr(module, '__dict__')"
+
+ # build index of module dictionaries
+ try:
+ modict = self.module_dict_ids
+ except AttributeError:
+ modict = {}
+ from sys import modules
+ for mod in modules.values():
+ if isinstance(mod, ModuleType):
+ modict[id(mod.__dict__)] = mod
+ self.module_dict_ids = modict
+
+ thisid = id(obj)
+ try:
+ themodule = modict[thisid]
+ except KeyError:
+ return None
+ from __builtin__ import getattr
+ return getattr, (themodule, '__dict__')
+
+
def save_inst(self, obj):
cls = obj.__class__
From antocuni at codespeak.net Tue Oct 5 10:02:46 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Tue, 5 Oct 2010 10:02:46 +0200 (CEST)
Subject: [pypy-svn] r77593 - in pypy/branch/jitffi/pypy: jit/metainterp/test
rlib rlib/test
Message-ID: <20101005080246.2338A36C228@codespeak.net>
Author: antocuni
Date: Tue Oct 5 10:02:44 2010
New Revision: 77593
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py
pypy/branch/jitffi/pypy/rlib/libffi.py
pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
Log:
unify the float() and int() methods of ArgChain into arg(), which is specialized by the type of the argument
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py Tue Oct 5 10:02:44 2010
@@ -47,7 +47,7 @@
driver.can_enter_jit(n=n, func=func)
func = hint(func, promote=True)
argchain = ArgChain()
- argchain.int(n).float(1.2)
+ argchain.arg(n).arg(1.2)
n = func.call(argchain, rffi.LONG)
return n
@@ -73,7 +73,7 @@
driver.can_enter_jit(n=n, func=func, res=res)
func = hint(func, promote=True)
argchain = ArgChain()
- argchain.float(float(-n))
+ argchain.arg(float(-n))
res = func.call(argchain, rffi.DOUBLE)
n += 1
return res
@@ -95,7 +95,7 @@
driver.can_enter_jit(n=n, func=func, res=res)
func = hint(func, promote=True)
argchain = ArgChain()
- argchain.int(0)
+ argchain.arg(0)
res = func.call(argchain, rffi.UCHAR)
n += 1
return res
Modified: pypy/branch/jitffi/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/libffi.py Tue Oct 5 10:02:44 2010
@@ -48,12 +48,16 @@
last = None
numargs = 0
- def int(self, intval):
- self._append(IntArg(intval))
- return self
-
- def float(self, floatval):
- self._append(FloatArg(floatval))
+ @specialize.argtype(1)
+ def arg(self, val):
+ TYPE = lltype.typeOf(val)
+ if TYPE is rffi.LONG:
+ cls = IntArg
+ elif TYPE is rffi.DOUBLE:
+ cls = FloatArg
+ else:
+ raise TypeError, 'Unsupported argument type: %s' % TYPE
+ self._append(cls(val))
return self
def _append(self, arg):
Modified: pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_libffi.py Tue Oct 5 10:02:44 2010
@@ -20,13 +20,13 @@
def test_argchain(self):
chain = ArgChain()
assert chain.numargs == 0
- chain2 = chain.int(42)
+ chain2 = chain.arg(42)
assert chain2 is chain
assert chain.numargs == 1
intarg = chain.first
assert chain.last is intarg
assert intarg.intval == 42
- chain.float(123.45)
+ chain.arg(123.45)
assert chain.numargs == 2
assert chain.first is intarg
assert intarg.next is chain.last
@@ -51,6 +51,6 @@
pow = libm.getpointer('pow', [types.double, types.double],
types.double)
argchain = ArgChain()
- argchain.float(2.0).float(3.0)
+ argchain.arg(2.0).arg(3.0)
res = pow.call(argchain, rffi.DOUBLE)
assert res == 8.0
From antocuni at codespeak.net Tue Oct 5 11:21:41 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Tue, 5 Oct 2010 11:21:41 +0200 (CEST)
Subject: [pypy-svn] r77594 - in pypy/branch/jitffi/pypy: jit/metainterp/test
rlib/test
Message-ID: <20101005092141.08E47282B90@codespeak.net>
Author: antocuni
Date: Tue Oct 5 11:21:39 2010
New Revision: 77594
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py
pypy/branch/jitffi/pypy/rlib/test/test_clibffi.py
pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
Log:
refactor the test to share most of the between test_libffi and test_fficall.
The reason is that ffi calls now really follow two different paths when jitted
or non jitted, so we need to always test both.
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py Tue Oct 5 11:21:39 2010
@@ -1,104 +1,43 @@
import py
from pypy.rlib.jit import JitDriver, hint
-from pypy.jit.metainterp.test.test_basic import LLJitMixin
-from pypy.rlib.libffi import CDLL, types, ArgChain, Func
-from pypy.tool.udir import udir
-from pypy.translator.tool.cbuild import ExternalCompilationInfo
-from pypy.translator.platform import platform
+from pypy.rlib.unroll import unrolling_iterable
+from pypy.rlib.libffi import ArgChain
+from pypy.rlib.test.test_libffi import TestLibffiCall as _TestLibffiCall
from pypy.rpython.lltypesystem import lltype, rffi
-
-class TestFfiCall(LLJitMixin):
- def setup_class(cls):
- # prepare C code as an example, so we can load it and call
- # it via rlib.libffi
- c_file = udir.ensure("test_jit_fficall", dir=1).join("xlib.c")
- c_file.write(py.code.Source('''
- int sum_xy(int x, double y)
- {
- return (x + (int)y);
- }
-
- float abs(double x)
- {
- if (x<0)
- return -x;
- return x;
- }
-
- unsigned char cast_to_uchar(int x)
- {
- return 200+(unsigned char)x;
- }
- '''))
- eci = ExternalCompilationInfo(export_symbols=[])
- cls.lib_name = str(platform.compile([c_file], eci, 'x',
- standalone=False))
-
- def test_simple(self):
- driver = JitDriver(reds = ['n', 'func'], greens = [])
-
- def f(n):
- cdll = CDLL(self.lib_name)
- func = cdll.getpointer('sum_xy', [types.sint, types.double],
- types.sint)
- while n < 10:
- driver.jit_merge_point(n=n, func=func)
- driver.can_enter_jit(n=n, func=func)
- func = hint(func, promote=True)
- argchain = ArgChain()
- argchain.arg(n).arg(1.2)
- n = func.call(argchain, rffi.LONG)
- return n
-
- res = self.meta_interp(f, [0])
- assert res == 10
- self.check_loops({
- 'call': 1,
- 'guard_no_exception': 1,
- 'int_lt': 1,
- 'guard_true': 1,
- 'jump': 1})
+from pypy.jit.metainterp.test.test_basic import LLJitMixin
- def test_float_result(self):
- driver = JitDriver(reds = ['n', 'func', 'res'], greens = [])
+class TestFfiCall(LLJitMixin, _TestLibffiCall):
+ def call(self, funcspec, args, RESULT, init_result=0):
+ """
+ Call the function specified by funcspec in a loop, and let the jit to
+ see and optimize it.
+ """
+ #
+ lib, name, argtypes, restype = funcspec
+ args = unrolling_iterable(args)
+ #
+ reds = ['n', 'res', 'func']
+ if type(init_result) is float:
+ reds = ['n', 'func', 'res'] # floats must be *after* refs
+ driver = JitDriver(reds=reds, greens=[])
+ #
def f(n):
- cdll = CDLL(self.lib_name)
- func = cdll.getpointer('abs', [types.double], types.double)
- res = 0.0
+ func = lib.getpointer(name, argtypes, restype)
+ res = init_result
while n < 10:
- driver.jit_merge_point(n=n, func=func, res=res)
- driver.can_enter_jit(n=n, func=func, res=res)
+ driver.jit_merge_point(n=n, res=res, func=func)
+ driver.can_enter_jit(n=n, res=res, func=func)
func = hint(func, promote=True)
argchain = ArgChain()
- argchain.arg(float(-n))
- res = func.call(argchain, rffi.DOUBLE)
+ for argval in args: # this loop is unrolled
+ argchain.arg(argval)
+ res = func.call(argchain, RESULT)
n += 1
return res
-
+ #
res = self.meta_interp(f, [0])
- assert res == 9
- self.check_loops(call=1)
-
- def test_cast_result(self):
- driver = JitDriver(reds = ['n', 'res', 'func'], greens = [])
+ return res
- def f(n):
- cdll = CDLL(self.lib_name)
- func = cdll.getpointer('cast_to_uchar', [types.sint],
- types.uchar)
- res = 0
- while n < 10:
- driver.jit_merge_point(n=n, func=func, res=res)
- driver.can_enter_jit(n=n, func=func, res=res)
- func = hint(func, promote=True)
- argchain = ArgChain()
- argchain.arg(0)
- res = func.call(argchain, rffi.UCHAR)
- n += 1
- return res
-
- res = self.meta_interp(f, [0])
- assert res == 200
Modified: pypy/branch/jitffi/pypy/rlib/test/test_clibffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_clibffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_clibffi.py Tue Oct 5 11:21:39 2010
@@ -20,7 +20,10 @@
return 'libm.so'
class BaseFfiTest(object):
-
+
+ CDLL = None # overridden by subclasses
+
+ @classmethod
def setup_class(cls):
for name in type_names:
# XXX force this to be seen by ll2ctypes
@@ -31,14 +34,17 @@
def setup_method(self, meth):
ALLOCATED.clear()
-
-class TestCLibffi(BaseFfiTest):
-
def get_libc(self):
- return CDLL(get_libc_name())
+ return self.CDLL(get_libc_name())
def get_libm(self):
- return CDLL(get_libm_name(sys.platform))
+ return self.CDLL(get_libm_name(sys.platform))
+
+
+
+class TestCLibffi(BaseFfiTest):
+
+ CDLL = CDLL
def test_library_open(self):
lib = self.get_libc()
Modified: pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_libffi.py Tue Oct 5 11:21:39 2010
@@ -5,17 +5,9 @@
from pypy.rlib.test.test_clibffi import BaseFfiTest, get_libm_name
from pypy.rlib.libffi import CDLL, Func, get_libc_name, ArgChain, types
+class TestLibffiMisc(BaseFfiTest):
-class TestLibffi(BaseFfiTest):
- """
- Test the new JIT-friendly interface to libffi
- """
-
- def get_libc(self):
- return CDLL(get_libc_name())
-
- def get_libm(self):
- return CDLL(get_libm_name(sys.platform))
+ CDLL = CDLL
def test_argchain(self):
chain = ArgChain()
@@ -46,11 +38,100 @@
del lib
assert not ALLOCATED
- def test_call_argchain(self):
+
+class TestLibffiCall(BaseFfiTest):
+ """
+ Test various kind of calls through libffi.
+
+ The peculiarity of these tests is that they are run both directly (going
+ really through libffi) and by jit/metainterp/test/test_fficall.py, which
+ tests the call when JITted.
+
+ If you need to test a behaviour than it's not affected by JITing (e.g.,
+ typechecking), you should put your test in TestLibffiMisc.
+ """
+
+ CDLL = CDLL
+
+ @classmethod
+ def setup_class(cls):
+ from pypy.tool.udir import udir
+ from pypy.translator.tool.cbuild import ExternalCompilationInfo
+ from pypy.translator.platform import platform
+
+ BaseFfiTest.setup_class()
+ # prepare C code as an example, so we can load it and call
+ # it via rlib.libffi
+ c_file = udir.ensure("test_libffi", dir=1).join("foolib.c")
+ c_file.write(py.code.Source('''
+ int sum_xy(int x, double y)
+ {
+ return (x + (int)y);
+ }
+
+ unsigned char cast_to_uchar_and_ovf(int x)
+ {
+ return 200+(unsigned char)x;
+ }
+ '''))
+ eci = ExternalCompilationInfo(export_symbols=[])
+ cls.libfoo_name = str(platform.compile([c_file], eci, 'x',
+ standalone=False))
+
+ def get_libfoo(self):
+ return self.CDLL(self.libfoo_name)
+
+ def call(self, funcspec, args, RESULT, init_result=0):
+ """
+ Call the specified function after constructing and ArgChain with the
+ arguments in ``args``.
+
+ The function is specified with ``funcspec``, which is a tuple of the
+ form (lib, name, argtypes, restype).
+
+ This method is overridden by metainterp/test/test_fficall.py in
+ order to do the call in a loop and JIT it. The optional arguments are
+ used only by that overridden method.
+
+ """
+ lib, name, argtypes, restype = funcspec
+ func = lib.getpointer(name, argtypes, restype)
+ chain = ArgChain()
+ for arg in args:
+ chain.arg(arg)
+ return func.call(chain, RESULT)
+
+ def check_loops(self, *args, **kwds):
+ """
+ Ignored here, but does something in the JIT tests
+ """
+ pass
+
+ # ------------------------------------------------------------------------
+
+ def test_simple(self):
+ libfoo = self.get_libfoo()
+ func = (libfoo, 'sum_xy', [types.sint, types.double], types.sint)
+ res = self.call(func, [38, 4.2], rffi.LONG)
+ assert res == 42
+ self.check_loops({
+ 'call': 1,
+ 'guard_no_exception': 1,
+ 'int_add': 1,
+ 'int_lt': 1,
+ 'guard_true': 1,
+ 'jump': 1})
+
+ def test_float_result(self):
libm = self.get_libm()
- pow = libm.getpointer('pow', [types.double, types.double],
- types.double)
- argchain = ArgChain()
- argchain.arg(2.0).arg(3.0)
- res = pow.call(argchain, rffi.DOUBLE)
+ func = (libm, 'pow', [types.double, types.double], types.double)
+ res = self.call(func, [2.0, 3.0], rffi.DOUBLE, init_result=0.0)
assert res == 8.0
+ self.check_loops(call=1)
+
+ def test_cast_result(self):
+ libfoo = self.get_libfoo()
+ func = (libfoo, 'cast_to_uchar_and_ovf', [types.sint], types.uchar)
+ res = self.call(func, [0], rffi.UCHAR)
+ assert res == 200
+ self.check_loops(call=1)
From fijal at codespeak.net Tue Oct 5 11:47:37 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Tue, 5 Oct 2010 11:47:37 +0200 (CEST)
Subject: [pypy-svn] r77595 - pypy/trunk/pypy/jit/metainterp/optimizeopt
Message-ID: <20101005094737.1290F36C228@codespeak.net>
Author: fijal
Date: Tue Oct 5 11:47:36 2010
New Revision: 77595
Modified:
pypy/trunk/pypy/jit/metainterp/optimizeopt/rewrite.py
Log:
Write down a comment that this code is dead
Modified: pypy/trunk/pypy/jit/metainterp/optimizeopt/rewrite.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/optimizeopt/rewrite.py (original)
+++ pypy/trunk/pypy/jit/metainterp/optimizeopt/rewrite.py Tue Oct 5 11:47:36 2010
@@ -245,6 +245,9 @@
def optimize_CALL_LOOPINVARIANT(self, op):
funcvalue = self.getvalue(op.getarg(0))
if not funcvalue.is_constant():
+ # XXX this code path is never executed in tests nor in production.
+ # in fact, it can't even happen since residual_call in codewriter
+ # expects a compile-time constant
self.emit_operation(op)
return
key = make_hashable_int(op.getarg(0).getint())
From fijal at codespeak.net Tue Oct 5 11:47:56 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Tue, 5 Oct 2010 11:47:56 +0200 (CEST)
Subject: [pypy-svn] r77596 - pypy/trunk/pypy/jit/metainterp
Message-ID: <20101005094756.7D19B36C228@codespeak.net>
Author: fijal
Date: Tue Oct 5 11:47:55 2010
New Revision: 77596
Modified:
pypy/trunk/pypy/jit/metainterp/simple_optimize.py
Log:
CALL_LOOPINVARIANT is a normal call (unlike CALL_PURE)
Modified: pypy/trunk/pypy/jit/metainterp/simple_optimize.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/simple_optimize.py (original)
+++ pypy/trunk/pypy/jit/metainterp/simple_optimize.py Tue Oct 5 11:47:55 2010
@@ -9,12 +9,13 @@
def transform(op):
from pypy.jit.metainterp.history import AbstractDescr
- # Rename CALL_PURE and CALL_INVARIANT to CALL.
+ # Rename CALL_PURE and CALL_LOOPINVARIANT to CALL.
# Simplify the VIRTUAL_REF_* so that they don't show up in the backend.
- if (op.getopnum() == rop.CALL_PURE or
- op.getopnum() == rop.CALL_LOOPINVARIANT):
+ if op.getopnum() == rop.CALL_PURE:
op = ResOperation(rop.CALL, op.getarglist()[1:], op.result,
op.getdescr())
+ elif op.getopnum() == rop.CALL_LOOPINVARIANT:
+ op = op.copy_and_change(rop.CALL)
elif op.getopnum() == rop.VIRTUAL_REF:
op = ResOperation(rop.SAME_AS, [op.getarg(0)], op.result)
elif op.getopnum() == rop.VIRTUAL_REF_FINISH:
From arigo at codespeak.net Tue Oct 5 13:09:42 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 13:09:42 +0200 (CEST)
Subject: [pypy-svn] r77597 - in pypy/branch/32ptr-on-64bit/pypy: config rlib
rpython/lltypesystem rpython/memory rpython/memory/gc
rpython/memory/gc/test rpython/memory/gctransform translator/c
Message-ID: <20101005110942.5451B282B90@codespeak.net>
Author: arigo
Date: Tue Oct 5 13:09:40 2010
New Revision: 77597
Added:
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimarkpage2.py
- copied, changed from r77571, pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimarkpage.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/test/test_minimarkpage2.py
- copied, changed from r77571, pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/test/test_minimarkpage.py
Modified:
pypy/branch/32ptr-on-64bit/pypy/config/translationoption.py
pypy/branch/32ptr-on-64bit/pypy/rlib/rmmap.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/ll2ctypes.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimarkpage.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gcwrapper.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py
Log:
Random progress.
Modified: pypy/branch/32ptr-on-64bit/pypy/config/translationoption.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/config/translationoption.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/config/translationoption.py Tue Oct 5 13:09:40 2010
@@ -196,7 +196,8 @@
BoolOption("compressptr", "Compress pointers; limits the program to 32GB",
default=False, cmdline="--compressptr",
requires=[("translation.type_system", "lltype"),
- ("translation.taggedpointers", False)]
+ ("translation.taggedpointers", False),
+ ("translation.gc", "minimark")] # for now
+ [("compressptr (64-bit only)", True)]*(not IS_64_BITS)),
# options for ootype
Modified: pypy/branch/32ptr-on-64bit/pypy/rlib/rmmap.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rlib/rmmap.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rlib/rmmap.py Tue Oct 5 13:09:40 2010
@@ -50,7 +50,7 @@
constant_names = ['MAP_SHARED', 'MAP_PRIVATE',
'PROT_READ', 'PROT_WRITE',
'MS_SYNC']
- opt_constant_names = ['MAP_ANON', 'MAP_ANONYMOUS',
+ opt_constant_names = ['MAP_ANON', 'MAP_ANONYMOUS', 'MAP_FIXED',
'PROT_EXEC',
'MAP_DENYWRITE', 'MAP_EXECUTABLE']
for name in constant_names:
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/ll2ctypes.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/ll2ctypes.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/ll2ctypes.py Tue Oct 5 13:09:40 2010
@@ -1154,16 +1154,14 @@
return hop.genop('direct_ptradd', [v_ptr, v_n],
resulttype = v_ptr.concretetype)
-class _lladdress(long):
+class _lladdress(object):
_TYPE = llmemory.Address
- def __new__(cls, void_p):
+ def __init__(self, void_p):
if isinstance(void_p, (int, long)):
void_p = ctypes.c_void_p(void_p)
- self = long.__new__(cls, void_p.value)
self.void_p = void_p
self.intval = intmask(void_p.value)
- return self
def _cast_to_ptr(self, TP):
return force_cast(TP, self.intval)
@@ -1179,6 +1177,12 @@
def __ne__(self, other):
return not self == other
+ def __add__(self, other):
+ return _lladdress(self.intval + other)
+
+ def __sub__(self, other):
+ return _lladdress(self.intval - other)
+
class _llgcopaque(lltype._container):
_TYPE = llmemory.GCREF.TO
_name = "_llgcopaque"
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py Tue Oct 5 13:09:40 2010
@@ -7,6 +7,13 @@
def get_compressed_gcref_repr(rtyper, baserepr):
+ # Return either the original baserepr, or another repr standing for
+ # a HiddenGcRef32. The idea is that we only get a HiddenGcRef32 for
+ # fixed-sized structures (XXX that are not too big); thus this is only
+ # for structures that gets allocated by the minimarkpage2 mmap()-
+ # within-32GB-of-RAM.
+ if baserepr.lowleveltype.TO._is_varsize():
+ return baserepr
try:
comprmgr = rtyper.compressed_gcref_manager
except AttributeError:
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py Tue Oct 5 13:09:40 2010
@@ -50,7 +50,7 @@
# collection. It is automatically set to True by test_gc.py. The
# checking logic is translatable, so the flag can be set to True
# here before translation.
- DEBUG = False
+ DEBUG = True
def set_query_functions(self, is_varsize, has_gcptr_in_varsize,
is_gcarrayofgcptr,
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py Tue Oct 5 13:09:40 2010
@@ -3,7 +3,7 @@
from pypy.rpython.lltypesystem.lloperation import llop
from pypy.rpython.lltypesystem.llmemory import raw_malloc_usage
from pypy.rpython.memory.gc.base import GCBase, MovingGCBase
-from pypy.rpython.memory.gc import minimarkpage, base, generation
+from pypy.rpython.memory.gc import base, generation
from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask, r_uint
from pypy.rlib.rarithmetic import LONG_BIT_SHIFT
from pypy.rlib.debug import ll_assert, debug_print, debug_start, debug_stop
@@ -102,11 +102,12 @@
# The system page size. Like obmalloc.c, we assume that it is 4K
# for 32-bit systems; unlike obmalloc.c, we assume that it is 8K
- # for 64-bit systems, for consistent results.
+ # for 64-bit systems, for consistent results. (ignored if we
+ # use minimarkpage2.py)
"page_size": 1024*WORD,
# The size of an arena. Arenas are groups of pages allocated
- # together.
+ # together. (ignored if we use minimarkpage2.py)
"arena_size": 65536*WORD,
# The maximum size of an object allocated compactly. All objects
@@ -181,7 +182,14 @@
#
# The ArenaCollection() handles the nonmovable objects allocation.
if ArenaCollectionClass is None:
- ArenaCollectionClass = minimarkpage.ArenaCollection
+ if self.translated_to_c and self.config.compressptr:
+ from pypy.rpython.memory.gc import minimarkpage2
+ ArenaCollectionClass = minimarkpage2.ArenaCollection2
+ arena_size = minimarkpage2.ARENA_SIZE
+ page_size = 4096
+ else:
+ from pypy.rpython.memory.gc import minimarkpage
+ ArenaCollectionClass = minimarkpage.ArenaCollection
self.ac = ArenaCollectionClass(arena_size, page_size,
small_request_threshold)
#
@@ -267,7 +275,7 @@
self.max_heap_size = float(max_heap_size)
#
self.minor_collection() # to empty the nursery
- llarena.arena_free(self.nursery)
+ self.ac.free_big_chunk(self.nursery)
self.nursery_size = newsize
self.allocate_nursery()
@@ -280,9 +288,11 @@
# in malloc_fixedsize_clear(). The few extra pages are never used
# anyway so it doesn't even count.
extra = self.nonlarge_gcptrs_max + 1
- self.nursery = llarena.arena_malloc(self.nursery_size + extra, 2)
+ fullsize = self.nursery_size + extra
+ self.nursery = self.ac.allocate_big_chunk(fullsize)
if not self.nursery:
raise MemoryError("cannot allocate nursery")
+ llarena.arena_reset(self.nursery, fullsize, 2)
# the current position in the nursery:
self.nursery_free = self.nursery
# the end of the nursery:
@@ -1526,6 +1536,12 @@
self.all_objects = []
self.total_memory_used = 0
+ def allocate_big_chunk(self, arena_size):
+ return llarena.arena_malloc(arena_size, False)
+
+ def free_big_chunk(self, arena):
+ llarena.arena_free(arena)
+
def malloc(self, size):
nsize = raw_malloc_usage(size)
ll_assert(nsize > 0, "malloc: size is null or negative")
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimarkpage.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimarkpage.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimarkpage.py Tue Oct 5 13:09:40 2010
@@ -138,6 +138,13 @@
self.total_memory_used = r_uint(0)
+ def allocate_big_chunk(self, arena_size):
+ return llarena.arena_malloc(arena_size, False)
+
+ def free_big_chunk(self, arena):
+ llarena.arena_free(arena)
+
+
def malloc(self, size):
"""Allocate a block from a page in an arena."""
nsize = llmemory.raw_malloc_usage(size)
Copied: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimarkpage2.py (from r77571, pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimarkpage.py)
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimarkpage.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimarkpage2.py Tue Oct 5 13:09:40 2010
@@ -2,6 +2,7 @@
from pypy.rlib.rarithmetic import LONG_BIT, r_uint
from pypy.rlib.objectmodel import we_are_translated
from pypy.rlib.debug import ll_assert
+from pypy.rlib import rmmap
WORD = LONG_BIT // 8
NULL = llmemory.NULL
@@ -13,32 +14,12 @@
# A page contains a number of allocated objects, called "blocks".
# The actual allocation occurs in whole arenas, which are then subdivided
-# into pages. For each arena we allocate one of the following structures:
+# into pages. Arenas are allocated (after translation to C) as an mmap()
+# at fixed addresses:
-ARENA_PTR = lltype.Ptr(lltype.ForwardReference())
-ARENA = lltype.Struct('ArenaReference',
- # -- The address of the arena, as returned by malloc()
- ('base', llmemory.Address),
- # -- The number of free and the total number of pages in the arena
- ('nfreepages', lltype.Signed),
- ('totalpages', lltype.Signed),
- # -- A chained list of free pages in the arena. Ends with NULL.
- ('freepages', llmemory.Address),
- # -- A linked list of arenas. See below.
- ('nextarena', ARENA_PTR),
- )
-ARENA_PTR.TO.become(ARENA)
-ARENA_NULL = lltype.nullptr(ARENA)
-
-# The idea is that when we need a free page, we take it from the arena
-# which currently has the *lowest* number of free pages. This allows
-# arenas with a lot of free pages to eventually become entirely free, at
-# which point they are returned to the OS. If an arena has a total of
-# 64 pages, then we have 64 global lists, arenas_lists[0] to
-# arenas_lists[63], such that arenas_lists[i] contains exactly those
-# arenas that have 'nfreepages == i'. We allocate pages out of the
-# arena in 'current_arena'; when it is exhausted we pick another arena
-# with the smallest value for nfreepages (but > 0).
+ARENA_SIZE = 0x100000 # 1MB
+ARENA_ADDR_START = 0x10000000 # 256MB (too low a number, segfault on linux)
+ARENA_ADDR_STOP = 0x800000000 # 32GB
# ____________________________________________________________
#
@@ -63,28 +44,30 @@
# pages, it is a chained list of pages having the same size class,
# rooted in 'page_for_size[size_class]'. For full pages, it is a
# different chained list rooted in 'full_page_for_size[size_class]'.
- # For free pages, it is the list 'freepages' in the arena header.
+ # For free pages, it is the list 'freepages'.
('nextpage', PAGE_PTR),
- # -- The arena this page is part of.
- ('arena', ARENA_PTR),
# -- The number of free blocks. The numbers of uninitialized and
# allocated blocks can be deduced from the context if needed.
- ('nfree', lltype.Signed),
- # -- The chained list of free blocks. It ends as a pointer to the
+ ('nfree', rffi.INT),
+ # -- The chained list of free blocks. It ends as a reference to the
# first uninitialized block (pointing to data that is uninitialized,
- # or to the end of the page).
- ('freeblock', llmemory.Address),
- # -- The structure above is 4 words, which is a good value:
- # '(1024-4) % N' is zero or very small for various small N's,
+ # or to the end of the page). Each entry in the free list is encoded
+ # as an offset to the start of the page.
+ ('freeblock', rffi.INT),
+ # -- The structure above is 2 words, which is a good value:
+ # '(512-2) % N' is zero or very small for various small N's,
# i.e. there is not much wasted space.
)
PAGE_PTR.TO.become(PAGE_HEADER)
PAGE_NULL = lltype.nullptr(PAGE_HEADER)
+FREEBLOCK = lltype.Struct('FreeBlock', ('freeblock', rffi.INT))
+FREEBLOCK_PTR = lltype.Ptr(FREEBLOCK)
+
# ----------
-class ArenaCollection(object):
+class ArenaCollection2(object):
_alloc_flavor_ = "raw"
def __init__(self, arena_size, page_size, small_request_threshold):
@@ -111,31 +94,19 @@
for i in range(1, length):
self.nblocks_for_size[i] = (page_size - self.hdrsize) // (WORD * i)
#
- self.max_pages_per_arena = arena_size // page_size
- self.arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
- self.max_pages_per_arena,
- flavor='raw', zero=True)
- # this is used in mass_free() only
- self.old_arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
- self.max_pages_per_arena,
- flavor='raw', zero=True)
- #
- # the arena currently consumed; it must have at least one page
- # available, or be NULL. The arena object that we point to is
- # not in any 'arenas_lists'. We will consume all its pages before
- # we choose a next arena, even if there is a major collection
- # in-between.
- self.current_arena = ARENA_NULL
+ # The next address to get an arena from
+ self.next_arena_addr = ARENA_ADDR_START
#
- # guarantee that 'arenas_lists[1:min_empty_nfreepages]' are all empty
- self.min_empty_nfreepages = self.max_pages_per_arena
- #
- # part of current_arena might still contain uninitialized pages
+ # Uninitialized pages from the current arena
+ self.next_uninitialized_page = NULL
self.num_uninitialized_pages = 0
#
# the total memory used, counting every block in use, without
# the additional bookkeeping stuff.
self.total_memory_used = r_uint(0)
+ #
+ # Chained list of pages that used to contain stuff but are now free.
+ self.freepages = NULL
def malloc(self, size):
@@ -153,24 +124,28 @@
page = self.allocate_new_page(size_class)
#
# The result is simply 'page.freeblock'
- result = page.freeblock
- if page.nfree > 0:
+ pageaddr = llarena.getfakearenaaddress(llmemory.cast_ptr_to_adr(page))
+ resultofs = rffi.getintfield(page, 'freeblock')
+ result = pageaddr + resultofs
+ page_nfree = rffi.getintfield(page, 'nfree')
+ if page_nfree > 0:
#
# The 'result' was part of the chained list; read the next.
- page.nfree -= 1
- freeblock = result.address[0]
+ page_nfree -= 1
+ rffi.setintfield(page, 'nfree', page_nfree)
+ freeblockptr = llmemory.cast_adr_to_ptr(result, FREEBLOCK_PTR)
+ freeblock = rffi.getintfield(freeblockptr, 'freeblock')
llarena.arena_reset(result,
- llmemory.sizeof(llmemory.Address),
+ llmemory.sizeof(FREEBLOCK),
0)
#
else:
# The 'result' is part of the uninitialized blocks.
- freeblock = result + nsize
+ freeblock = resultofs + nsize
#
- page.freeblock = freeblock
+ rffi.setintfield(page, 'freeblock', freeblock)
#
- pageaddr = llarena.getfakearenaaddress(llmemory.cast_ptr_to_adr(page))
- if freeblock - pageaddr > self.page_size - nsize:
+ if freeblock > self.page_size - nsize:
# This was the last free block, so unlink the page from the
# chained list and put it in the 'full_page_for_size' list.
self.page_for_size[size_class] = page.nextpage
@@ -184,48 +159,38 @@
def allocate_new_page(self, size_class):
"""Allocate and return a new page for the given size_class."""
#
- # Allocate a new arena if needed.
- if self.current_arena == ARENA_NULL:
- self.allocate_new_arena()
- #
- # The result is simply 'current_arena.freepages'.
- arena = self.current_arena
- result = arena.freepages
- if arena.nfreepages > 0:
- #
- # The 'result' was part of the chained list; read the next.
- arena.nfreepages -= 1
- freepages = result.address[0]
+ # If available, return the next page in self.freepages
+ if self.freepages != NULL:
+ result = self.freepages
+ self.freepages = result.address[0]
llarena.arena_reset(result,
llmemory.sizeof(llmemory.Address),
0)
#
else:
- # The 'result' is part of the uninitialized pages.
+ #
+ # No more free page. Allocate a new arena if needed.
+ if self.next_uninitialized_page == NULL:
+ self.allocate_new_arena()
+ #
+ # The result is simply 'self.next_uninitialized_page'.
+ result = self.next_uninitialized_page
+ #
ll_assert(self.num_uninitialized_pages > 0,
- "fully allocated arena found in self.current_arena")
+ "fully allocated arena found in next_uninitialized_page")
self.num_uninitialized_pages -= 1
if self.num_uninitialized_pages > 0:
freepages = result + self.page_size
else:
freepages = NULL
- #
- arena.freepages = freepages
- if freepages == NULL:
- # This was the last page, so put the arena away into
- # arenas_lists[0].
- ll_assert(arena.nfreepages == 0,
- "freepages == NULL but nfreepages > 0")
- arena.nextarena = self.arenas_lists[0]
- self.arenas_lists[0] = arena
- self.current_arena = ARENA_NULL
+ #
+ self.next_uninitialized_page = freepages
#
# Initialize the fields of the resulting page
llarena.arena_reserve(result, llmemory.sizeof(PAGE_HEADER))
page = llmemory.cast_adr_to_ptr(result, PAGE_PTR)
- page.arena = arena
- page.nfree = 0
- page.freeblock = result + self.hdrsize
+ rffi.setintfield(page, 'nfree', 0)
+ rffi.setintfield(page, 'freeblock', self.hdrsize)
page.nextpage = PAGE_NULL
ll_assert(self.page_for_size[size_class] == PAGE_NULL,
"allocate_new_page() called but a page is already waiting")
@@ -233,63 +198,52 @@
return page
- def _all_arenas(self):
- """For testing. Enumerates all arenas."""
- if self.current_arena:
- yield self.current_arena
- for arena in self.arenas_lists:
- while arena:
- yield arena
- arena = arena.nextarena
+ def allocate_new_arena(self):
+ """Allocates an arena and load it in self.next_uninitialized_page."""
+ arena_base = self.allocate_big_chunk(self.arena_size)
+ self.next_uninitialized_page = arena_base
+ self.num_uninitialized_pages = self.arena_size // self.page_size
+ allocate_new_arena._dont_inline_ = True
+ def allocate_big_chunk(self, arena_size):
+ if we_are_translated():
+ return self._allocate_new_arena_mmap(arena_size)
+ else:
+ return llarena.arena_malloc(arena_size, False)
- def allocate_new_arena(self):
- """Loads in self.current_arena the arena to allocate from next."""
+ def free_big_chunk(self, arena):
+ if we_are_translated():
+ pass # good enough
+ else:
+ llarena.arena_free(arena)
+
+ def _allocate_new_arena_mmap(self, arena_size):
#
- # Pick an arena from 'arenas_lists[i]', with i as small as possible
- # but > 0. Use caching with 'min_empty_nfreepages', which guarantees
- # that 'arenas_lists[1:min_empty_nfreepages]' are all empty.
- i = self.min_empty_nfreepages
- while i < self.max_pages_per_arena:
- #
- if self.arenas_lists[i] != ARENA_NULL:
- #
- # Found it.
- self.current_arena = self.arenas_lists[i]
- self.arenas_lists[i] = self.current_arena.nextarena
- return
- #
- i += 1
- self.min_empty_nfreepages = i
- #
- # No more arena with any free page. We must allocate a new arena.
- if not we_are_translated():
- for a in self._all_arenas():
- assert a.nfreepages == 0
- #
- # 'arena_base' points to the start of malloced memory; it might not
- # be a page-aligned address
- arena_base = llarena.arena_malloc(self.arena_size, False)
- if not arena_base:
- raise MemoryError("couldn't allocate the next arena")
- arena_end = arena_base + self.arena_size
- #
- # 'firstpage' points to the first unused page
- firstpage = start_of_page(arena_base + self.page_size - 1,
- self.page_size)
- # 'npages' is the number of full pages just allocated
- npages = (arena_end - firstpage) // self.page_size
- #
- # Allocate an ARENA object and initialize it
- arena = lltype.malloc(ARENA, flavor='raw')
- arena.base = arena_base
- arena.nfreepages = 0 # they are all uninitialized pages
- arena.totalpages = npages
- arena.freepages = firstpage
- self.num_uninitialized_pages = npages
- self.current_arena = arena
+ # Round up the number in arena_size.
+ arena_size = (arena_size + ARENA_SIZE - 1) & ~(ARENA_SIZE-1)
#
- allocate_new_arena._dont_inline_ = True
+ # Try to mmap() at a MAP_FIXED address, in a 'while' loop until it
+ # succeeds. The important part is that it must return an address
+ # that is in the lower 32GB of the addressable space.
+ while 1:
+ addr = self.next_arena_addr
+ if addr + arena_size > ARENA_ADDR_STOP:
+ raise MemoryError("exhausted the 32GB of memory")
+ self.next_arena_addr = addr + arena_size
+ flags = rmmap.MAP_PRIVATE | rmmap.MAP_ANONYMOUS | rmmap.MAP_FIXED
+ prot = rmmap.PROT_READ | rmmap.PROT_WRITE
+ arena_base = rmmap.c_mmap_safe(rffi.cast(rffi.CCHARP, addr),
+ arena_size, prot, flags, -1, 0)
+ if arena_base != rffi.cast(rffi.CCHARP, -1):
+ break
+ #
+ # 'arena_base' points to the start of mmap()ed memory.
+ # Sanity-check it.
+ if rffi.cast(lltype.Unsigned, arena_base) >= ARENA_ADDR_STOP:
+ raise MMapIgnoredFIXED("mmap() ignored the MAP_FIXED and returned"
+ " an address that is not in the first 32GB")
+ #
+ return rffi.cast(llmemory.Address, arena_base)
def mass_free(self, ok_to_free_func):
@@ -304,48 +258,13 @@
#
# Walk the pages in 'page_for_size[size_class]' and
# 'full_page_for_size[size_class]' and free some objects.
- # Pages completely freed are added to 'page.arena.freepages',
+ # Pages completely freed are added to 'self.freepages',
# and become available for reuse by any size class. Pages
# not completely freed are re-chained either in
# 'full_page_for_size[]' or 'page_for_size[]'.
self.mass_free_in_pages(size_class, ok_to_free_func)
#
size_class -= 1
- #
- # Rehash arenas into the correct arenas_lists[i]. If
- # 'self.current_arena' contains an arena too, it remains there.
- (self.old_arenas_lists, self.arenas_lists) = (
- self.arenas_lists, self.old_arenas_lists)
- #
- i = 0
- while i < self.max_pages_per_arena:
- self.arenas_lists[i] = ARENA_NULL
- i += 1
- #
- i = 0
- while i < self.max_pages_per_arena:
- arena = self.old_arenas_lists[i]
- while arena != ARENA_NULL:
- nextarena = arena.nextarena
- #
- if arena.nfreepages == arena.totalpages:
- #
- # The whole arena is empty. Free it.
- llarena.arena_free(arena.base)
- lltype.free(arena, flavor='raw')
- #
- else:
- # Insert 'arena' in the correct arenas_lists[n]
- n = arena.nfreepages
- ll_assert(n < self.max_pages_per_arena,
- "totalpages != nfreepages >= max_pages_per_arena")
- arena.nextarena = self.arenas_lists[n]
- self.arenas_lists[n] = arena
- #
- arena = nextarena
- i += 1
- #
- self.min_empty_nfreepages = 1
def mass_free_in_pages(self, size_class, ok_to_free_func):
@@ -398,33 +317,29 @@
def free_page(self, page):
"""Free a whole page."""
#
- # Insert the freed page in the arena's 'freepages' list.
- # If nfreepages == totalpages, then it will be freed at the
- # end of mass_free().
- arena = page.arena
- arena.nfreepages += 1
+ # Insert the freed page in the 'freepages' list.
pageaddr = llmemory.cast_ptr_to_adr(page)
pageaddr = llarena.getfakearenaaddress(pageaddr)
llarena.arena_reset(pageaddr, self.page_size, 0)
llarena.arena_reserve(pageaddr, llmemory.sizeof(llmemory.Address))
- pageaddr.address[0] = arena.freepages
- arena.freepages = pageaddr
+ pageaddr.address[0] = self.freepages
+ self.freepages = pageaddr
def walk_page(self, page, block_size, ok_to_free_func):
"""Walk over all objects in a page, and ask ok_to_free_func()."""
#
+ pageaddr = llarena.getfakearenaaddress(llmemory.cast_ptr_to_adr(page))
+ #
# 'freeblock' is the next free block
- freeblock = page.freeblock
+ freeblock = pageaddr + rffi.getintfield(page, 'freeblock')
#
# 'prevfreeblockat' is the address of where 'freeblock' was read from.
prevfreeblockat = lltype.direct_fieldptr(page, 'freeblock')
- prevfreeblockat = llmemory.cast_ptr_to_adr(prevfreeblockat)
#
- obj = llarena.getfakearenaaddress(llmemory.cast_ptr_to_adr(page))
- obj += self.hdrsize
+ obj = pageaddr + self.hdrsize
surviving = 0 # initially
- skip_free_blocks = page.nfree
+ skip_free_blocks = rffi.getintfield(page, 'nfree')
#
while True:
#
@@ -437,11 +352,14 @@
break
#
# 'obj' points to a free block. It means that
- # 'prevfreeblockat.address[0]' does not need to be updated.
+ # 'prevfreeblockat[0]' does not need to be updated.
# Just read the next free block from 'obj.address[0]'.
skip_free_blocks -= 1
- prevfreeblockat = obj
- freeblock = obj.address[0]
+ prevfreeblockat = llmemory.cast_adr_to_ptr(obj, FREEBLOCK_PTR)
+ freeblock = pageaddr + rffi.getintfield(prevfreeblockat,
+ 'freeblock')
+ prevfreeblockat = lltype.direct_fieldptr(prevfreeblockat,
+ 'freeblock')
#
else:
# 'obj' points to a valid object.
@@ -452,15 +370,20 @@
#
# The object should die.
llarena.arena_reset(obj, _dummy_size(block_size), 0)
- llarena.arena_reserve(obj,
- llmemory.sizeof(llmemory.Address))
+ llarena.arena_reserve(obj, llmemory.sizeof(FREEBLOCK))
# Insert 'obj' in the linked list of free blocks.
- prevfreeblockat.address[0] = obj
- prevfreeblockat = obj
- obj.address[0] = freeblock
+ prevfreeblockat[0] = rffi.cast(rffi.INT, obj - pageaddr)
+ prevfreeblockat = llmemory.cast_adr_to_ptr(obj,
+ FREEBLOCK_PTR)
+ prevfreeblockat.freeblock = rffi.cast(rffi.INT,
+ freeblock - pageaddr)
+ prevfreeblockat = lltype.direct_fieldptr(prevfreeblockat,
+ 'freeblock')
#
# Update the number of free objects in the page.
- page.nfree += 1
+ page_nfree = rffi.getintfield(page, 'nfree')
+ page_nfree += 1
+ rffi.setintfield(page, 'nfree', page_nfree)
#
else:
# The object survives.
@@ -477,35 +400,26 @@
def _nuninitialized(self, page, size_class):
# Helper for debugging: count the number of uninitialized blocks
- freeblock = page.freeblock
+ freeblock = rffi.getintfield(page, 'freeblock')
+ pageaddr = llmemory.cast_ptr_to_adr(page)
+ pageaddr = llarena.getfakearenaaddress(pageaddr)
for i in range(page.nfree):
- freeblock = freeblock.address[0]
- assert freeblock != NULL
- pageaddr = llarena.getfakearenaaddress(llmemory.cast_ptr_to_adr(page))
+ freeblockaddr = pageaddr + freeblock
+ freeblockptr = llmemory.cast_adr_to_ptr(freeblockaddr,
+ FREEBLOCK_PTR)
+ freeblock = rffi.getintfield(freeblockptr, 'freeblock')
+ assert freeblock != 0
num_initialized_blocks, rem = divmod(
- freeblock - pageaddr - self.hdrsize, size_class * WORD)
+ freeblock - self.hdrsize, size_class * WORD)
assert rem == 0, "page size_class misspecified?"
nblocks = self.nblocks_for_size[size_class]
return nblocks - num_initialized_blocks
# ____________________________________________________________
-# Helpers to go from a pointer to the start of its page
-def start_of_page(addr, page_size):
- """Return the address of the start of the page that contains 'addr'."""
- if we_are_translated():
- offset = llmemory.cast_adr_to_int(addr) % page_size
- return addr - offset
- else:
- return _start_of_page_untranslated(addr, page_size)
-
-def _start_of_page_untranslated(addr, page_size):
- assert isinstance(addr, llarena.fakearenaaddress)
- shift = WORD # for testing, we assume that the whole arena is not
- # on a page boundary
- ofs = ((addr.offset - shift) // page_size) * page_size + shift
- return llarena.fakearenaaddress(addr.arena, ofs)
+class MMapIgnoredFIXED(Exception):
+ pass
def _dummy_size(size):
if we_are_translated():
Copied: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/test/test_minimarkpage2.py (from r77571, pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/test/test_minimarkpage.py)
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/test/test_minimarkpage.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/test/test_minimarkpage2.py Tue Oct 5 13:09:40 2010
@@ -1,30 +1,23 @@
import py
-from pypy.rpython.memory.gc.minimarkpage import ArenaCollection
-from pypy.rpython.memory.gc.minimarkpage import PAGE_HEADER, PAGE_PTR
-from pypy.rpython.memory.gc.minimarkpage import PAGE_NULL, WORD
-from pypy.rpython.memory.gc.minimarkpage import _dummy_size
-from pypy.rpython.lltypesystem import lltype, llmemory, llarena
+from pypy.rpython.memory.gc.minimarkpage2 import ArenaCollection2
+from pypy.rpython.memory.gc.minimarkpage2 import PAGE_HEADER, PAGE_PTR
+from pypy.rpython.memory.gc.minimarkpage2 import PAGE_NULL, WORD
+from pypy.rpython.memory.gc.minimarkpage2 import FREEBLOCK, FREEBLOCK_PTR
+from pypy.rpython.memory.gc.minimarkpage2 import _dummy_size
+from pypy.rpython.lltypesystem import lltype, llmemory, llarena, rffi
from pypy.rpython.lltypesystem.llmemory import cast_ptr_to_adr
NULL = llmemory.NULL
-SHIFT = WORD
hdrsize = llmemory.raw_malloc_usage(llmemory.sizeof(PAGE_HEADER))
def test_allocate_arena():
- ac = ArenaCollection(SHIFT + 64*20, 64, 1)
+ ac = ArenaCollection2(64*20, 64, 1)
ac.allocate_new_arena()
assert ac.num_uninitialized_pages == 20
- upages = ac.current_arena.freepages
+ upages = ac.next_uninitialized_page
upages + 64*20 # does not raise
py.test.raises(llarena.ArenaError, "upages + 64*20 + 1")
- #
- ac = ArenaCollection(SHIFT + 64*20 + 7, 64, 1)
- ac.allocate_new_arena()
- assert ac.num_uninitialized_pages == 20
- upages = ac.current_arena.freepages
- upages + 64*20 + 7 # does not raise
- py.test.raises(llarena.ArenaError, "upages + 64*20 + 64")
def test_allocate_new_page():
@@ -36,24 +29,24 @@
assert (ac._nuninitialized(page, size_class) ==
(pagesize - hdrsize) // size)
assert page.nfree == 0
- page1 = page.freeblock - hdrsize
- assert llmemory.cast_ptr_to_adr(page) == page1
+ assert page.freeblock == hdrsize
assert page.nextpage == PAGE_NULL
#
- ac = ArenaCollection(arenasize, pagesize, 99)
+ ac = ArenaCollection2(arenasize, pagesize, 99)
assert ac.num_uninitialized_pages == 0
assert ac.total_memory_used == 0
#
page = ac.allocate_new_page(5)
checknewpage(page, 5)
assert ac.num_uninitialized_pages == 2
- assert ac.current_arena.freepages - pagesize == cast_ptr_to_adr(page)
+ assert lltype.typeOf(ac.next_uninitialized_page) == llmemory.Address
+ assert ac.next_uninitialized_page - pagesize == cast_ptr_to_adr(page)
assert ac.page_for_size[5] == page
#
page = ac.allocate_new_page(3)
checknewpage(page, 3)
assert ac.num_uninitialized_pages == 1
- assert ac.current_arena.freepages - pagesize == cast_ptr_to_adr(page)
+ assert ac.next_uninitialized_page - pagesize == cast_ptr_to_adr(page)
assert ac.page_for_size[3] == page
#
page = ac.allocate_new_page(4)
@@ -66,25 +59,24 @@
assert " " not in pagelayout.rstrip(" ")
nb_pages = len(pagelayout)
arenasize = pagesize * (nb_pages + 1) - 1
- ac = ArenaCollection(arenasize, pagesize, 9*WORD)
+ ac = ArenaCollection2(arenasize, pagesize, 9*WORD)
#
def link(pageaddr, size_class, size_block, nblocks, nusedblocks, step=1):
assert step in (1, 2)
llarena.arena_reserve(pageaddr, llmemory.sizeof(PAGE_HEADER))
page = llmemory.cast_adr_to_ptr(pageaddr, PAGE_PTR)
if step == 1:
- page.nfree = 0
+ page.nfree = rffi.cast(rffi.INT, 0)
nuninitialized = nblocks - nusedblocks
else:
- page.nfree = nusedblocks
+ page.nfree = rffi.cast(rffi.INT, nusedblocks)
nuninitialized = nblocks - 2*nusedblocks
- page.freeblock = pageaddr + hdrsize + nusedblocks * size_block
+ page.freeblock = rffi.cast(rffi.INT, hdrsize + nusedblocks*size_block)
if nusedblocks < nblocks:
chainedlists = ac.page_for_size
else:
chainedlists = ac.full_page_for_size
page.nextpage = chainedlists[size_class]
- page.arena = ac.current_arena
chainedlists[size_class] = page
if fill_with_objects:
for i in range(0, nusedblocks*step, step):
@@ -93,23 +85,27 @@
if step == 2:
prev = 'page.freeblock'
for i in range(1, nusedblocks*step, step):
- holeaddr = pageaddr + hdrsize + i * size_block
- llarena.arena_reserve(holeaddr,
- llmemory.sizeof(llmemory.Address))
- exec '%s = holeaddr' % prev in globals(), locals()
- prevhole = holeaddr
- prev = 'prevhole.address[0]'
- endaddr = pageaddr + hdrsize + 2*nusedblocks * size_block
- exec '%s = endaddr' % prev in globals(), locals()
+ holeofs = hdrsize + i * size_block
+ llarena.arena_reserve(pageaddr + holeofs,
+ llmemory.sizeof(FREEBLOCK))
+ exec '%s = rffi.cast(rffi.INT, holeofs)' % prev \
+ in globals(), locals()
+ prevhole = pageaddr + holeofs
+ prevhole = llmemory.cast_adr_to_ptr(prevhole,
+ FREEBLOCK_PTR)
+ prev = 'prevhole.freeblock'
+ endofs = hdrsize + 2*nusedblocks * size_block
+ exec '%s = rffi.cast(rffi.INT, endofs)' % prev \
+ in globals(), locals()
assert ac._nuninitialized(page, size_class) == nuninitialized
#
ac.allocate_new_arena()
num_initialized_pages = len(pagelayout.rstrip(" "))
- ac._startpageaddr = ac.current_arena.freepages
+ ac._startpageaddr = ac.next_uninitialized_page
if pagelayout.endswith(" "):
- ac.current_arena.freepages += pagesize * num_initialized_pages
+ ac.next_uninitialized_page += pagesize * num_initialized_pages
else:
- ac.current_arena.freepages = NULL
+ ac.next_uninitialized_page = NULL
ac.num_uninitialized_pages -= num_initialized_pages
#
for i in reversed(range(num_initialized_pages)):
@@ -122,9 +118,8 @@
link(pageaddr, size_class, size_block, nblocks, nblocks-1)
elif c == '.': # a free, but initialized, page
llarena.arena_reserve(pageaddr, llmemory.sizeof(llmemory.Address))
- pageaddr.address[0] = ac.current_arena.freepages
- ac.current_arena.freepages = pageaddr
- ac.current_arena.nfreepages += 1
+ pageaddr.address[0] = ac.freepages
+ ac.freepages = pageaddr
elif c == '#': # a random full page, in the list 'full_pages'
size_class = fill_with_objects or 1
size_block = WORD * size_class
@@ -151,7 +146,7 @@
assert llmemory.cast_ptr_to_adr(page) == pagenum(ac, expected_position)
def freepages(ac):
- return ac.current_arena.freepages
+ return ac.freepages or ac.next_uninitialized_page
def test_simple_arena_collection():
@@ -172,7 +167,7 @@
page = ac.allocate_new_page(6); checkpage(ac, page, 8)
assert freepages(ac) == pagenum(ac, 9) and ac.num_uninitialized_pages == 1
page = ac.allocate_new_page(7); checkpage(ac, page, 9)
- assert not ac.current_arena and ac.num_uninitialized_pages == 0
+ assert freepages(ac) == NULL and ac.num_uninitialized_pages == 0
def chkob(ac, num_page, pos_obj, obj):
@@ -217,18 +212,18 @@
page = getpage(ac, 0)
assert page.nfree == 3
assert ac._nuninitialized(page, 2) == 3
- chkob(ac, 0, 2*WORD, page.freeblock)
+ assert page.freeblock == hdrsize + 2*WORD
#
obj = ac.malloc(2*WORD); chkob(ac, 0, 2*WORD, obj)
obj = ac.malloc(2*WORD); chkob(ac, 0, 6*WORD, obj)
assert page.nfree == 1
assert ac._nuninitialized(page, 2) == 3
- chkob(ac, 0, 10*WORD, page.freeblock)
+ assert page.freeblock == hdrsize + 10*WORD
#
obj = ac.malloc(2*WORD); chkob(ac, 0, 10*WORD, obj)
assert page.nfree == 0
assert ac._nuninitialized(page, 2) == 3
- chkob(ac, 0, 12*WORD, page.freeblock)
+ assert page.freeblock == hdrsize + 12*WORD
#
obj = ac.malloc(2*WORD); chkob(ac, 0, 12*WORD, obj)
assert ac._nuninitialized(page, 2) == 2
@@ -288,7 +283,7 @@
assert page.nextpage == PAGE_NULL
assert ac._nuninitialized(page, 2) == 1
assert page.nfree == 0
- chkob(ac, 0, 4*WORD, page.freeblock)
+ assert page.freeblock == hdrsize + 4*WORD
assert freepages(ac) == NULL
def test_mass_free_emptied_page():
@@ -319,6 +314,14 @@
assert freepages(ac) == NULL
assert ac.page_for_size[2] == PAGE_NULL
+def deref(page, freeblock, repeat=1):
+ for i in range(repeat):
+ pageaddr = llmemory.cast_ptr_to_adr(page)
+ pageaddr = llarena.getfakearenaaddress(pageaddr)
+ obj = llmemory.cast_adr_to_ptr(pageaddr + freeblock, FREEBLOCK_PTR)
+ freeblock = obj.freeblock
+ return freeblock
+
def test_mass_free_full_is_partially_emptied():
pagesize = hdrsize + 9*WORD
ac = arena_collection_for_test(pagesize, "#", fill_with_objects=2)
@@ -334,9 +337,9 @@
assert page.nextpage == PAGE_NULL
assert ac._nuninitialized(page, 2) == 0
assert page.nfree == 2
- assert page.freeblock == pageaddr + hdrsize + 2*WORD
- assert page.freeblock.address[0] == pageaddr + hdrsize + 6*WORD
- assert page.freeblock.address[0].address[0] == pageaddr + hdrsize + 8*WORD
+ assert page.freeblock == hdrsize + 2*WORD
+ assert deref(page, page.freeblock) == hdrsize + 6*WORD
+ assert deref(page, page.freeblock, 2) == hdrsize + 8*WORD
assert freepages(ac) == NULL
assert ac.full_page_for_size[2] == PAGE_NULL
@@ -359,12 +362,10 @@
assert page.nextpage == PAGE_NULL
assert ac._nuninitialized(page, 2) == 4
assert page.nfree == 4
- assert page.freeblock == pageaddr + hdrsize + 2*WORD
- assert page.freeblock.address[0] == pageaddr + hdrsize + 6*WORD
- assert page.freeblock.address[0].address[0] == \
- pageaddr + hdrsize + 10*WORD
- assert page.freeblock.address[0].address[0].address[0] == \
- pageaddr + hdrsize + 14*WORD
+ assert deref(page, page.freeblock, 0) == hdrsize + 2*WORD
+ assert deref(page, page.freeblock, 1) == hdrsize + 6*WORD
+ assert deref(page, page.freeblock, 2) == hdrsize + 10*WORD
+ assert deref(page, page.freeblock, 3) == hdrsize + 14*WORD
assert freepages(ac) == NULL
assert ac.full_page_for_size[2] == PAGE_NULL
@@ -387,16 +388,12 @@
assert page.nextpage == PAGE_NULL
assert ac._nuninitialized(page, 2) == 4
assert page.nfree == 6
- fb = page.freeblock
- assert fb == pageaddr + hdrsize + 2*WORD
- assert fb.address[0] == pageaddr + hdrsize + 4*WORD
- assert fb.address[0].address[0] == pageaddr + hdrsize + 6*WORD
- assert fb.address[0].address[0].address[0] == \
- pageaddr + hdrsize + 10*WORD
- assert fb.address[0].address[0].address[0].address[0] == \
- pageaddr + hdrsize + 12*WORD
- assert fb.address[0].address[0].address[0].address[0].address[0] == \
- pageaddr + hdrsize + 14*WORD
+ assert deref(page, page.freeblock, 0) == hdrsize + 2*WORD
+ assert deref(page, page.freeblock, 1) == hdrsize + 4*WORD
+ assert deref(page, page.freeblock, 2) == hdrsize + 6*WORD
+ assert deref(page, page.freeblock, 3) == hdrsize + 10*WORD
+ assert deref(page, page.freeblock, 4) == hdrsize + 12*WORD
+ assert deref(page, page.freeblock, 5) == hdrsize + 14*WORD
assert freepages(ac) == NULL
assert ac.full_page_for_size[2] == PAGE_NULL
@@ -408,25 +405,20 @@
num_pages = 3
ac = arena_collection_for_test(pagesize, " " * num_pages)
live_objects = {}
+ all_arenas = []
#
- # Run the test until three arenas are freed. This is a quick test
- # that the arenas are really freed by the logic.
class DoneTesting(Exception):
counter = 0
def my_allocate_new_arena():
- # the following output looks cool on a 112-character-wide terminal.
- lst = sorted(ac._all_arenas(), key=lambda a: a.base.arena._arena_index)
- for a in lst:
- print a.base.arena, a.base.arena.usagemap
+ # the following output looks cool on a 208-character-wide terminal.
+ DoneTesting.counter += 1
+ if DoneTesting.counter > 9:
+ raise DoneTesting
+ for a in all_arenas:
+ print a.arena.usagemap
print '-' * 80
ac.__class__.allocate_new_arena(ac)
- a = ac.current_arena.base.arena
- def my_mark_freed():
- a.freed = True
- DoneTesting.counter += 1
- if DoneTesting.counter > 3:
- raise DoneTesting
- a.mark_freed = my_mark_freed
+ all_arenas.append(ac.next_uninitialized_page)
ac.allocate_new_arena = my_allocate_new_arena
try:
while True:
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py Tue Oct 5 13:09:40 2010
@@ -10,12 +10,12 @@
from pypy.rlib import rstack, rgc
from pypy.rlib.debug import ll_assert
from pypy.translator.backendopt import graphanalyze
-from pypy.translator.backendopt.support import var_needsgc
from pypy.annotation import model as annmodel
from pypy.rpython import annlowlevel
from pypy.rpython.rbuiltin import gen_cast
from pypy.rpython.memory.gctypelayout import ll_weakref_deref, WEAKREF
from pypy.rpython.memory.gctypelayout import convert_weakref_to, WEAKREFPTR
+from pypy.rpython.memory.gctypelayout import is_gc_pointer_or_hidden
from pypy.rpython.memory.gctransform.log import log
from pypy.tool.sourcetools import func_with_new_name
from pypy.rpython.lltypesystem.lloperation import llop, LL_OPERATIONS
@@ -68,9 +68,7 @@
mallocvars[op.result] = True
elif op.opname in ("setfield", "setarrayitem", "setinteriorfield"):
TYPE = op.args[-1].concretetype
- if (op.args[0] in mallocvars and
- isinstance(TYPE, lltype.Ptr) and
- TYPE.TO._gckind == "gc"):
+ if op.args[0] in mallocvars and is_gc_pointer_or_hidden(TYPE):
result.add(op)
else:
if collect_analyzer.analyze(op):
@@ -1017,15 +1015,20 @@
v_struct = hop.spaceop.args[0]
v_newvalue = hop.spaceop.args[-1]
assert opname in ('setfield', 'setarrayitem', 'setinteriorfield')
- assert isinstance(v_newvalue.concretetype, lltype.Ptr)
+ assert is_gc_pointer_or_hidden(v_newvalue.concretetype)
# XXX for some GCs the skipping if the newvalue is a constant won't be
# ok
if (self.write_barrier_ptr is not None
and not isinstance(v_newvalue, Constant)
and v_struct.concretetype.TO._gckind == "gc"
and hop.spaceop not in self.clean_sets):
- v_newvalue = hop.genop("cast_ptr_to_adr", [v_newvalue],
- resulttype = llmemory.Address)
+ if v_newvalue.concretetype == llmemory.HiddenGcRef32:
+ ...
+ v_newvalue = hop.genop("cast_ptr_to_adr", [v_newvalue],
+ resulttype = llmemory.Address)
+ else:
+ v_newvalue = hop.genop("cast_ptr_to_adr", [v_newvalue],
+ resulttype = llmemory.Address)
v_structaddr = hop.genop("cast_ptr_to_adr", [v_struct],
resulttype = llmemory.Address)
if (self.write_barrier_from_array_ptr is not None and
@@ -1089,7 +1092,7 @@
GCTransformer.gct_setfield(self, hop)
def var_needs_set_transform(self, var):
- return var_needsgc(var)
+ return is_gc_pointer_or_hidden(var.concretetype)
def push_alive_nopyobj(self, var, llops):
pass
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py Tue Oct 5 13:09:40 2010
@@ -366,6 +366,10 @@
#
# Helpers to discover GC pointers inside structures
+def is_gc_pointer_or_hidden(TYPE):
+ return ((isinstance(TYPE, lltype.Ptr) and TYPE.TO._gckind == 'gc')
+ or TYPE == llmemory.HiddenGcRef32)
+
def offsets_to_gc_pointers(TYPE):
offsets = []
if isinstance(TYPE, lltype.Struct):
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gcwrapper.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gcwrapper.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gcwrapper.py Tue Oct 5 13:09:40 2010
@@ -1,4 +1,5 @@
from pypy.rpython.lltypesystem import lltype, llmemory, llheap
+from pypy.rpython.lltypesystem.lloperation import llop
from pypy.rpython import llinterp
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.memory import gctypelayout
@@ -88,9 +89,12 @@
def setinterior(self, toplevelcontainer, inneraddr, INNERTYPE, newvalue,
offsets=()):
if (lltype.typeOf(toplevelcontainer).TO._gckind == 'gc' and
- (isinstance(INNERTYPE, lltype.Ptr) and
- INNERTYPE.TO._gckind == 'gc')
- or INNERTYPE == llmemory.HiddenGcRef32):
+ is_gc_pointer_or_hidden(INNERTYPE)):
+ #
+ if INNERTYPE == llmemory.HiddenGcRef32:
+ newvalueaddr = llop.show_from_adr32(llmemory.Address, newvalue)
+ else:
+ newvalueaddr = llmemory.cast_ptr_to_adr(newvalue)
#
wb = True
if self.has_write_barrier_from_array:
@@ -99,7 +103,7 @@
assert (type(index) is int # <- fast path
or lltype.typeOf(index) == lltype.Signed)
self.gc.write_barrier_from_array(
- llmemory.cast_ptr_to_adr(newvalue),
+ newvalueaddr,
llmemory.cast_ptr_to_adr(toplevelcontainer),
index)
wb = False
@@ -107,7 +111,7 @@
#
if wb:
self.gc.write_barrier(
- llmemory.cast_ptr_to_adr(newvalue),
+ newvalueaddr,
llmemory.cast_ptr_to_adr(toplevelcontainer))
llheap.setinterior(toplevelcontainer, inneraddr, INNERTYPE, newvalue)
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py Tue Oct 5 13:09:40 2010
@@ -881,15 +881,20 @@
def gen_startupcode(f, database):
# generate the start-up code and put it into a function
+ if database.late_initializations_hiddengcref32:
+ gen_late_initializations_hiddengcref32(f, database)
print >> f, 'char *RPython_StartupCode(void) {'
print >> f, '\tchar *error = NULL;'
- for line in database.gcpolicy.gc_startup_code():
- print >> f,"\t" + line
# put float infinities in global constants, we should not have so many of them for now to make
# a table+loop preferable
for dest, value in database.late_initializations:
print >> f, "\t%s = %s;" % (dest, value)
+ if database.late_initializations_hiddengcref32:
+ print >> f, "\tpypy_init_hiddengcref32();"
+
+ for line in database.gcpolicy.gc_startup_code():
+ print >> f,"\t" + line
firsttime = True
for node in database.containerlist:
@@ -904,6 +909,25 @@
print >> f, '\treturn error;'
print >> f, '}'
+def gen_late_initializations_hiddengcref32(f, database):
+ print >> f, 'static void* pypy_hiddengcref32[] = {'
+ for access_expr, name in database.late_initializations_hiddengcref32:
+ print >> f, '\t&%s, %s,' % (access_expr, name)
+ print >> f, '''\tNULL /* sentinel */
+};
+
+static void pypy_init_hiddengcref32(void)
+{
+\tvoid** p;
+\tfor (p = pypy_hiddengcref32; p[0] != NULL; p += 2)
+\t{
+\t\thiddengcref32_t h;
+\t\tOP_HIDE_INTO_ADR32((p[1]), h);
+\t\t*(hiddengcref32_t**)(p[0]) = h;
+\t}
+}
+'''
+
def commondefs(defines):
from pypy.rlib.rarithmetic import LONG_BIT
defines['PYPY_LONG_BIT'] = LONG_BIT
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py Tue Oct 5 13:09:40 2010
@@ -790,9 +790,12 @@
'-+'[value > 0])
elif TYPE == llmemory.HiddenGcRef32:
if value.adr64:
+ name = db.get(value.adr64.ptr)
db.late_initializations_hiddengcref32.append((access_expr,
- value))
- expr = '0 /*HIDE_INTO_ADR32%s*/' % db.get(value.adr64.ptr)
+ name))
+ if not name.startswith('('):
+ name = '(%s)' % name
+ expr = '0 /*HIDE_INTO_ADR32%s*/' % name
else:
expr = '0'
else:
From arigo at codespeak.net Tue Oct 5 13:33:41 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 13:33:41 +0200 (CEST)
Subject: [pypy-svn] r77598 - in pypy/branch/32ptr-on-64bit/pypy:
rpython/memory/gctransform translator/c translator/c/test
Message-ID: <20101005113341.5032D282B90@codespeak.net>
Author: arigo
Date: Tue Oct 5 13:33:39 2010
New Revision: 77598
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_newgc.py
Log:
Finish the basic version. test_newgc starts to pass.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py Tue Oct 5 13:33:39 2010
@@ -11,6 +11,7 @@
from pypy.rlib.debug import ll_assert
from pypy.translator.backendopt import graphanalyze
from pypy.annotation import model as annmodel
+from pypy.objspace.flow.model import Constant
from pypy.rpython import annlowlevel
from pypy.rpython.rbuiltin import gen_cast
from pypy.rpython.memory.gctypelayout import ll_weakref_deref, WEAKREF
@@ -1010,45 +1011,60 @@
return None
def transform_generic_set(self, hop):
- from pypy.objspace.flow.model import Constant
opname = hop.spaceop.opname
v_struct = hop.spaceop.args[0]
v_newvalue = hop.spaceop.args[-1]
assert opname in ('setfield', 'setarrayitem', 'setinteriorfield')
assert is_gc_pointer_or_hidden(v_newvalue.concretetype)
- # XXX for some GCs the skipping if the newvalue is a constant won't be
- # ok
if (self.write_barrier_ptr is not None
- and not isinstance(v_newvalue, Constant)
and v_struct.concretetype.TO._gckind == "gc"
and hop.spaceop not in self.clean_sets):
- if v_newvalue.concretetype == llmemory.HiddenGcRef32:
- ...
- v_newvalue = hop.genop("cast_ptr_to_adr", [v_newvalue],
- resulttype = llmemory.Address)
- else:
- v_newvalue = hop.genop("cast_ptr_to_adr", [v_newvalue],
- resulttype = llmemory.Address)
- v_structaddr = hop.genop("cast_ptr_to_adr", [v_struct],
- resulttype = llmemory.Address)
- if (self.write_barrier_from_array_ptr is not None and
- self._set_into_gc_array_part(hop.spaceop) is not None):
- self.write_barrier_from_array_calls += 1
- v_index = self._set_into_gc_array_part(hop.spaceop)
- assert v_index.concretetype == lltype.Signed
- hop.genop("direct_call", [self.write_barrier_from_array_ptr,
- self.c_const_gc,
- v_newvalue,
- v_structaddr,
- v_index])
- else:
- self.write_barrier_calls += 1
- hop.genop("direct_call", [self.write_barrier_ptr,
- self.c_const_gc,
- v_newvalue,
- v_structaddr])
+ self._add_write_barrier_call(hop, v_struct, v_newvalue)
hop.rename('bare_' + opname)
+ def _add_write_barrier_call(self, hop, v_struct, v_newvalue):
+ if isinstance(v_newvalue, Constant):
+ # comes from a Constant -- skip
+ # XXX for some GCs the skipping if the newvalue is
+ # a constant won't be ok
+ return
+ if v_newvalue.concretetype == llmemory.HiddenGcRef32:
+ v_newvalue = self._fetch_unpacked_pointer(hop, v_newvalue)
+ if isinstance(v_newvalue, Constant):
+ # comes from a Constant -- skip
+ return
+ else:
+ v_newvalue = hop.genop("cast_ptr_to_adr", [v_newvalue],
+ resulttype = llmemory.Address)
+ v_structaddr = hop.genop("cast_ptr_to_adr", [v_struct],
+ resulttype = llmemory.Address)
+ if (self.write_barrier_from_array_ptr is not None and
+ self._set_into_gc_array_part(hop.spaceop) is not None):
+ self.write_barrier_from_array_calls += 1
+ v_index = self._set_into_gc_array_part(hop.spaceop)
+ assert v_index.concretetype == lltype.Signed
+ hop.genop("direct_call", [self.write_barrier_from_array_ptr,
+ self.c_const_gc,
+ v_newvalue,
+ v_structaddr,
+ v_index])
+ else:
+ self.write_barrier_calls += 1
+ hop.genop("direct_call", [self.write_barrier_ptr,
+ self.c_const_gc,
+ v_newvalue,
+ v_structaddr])
+
+ def _fetch_unpacked_pointer(self, hop, v_value):
+ # optimization for the common case where this setfield is preceded
+ # by hide_into_adr32()
+ for op in hop.llops[::-1]:
+ if op.opname == 'hide_into_adr32' and op.result == v_value:
+ return op.args[0]
+ else:
+ return hop.genop("show_from_adr32", [v_value],
+ resulttype = llmemory.Address)
+
def transform_getfield_typeptr(self, hop):
# this would become quite a lot of operations, even if it compiles
# to C code that is just as efficient as "obj->typeptr". To avoid
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py Tue Oct 5 13:33:39 2010
@@ -923,7 +923,7 @@
\t{
\t\thiddengcref32_t h;
\t\tOP_HIDE_INTO_ADR32((p[1]), h);
-\t\t*(hiddengcref32_t**)(p[0]) = h;
+\t\t*(hiddengcref32_t*)(p[0]) = h;
\t}
}
'''
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_newgc.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_newgc.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_newgc.py Tue Oct 5 13:33:39 2010
@@ -673,7 +673,7 @@
return f
- def test_callback_with_collect(self):
+ def XXXXXXXXXtest_callback_with_collect(self):
assert self.run('callback_with_collect')
def define_can_move(cls):
From arigo at codespeak.net Tue Oct 5 14:09:29 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 14:09:29 +0200 (CEST)
Subject: [pypy-svn] r77599 -
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc
Message-ID: <20101005120929.B14E336C228@codespeak.net>
Author: arigo
Date: Tue Oct 5 14:09:28 2010
New Revision: 77599
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py
Log:
In case of compressptr, complain (crash with FixedSizeObjectTooLarge) if
allocating a fixed-size object that is too large to fit in the
ArenaCollection. The default limit should probably be increased quite a
bit.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py Tue Oct 5 14:09:28 2010
@@ -514,6 +514,11 @@
cardheadersize = 0
extra_flags = 0
#
+ # We get here for large fixed-size objects. Complain if
+ # self.config.compressptr is set.
+ if self.config.compressptr and not self.is_varsize(typeid):
+ raise FixedSizeObjectTooLarge
+ #
else:
# Reserve N extra words containing card bits before the object.
extra_words = self.card_marking_words_for_length(length)
@@ -1564,3 +1569,7 @@
else:
self.all_objects.append((rawobj, nsize))
self.total_memory_used += nsize
+
+
+class FixedSizeObjectTooLarge(Exception):
+ pass
From arigo at codespeak.net Tue Oct 5 14:27:23 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 14:27:23 +0200 (CEST)
Subject: [pypy-svn] r77600 - in pypy/branch/32ptr-on-64bit/pypy: rpython
rpython/lltypesystem translator/c
Message-ID: <20101005122723.A07F4282B90@codespeak.net>
Author: arigo
Date: Tue Oct 5 14:27:21 2010
New Revision: 77600
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py
pypy/branch/32ptr-on-64bit/pypy/rpython/rmodel.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py
Log:
Check-in before I think about changing the approach.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py Tue Oct 5 14:27:21 2010
@@ -3,7 +3,8 @@
from pypy.config.translationoption import IS_64_BITS
from pypy.rpython.rmodel import Repr, inputconst
from pypy.rpython.lltypesystem import lltype, llmemory, rffi
-
+from pypy.rpython.lltypesystem.lloperation import llop
+from pypy.rpython.error import TyperError
def get_compressed_gcref_repr(rtyper, baserepr):
@@ -38,6 +39,8 @@
class CompressedGcRefRepr(Repr):
lowleveltype = llmemory.HiddenGcRef32
+ ll_hash_function = None
+ ll_fasthash_function = None
def __init__(self, mgr, baserepr):
self.mgr = mgr
@@ -50,6 +53,42 @@
assert T == self.BASETYPE
return llmemory._hiddengcref32(llmemory.cast_ptr_to_adr(ptr))
+ def get_ll_eq_function(self):
+ if self.baserepr.get_ll_eq_function() is not None:
+ raise TyperError("%r has an eq function" % (self.baserepr,))
+ return None
+
+ def get_ll_hash_function(self):
+ if self.ll_hash_function is None:
+ basefunc = self.baserepr.get_ll_hash_function()
+ BASETYPE = self.BASETYPE
+ #
+ def ll_hiddengcref32_hash(x):
+ x = llop.show_from_adr32(BASETYPE, x)
+ return basefunc(x)
+ #
+ return self.ll_hash_function
+
+ def get_ll_fasthash_function(self):
+ if self.ll_fasthash_function is None:
+ basefunc = self.baserepr.get_ll_fasthash_function()
+ if basefunc is None:
+ return None
+ BASETYPE = self.BASETYPE
+ #
+ def ll_hiddengcref32_fasthash(x):
+ x = llop.show_from_adr32(BASETYPE, x)
+ return basefunc(x)
+ #
+ return self.ll_fasthash_function
+
+ def get_ll_dummyval_obj(self, rtyper, s_value):
+ DummyVal()
+
+
+class DummyVal(object):
+ ll_dummy_value = llop.hide_into_adr32xxx
+
class __extend__(pairtype(Repr, CompressedGcRefRepr)):
def convert_from_to((r_from, r_to), v, llops):
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/rmodel.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/rmodel.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/rmodel.py Tue Oct 5 14:27:21 2010
@@ -438,8 +438,11 @@
from pypy.rpython import rclass
if (isinstance(item_repr, rclass.AbstractInstanceRepr) and
getattr(item_repr, 'gcflavor', 'gc') == 'gc'):
- #if rtyper.annotator.translator.config.translation.compressptr:
- # return externalvsinternalfield(rtyper, item_repr)
+ if rtyper.annotator.translator.config.translation.compressptr:
+ item_repr, internal_item_repr = externalvsinternalfield(rtyper,
+ item_repr)
+ if internal_item_repr is not item_repr:
+ return item_repr, internal_item_repr
return item_repr, rclass.getinstancerepr(rtyper, None)
return item_repr, item_repr
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py Tue Oct 5 14:27:21 2010
@@ -789,7 +789,7 @@
expr = '0.0 /* patched later by %sinfinity */' % (
'-+'[value > 0])
elif TYPE == llmemory.HiddenGcRef32:
- if value.adr64:
+ if value.special_value is None:
name = db.get(value.adr64.ptr)
db.late_initializations_hiddengcref32.append((access_expr,
name))
@@ -797,7 +797,8 @@
name = '(%s)' % name
expr = '0 /*HIDE_INTO_ADR32%s*/' % name
else:
- expr = '0'
+ assert isinstance(value.special_value, int)
+ expr = str(value.special_value)
else:
expr = db.get(value)
if TYPE is Void:
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py Tue Oct 5 14:27:21 2010
@@ -150,7 +150,8 @@
def name_hiddengcref32(value, db):
# The only prebuilt HiddenGcRef32 that should occur in a translated C
# program occur as fields or items of a GcStruct or GcArray.
- db.get(value.adr64)
+ if not value.special_value:
+ db.get(value.adr64)
return 'HIDE_INTO_ADR32(???) /* see primitive.py, name_hiddengcref32() */'
def name_small_integer(value, db):
From fijal at codespeak.net Tue Oct 5 14:28:34 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Tue, 5 Oct 2010 14:28:34 +0200 (CEST)
Subject: [pypy-svn] r77601 - in pypy/benchmarks: . unladen_swallow
Message-ID: <20101005122834.6007A36C228@codespeak.net>
Author: fijal
Date: Tue Oct 5 14:28:32 2010
New Revision: 77601
Modified:
pypy/benchmarks/runner.py
pypy/benchmarks/unladen_swallow/perf.py
Log:
An ability to store raw results
Modified: pypy/benchmarks/runner.py
==============================================================================
--- pypy/benchmarks/runner.py (original)
+++ pypy/benchmarks/runner.py Tue Oct 5 14:28:32 2010
@@ -11,7 +11,8 @@
def run_and_store(benchmark_set, result_filename, pypy_c_path, revision=0,
options='', branch='trunk', args='', upload=False,
- force_host=None, fast=False, baseline=sys.executable):
+ force_host=None, fast=False, baseline=sys.executable,
+ full_store=False):
funcs = perf.BENCH_FUNCS.copy()
funcs.update(perf._FindAllBenchmarks(benchmarks.__dict__))
opts = ['-b', ','.join(benchmark_set), '--inherit_env=PATH',
@@ -20,6 +21,8 @@
opts += ['--fast']
if args:
opts += ['--args', args]
+ if full_store:
+ opts.append('--no_statistics')
opts += [baseline, pypy_c_path]
results = perf.main(opts, funcs)
f = open(str(result_filename), "w")
@@ -93,6 +96,8 @@
help="Force the hostname")
parser.add_option("--fast", default=False, action="store_true",
help="Run shorter benchmark runs")
+ parser.add_option("--full-store", default=False, action="store_true",
+ help="")
options, args = parser.parse_args(argv)
benchmarks = options.benchmarks.split(',')
for benchmark in benchmarks:
@@ -101,7 +106,7 @@
run_and_store(benchmarks, options.output_filename, options.pypy_c,
options.revision, args=options.args, upload=options.upload,
force_host=options.force_host, fast=options.fast,
- baseline=options.baseline)
+ baseline=options.baseline, full_store=options.full_store)
if __name__ == '__main__':
main(sys.argv[1:])
Modified: pypy/benchmarks/unladen_swallow/perf.py
==============================================================================
--- pypy/benchmarks/unladen_swallow/perf.py (original)
+++ pypy/benchmarks/unladen_swallow/perf.py Tue Oct 5 14:28:32 2010
@@ -406,6 +406,14 @@
return ("%(base_time)f -> %(changed_time)f: %(time_delta)s"
% self.__dict__)
+class RawResult(object):
+ def __init__(self, base_times, changed_times):
+ self.base_times = base_times
+ self.changed_times = changed_times
+
+ def string_representation(self):
+ return "Raw results: %s %s" % (self.base_times, self.changed_times)
+
def CompareMemoryUsage(base_usage, changed_usage, options):
"""Like CompareMultipleRuns, but for memory usage."""
max_base, max_changed = max(base_usage), max(changed_usage)
@@ -660,6 +668,8 @@
human consumption.
"""
assert len(base_times) == len(changed_times)
+ if options.no_statistics:
+ return RawResult(base_times, changed_times)
if len(base_times) == 1:
# With only one data point, we can't do any of the interesting stats
# below.
@@ -1564,6 +1574,8 @@
parser.add_option("--no_charts", default=False, action="store_true",
help=("Don't use google charts for displaying the"
" graph outcome"))
+ parser.add_option("--no_statistics", default=False, action="store_true",
+ help=("Don't perform statistics - return raw data"))
options, args = parser.parse_args(argv)
if len(args) != 2:
From arigo at codespeak.net Tue Oct 5 14:36:56 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 14:36:56 +0200 (CEST)
Subject: [pypy-svn] r77602 - in pypy/branch/minor-cleanup/pypy/translator:
jvm/test oosupport/test_template
Message-ID: <20101005123656.D477B36C228@codespeak.net>
Author: arigo
Date: Tue Oct 5 14:36:55 2010
New Revision: 77602
Modified:
pypy/branch/minor-cleanup/pypy/translator/jvm/test/test_class.py
pypy/branch/minor-cleanup/pypy/translator/oosupport/test_template/class_.py
Log:
Kill stuff here.
Modified: pypy/branch/minor-cleanup/pypy/translator/jvm/test/test_class.py
==============================================================================
--- pypy/branch/minor-cleanup/pypy/translator/jvm/test/test_class.py (original)
+++ pypy/branch/minor-cleanup/pypy/translator/jvm/test/test_class.py Tue Oct 5 14:36:55 2010
@@ -1,6 +1,6 @@
import py
from pypy.translator.jvm.test.runtest import JvmTest
-from pypy.translator.oosupport.test_template.class_ import BaseTestClass, BaseTestSpecialcase
+from pypy.translator.oosupport.test_template.class_ import BaseTestClass
class TestJvmClass(JvmTest, BaseTestClass):
def test_overridden_classattr_as_defaults(self):
@@ -26,6 +26,3 @@
def test_specialize_methods(self):
py.test.skip('ABSTRACT METHOD FIX: RE-TEST AFTER MERGE')
-
-class TestJvmSpecialCase(JvmTest, BaseTestSpecialcase):
- pass
Modified: pypy/branch/minor-cleanup/pypy/translator/oosupport/test_template/class_.py
==============================================================================
--- pypy/branch/minor-cleanup/pypy/translator/oosupport/test_template/class_.py (original)
+++ pypy/branch/minor-cleanup/pypy/translator/oosupport/test_template/class_.py Tue Oct 5 14:36:55 2010
@@ -1,6 +1,5 @@
import py
from pypy.rpython.test import test_rclass
-from pypy.rpython.test.test_rspecialcase import BaseTestRspecialcase
class BaseTestClass(test_rclass.TestOOtype):
def test_abstract_method(self):
@@ -66,6 +65,3 @@
def test_cast_object_mix_null(self):
py.test.skip('cannot return ootype.NULL from translated functions')
-
-class BaseTestSpecialcase(BaseTestRspecialcase):
- pass
From arigo at codespeak.net Tue Oct 5 14:37:46 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 14:37:46 +0200 (CEST)
Subject: [pypy-svn] r77603 - in pypy/trunk/pypy: annotation annotation/test
module/sys objspace/std rpython rpython/test translator/goal
translator/jvm/test translator/oosupport/test_template
Message-ID: <20101005123746.6948536C224@codespeak.net>
Author: arigo
Date: Tue Oct 5 14:37:44 2010
New Revision: 77603
Removed:
pypy/trunk/pypy/rpython/rspecialcase.py
pypy/trunk/pypy/rpython/test/test_rspecialcase.py
Modified:
pypy/trunk/pypy/annotation/policy.py
pypy/trunk/pypy/annotation/test/test_annrpython.py
pypy/trunk/pypy/module/sys/__init__.py
pypy/trunk/pypy/module/sys/state.py
pypy/trunk/pypy/objspace/std/fake.py
pypy/trunk/pypy/objspace/std/objspace.py
pypy/trunk/pypy/rpython/rtyper.py
pypy/trunk/pypy/translator/goal/ann_override.py
pypy/trunk/pypy/translator/jvm/test/test_class.py
pypy/trunk/pypy/translator/oosupport/test_template/class_.py
Log:
Merge branch/minor-cleanup:
Kill an old way to do specialization at the annotator level.
Modified: pypy/trunk/pypy/annotation/policy.py
==============================================================================
--- pypy/trunk/pypy/annotation/policy.py (original)
+++ pypy/trunk/pypy/annotation/policy.py Tue Oct 5 14:37:44 2010
@@ -1,4 +1,4 @@
-# base annotation policy for overrides and specialization
+# base annotation policy for specialization
from pypy.annotation.specialize import default_specialize as default
from pypy.annotation.specialize import specialize_argvalue, specialize_argtype, specialize_arglistitemtype
from pypy.annotation.specialize import memo
@@ -41,7 +41,7 @@
if directive is None:
return pol.default_specialize
- # specialize|override:name[(args)]
+ # specialize[(args)]
directive_parts = directive.split('(', 1)
if len(directive_parts) == 1:
[name] = directive_parts
@@ -60,14 +60,6 @@
except AttributeError:
raise AttributeError("%r specialize tag not defined in annotation"
"policy %s" % (name, pol))
- if directive.startswith('override:'):
- # different signature: override__xyz(*args_s)
- if parms:
- raise Exception, "override:* specialisations don't support parameters"
- def specialize_override(funcdesc, args_s):
- funcdesc.overridden = True
- return specializer(*args_s)
- return specialize_override
else:
if not parms:
return specializer
@@ -92,9 +84,5 @@
from pypy.rpython.annlowlevel import LowLevelAnnotatorPolicy
return LowLevelAnnotatorPolicy.specialize__ll_and_arg(*args)
- def override__ignore(pol, *args):
- bk = getbookkeeper()
- return bk.immutablevalue(None)
-
class StrictAnnotatorPolicy(AnnotatorPolicy):
allow_someobjects = False
Modified: pypy/trunk/pypy/annotation/test/test_annrpython.py
==============================================================================
--- pypy/trunk/pypy/annotation/test/test_annrpython.py (original)
+++ pypy/trunk/pypy/annotation/test/test_annrpython.py Tue Oct 5 14:37:44 2010
@@ -766,28 +766,6 @@
s = a.build_types(f, [list])
assert s.classdef is a.bookkeeper.getuniqueclassdef(IndexError) # KeyError ignored because l is a list
- def test_overrides(self):
- excs = []
- def record_exc(e):
- """NOT_RPYTHON"""
- excs.append(sys.exc_info)
- record_exc._annspecialcase_ = "override:record_exc"
- def g():
- pass
- def f():
- try:
- g()
- except Exception, e:
- record_exc(e)
- class MyAnnotatorPolicy(policy.AnnotatorPolicy):
-
- def override__record_exc(pol, s_e):
- return a.bookkeeper.immutablevalue(None)
-
- a = self.RPythonAnnotator(policy=MyAnnotatorPolicy())
- s = a.build_types(f, [])
- assert s.const is None
-
def test_freeze_protocol(self):
class Stuff:
def __init__(self, flag):
Modified: pypy/trunk/pypy/module/sys/__init__.py
==============================================================================
--- pypy/trunk/pypy/module/sys/__init__.py (original)
+++ pypy/trunk/pypy/module/sys/__init__.py Tue Oct 5 14:37:44 2010
@@ -7,13 +7,15 @@
"""Sys Builtin Module. """
def __init__(self, space, w_name):
"""NOT_RPYTHON""" # because parent __init__ isn't
+ if space.config.translating:
+ del self.__class__.interpleveldefs['pypy_getudir']
super(Module, self).__init__(space, w_name)
self.checkinterval = 100
self.recursionlimit = 100
self.w_default_encoder = None
self.defaultencoding = "ascii"
self.filesystemencoding = None
-
+
interpleveldefs = {
'__name__' : '(space.wrap("sys"))',
'__doc__' : '(space.wrap("PyPy sys module"))',
@@ -37,7 +39,7 @@
'argv' : 'state.get(space).w_argv',
'warnoptions' : 'state.get(space).w_warnoptions',
'builtin_module_names' : 'state.w_None',
- 'pypy_getudir' : 'state.pypy_getudir',
+ 'pypy_getudir' : 'state.pypy_getudir', # not translated
'pypy_initial_path' : 'state.pypy_initial_path',
'_getframe' : 'vm._getframe',
Modified: pypy/trunk/pypy/module/sys/state.py
==============================================================================
--- pypy/trunk/pypy/module/sys/state.py (original)
+++ pypy/trunk/pypy/module/sys/state.py Tue Oct 5 14:37:44 2010
@@ -95,15 +95,8 @@
def getio(space):
return space.fromcache(IOState)
-def _pypy_getudir(space):
- """NOT_RPYTHON"""
+def pypy_getudir(space):
+ """NOT_RPYTHON
+ (should be removed from interpleveldefs before translation)"""
from pypy.tool.udir import udir
return space.wrap(str(udir))
-_pypy_getudir._annspecialcase_ = "override:ignore"
-
-# we need the indirection because this function will live in a dictionary with other
-# RPYTHON functions and share call sites with them. Better it not be a special-case
-# directly.
-def pypy_getudir(space):
- return _pypy_getudir(space)
-
Modified: pypy/trunk/pypy/objspace/std/fake.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/fake.py (original)
+++ pypy/trunk/pypy/objspace/std/fake.py Tue Oct 5 14:37:44 2010
@@ -21,7 +21,6 @@
#debug_print("faking obj %s" % x)
ft = fake_type(type(x))
return ft(space, x)
-fake_object._annspecialcase_ = "override:fake_object"
import sys
@@ -47,7 +46,6 @@
w_exc = space.wrap(exc)
w_value = space.wrap(value)
raise OperationError, OperationError(w_exc, w_value), tb
-wrap_exception._annspecialcase_ = "override:ignore"
def fake_type(cpy_type):
assert type(cpy_type) is type
Modified: pypy/trunk/pypy/objspace/std/objspace.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/objspace.py (original)
+++ pypy/trunk/pypy/objspace/std/objspace.py Tue Oct 5 14:37:44 2010
@@ -242,7 +242,6 @@
w_result = getattr(self, 'w_' + x.__name__)
return w_result
return None
- wrap_exception_cls._annspecialcase_ = "override:wrap_exception_cls"
def unwrap(self, w_obj):
if isinstance(w_obj, Wrappable):
Modified: pypy/trunk/pypy/rpython/rtyper.py
==============================================================================
--- pypy/trunk/pypy/rpython/rtyper.py (original)
+++ pypy/trunk/pypy/rpython/rtyper.py Tue Oct 5 14:37:44 2010
@@ -421,7 +421,7 @@
assert noexclink.exitcase is None
if pos == "removed":
# the exception cannot actually occur at all.
- # See for example rspecialcase.rtype_call_specialcase().
+ # This is set by calling exception_cannot_occur().
# We just remove all exception links.
block.exitswitch = None
block.exits = block.exits[:1]
@@ -1019,7 +1019,7 @@
from pypy.rpython import rint, rbool, rfloat
from pypy.rpython import rrange
from pypy.rpython import rstr, rdict, rlist
-from pypy.rpython import rclass, rbuiltin, rpbc, rspecialcase
+from pypy.rpython import rclass, rbuiltin, rpbc
from pypy.rpython import rexternalobj
from pypy.rpython import rptr
from pypy.rpython import rgeneric
Modified: pypy/trunk/pypy/translator/goal/ann_override.py
==============================================================================
--- pypy/trunk/pypy/translator/goal/ann_override.py (original)
+++ pypy/trunk/pypy/translator/goal/ann_override.py Tue Oct 5 14:37:44 2010
@@ -27,21 +27,6 @@
pol.pypytypes = {}
pol.single_space = single_space
- #def override__wrap_exception_cls(pol, space, x):
- # import pypy.objspace.std.typeobject as typeobject
- # clsdef = getbookkeeper().getuniqueclassdef(typeobject.W_TypeObject)
- # return annmodel.SomeInstance(clsdef, can_be_None=True)
- #
- #def override__fake_object(pol, space, x):
- # from pypy.interpreter import typedef
- # clsdef = getbookkeeper().getuniqueclassdef(typedef.W_Root)
- # return annmodel.SomeInstance(clsdef)
- #
- #def override__cpy_compile(pol, self, source, filename, mode, flags):
- # from pypy.interpreter import pycode
- # clsdef = getbookkeeper().getuniqueclassdef(pycode.PyCode)
- # return annmodel.SomeInstance(clsdef)
-
def specialize__wrap(pol, funcdesc, args_s):
from pypy.interpreter.baseobjspace import Wrappable
from pypy.annotation.classdef import ClassDef
Modified: pypy/trunk/pypy/translator/jvm/test/test_class.py
==============================================================================
--- pypy/trunk/pypy/translator/jvm/test/test_class.py (original)
+++ pypy/trunk/pypy/translator/jvm/test/test_class.py Tue Oct 5 14:37:44 2010
@@ -1,6 +1,6 @@
import py
from pypy.translator.jvm.test.runtest import JvmTest
-from pypy.translator.oosupport.test_template.class_ import BaseTestClass, BaseTestSpecialcase
+from pypy.translator.oosupport.test_template.class_ import BaseTestClass
class TestJvmClass(JvmTest, BaseTestClass):
def test_overridden_classattr_as_defaults(self):
@@ -26,6 +26,3 @@
def test_specialize_methods(self):
py.test.skip('ABSTRACT METHOD FIX: RE-TEST AFTER MERGE')
-
-class TestJvmSpecialCase(JvmTest, BaseTestSpecialcase):
- pass
Modified: pypy/trunk/pypy/translator/oosupport/test_template/class_.py
==============================================================================
--- pypy/trunk/pypy/translator/oosupport/test_template/class_.py (original)
+++ pypy/trunk/pypy/translator/oosupport/test_template/class_.py Tue Oct 5 14:37:44 2010
@@ -1,6 +1,5 @@
import py
from pypy.rpython.test import test_rclass
-from pypy.rpython.test.test_rspecialcase import BaseTestRspecialcase
class BaseTestClass(test_rclass.TestOOtype):
def test_abstract_method(self):
@@ -66,6 +65,3 @@
def test_cast_object_mix_null(self):
py.test.skip('cannot return ootype.NULL from translated functions')
-
-class BaseTestSpecialcase(BaseTestRspecialcase):
- pass
From arigo at codespeak.net Tue Oct 5 14:38:03 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 14:38:03 +0200 (CEST)
Subject: [pypy-svn] r77604 - pypy/branch/minor-cleanup
Message-ID: <20101005123803.95F5F36C228@codespeak.net>
Author: arigo
Date: Tue Oct 5 14:38:01 2010
New Revision: 77604
Removed:
pypy/branch/minor-cleanup/
Log:
Branch merged.
From antocuni at codespeak.net Tue Oct 5 15:36:57 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Tue, 5 Oct 2010 15:36:57 +0200 (CEST)
Subject: [pypy-svn] r77606 - in pypy/branch/jitffi/pypy/annotation: . test
Message-ID: <20101005133657.83F4D282B90@codespeak.net>
Author: antocuni
Date: Tue Oct 5 15:36:55 2010
New Revision: 77606
Modified:
pypy/branch/jitffi/pypy/annotation/model.py
pypy/branch/jitffi/pypy/annotation/test/test_model.py
Log:
add a failing test and fix it. The problem was that annotation_to_lltype
tries to see if the annotation is contained in one of those inside
annotation_to_ll_map, but for SomeSingleFloat whose .const!=None the
.contains() raises TypeError when trying to compare the two consts. Moving it
as the first solves the problem.
Modified: pypy/branch/jitffi/pypy/annotation/model.py
==============================================================================
--- pypy/branch/jitffi/pypy/annotation/model.py (original)
+++ pypy/branch/jitffi/pypy/annotation/model.py Tue Oct 5 15:36:55 2010
@@ -574,11 +574,11 @@
NUMBER = object()
annotation_to_ll_map = [
+ (SomeSingleFloat(), lltype.SingleFloat),
(s_None, lltype.Void), # also matches SomeImpossibleValue()
(s_Bool, lltype.Bool),
(SomeInteger(knowntype=r_ulonglong), NUMBER),
(SomeFloat(), lltype.Float),
- (SomeSingleFloat(), lltype.SingleFloat),
(SomeChar(), lltype.Char),
(SomeUnicodeCodePoint(), lltype.UniChar),
(SomeAddress(), llmemory.Address),
Modified: pypy/branch/jitffi/pypy/annotation/test/test_model.py
==============================================================================
--- pypy/branch/jitffi/pypy/annotation/test/test_model.py (original)
+++ pypy/branch/jitffi/pypy/annotation/test/test_model.py Tue Oct 5 15:36:55 2010
@@ -128,7 +128,7 @@
assert isinstance(s_p, SomeOOInstance) and s_p.ootype == C
def test_annotation_to_lltype():
- from pypy.rlib.rarithmetic import r_uint
+ from pypy.rlib.rarithmetic import r_uint, r_singlefloat
s_i = SomeInteger()
s_pos = SomeInteger(nonneg=True)
s_1 = SomeInteger(nonneg=True); s_1.const = 1
@@ -151,6 +151,9 @@
C = ootype.Instance('C', ROOT, {})
ref = SomeOOInstance(C)
assert annotation_to_lltype(ref) == C
+ s_singlefloat = SomeSingleFloat()
+ s_singlefloat.const = r_singlefloat(0.0)
+ assert annotation_to_lltype(s_singlefloat) == lltype.SingleFloat
def test_ll_union():
PS1 = lltype.Ptr(lltype.GcStruct('s'))
From arigo at codespeak.net Tue Oct 5 15:41:31 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 15:41:31 +0200 (CEST)
Subject: [pypy-svn] r77607 - in pypy/branch/32ptr-on-64bit/pypy: annotation
rpython rpython/lltypesystem rpython/lltypesystem/test
rpython/memory
Message-ID: <20101005134131.B807F282B90@codespeak.net>
Author: arigo
Date: Tue Oct 5 15:41:29 2010
New Revision: 77607
Modified:
pypy/branch/32ptr-on-64bit/pypy/annotation/model.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/test/test_llmemory.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py
pypy/branch/32ptr-on-64bit/pypy/rpython/raddress.py
Log:
In-progress: make HiddenGcRef32 a Ptr(GcOpaque), like GCREF.
Modified: pypy/branch/32ptr-on-64bit/pypy/annotation/model.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/annotation/model.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/annotation/model.py Tue Oct 5 15:41:29 2010
@@ -518,14 +518,6 @@
def can_be_none(self):
return False
-# for compressed 32-bit "pointers" on 64-bit platforms
-
-class SomeHiddenGcRef32(SomeObject):
- immutable = True
-
- def can_be_none(self):
- return False
-
#____________________________________________________________
# annotation of low-level types
@@ -590,7 +582,6 @@
(SomeChar(), lltype.Char),
(SomeUnicodeCodePoint(), lltype.UniChar),
(SomeAddress(), llmemory.Address),
- (SomeHiddenGcRef32(), llmemory.HiddenGcRef32),
]
def annotation_to_lltype(s_val, info=None):
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py Tue Oct 5 15:41:29 2010
@@ -9,6 +9,7 @@
from pypy.rpython.lltypesystem import lltype
from pypy.tool.uid import uid
from pypy.rlib.objectmodel import we_are_translated
+from pypy.rlib import rarithmetic
class AddressOffset(Symbolic):
@@ -565,22 +566,11 @@
# ____________________________________________________________
-class _hiddengcref32(object):
- """A 'hidden' compressed 32-bit value that represents a
- full, 64-bit GC pointer."""
-
- def __init__(self, adr64):
- self.adr64 = adr64
-
- def __repr__(self):
- return '<_hiddengcref32>'
-
- def __str__(self):
- return '<_hiddengcref32 %s>' % (self.adr64,)
-
-compressed_null_addr = _hiddengcref32(NULL)
-HiddenGcRef32 = lltype.Primitive("HiddenGcRef32", compressed_null_addr)
-_hiddengcref32._TYPE = HiddenGcRef32
+# HiddenGcRef32 is only on 64-bits with 'compressptr' enabled.
+# It is actually a 32-bit pointer, not a normal 64-bit pointer,
+# but it's represented as a regular lltype.Ptr() in order to integrate
+# better with the rest of the code.
+HiddenGcRef32 = lltype.Ptr(lltype.GcOpaqueType('HIDDENGCREF32'))
class OddValueMarker(AddressOffset):
"""This is the value '1'. Used as an odd-valued marker by the GC."""
@@ -653,8 +643,11 @@
class _char_fakeaccessor(_fakeaccessor):
TYPE = lltype.Char
-class _hiddengcref32_fakeaccessor(_fakeaccessor):
- TYPE = HiddenGcRef32
+r_int32 = rarithmetic.build_int("r_int", True, 32)
+INT32 = lltype.build_number("INT", r_int32)
+
+class _int32_fakeaccessor(_fakeaccessor):
+ TYPE = INT32
class _address_fakeaccessor(_fakeaccessor):
TYPE = Address
@@ -683,14 +676,14 @@
"char": lltype.Char,
"address": Address,
"float": lltype.Float,
- "hiddengcref32": HiddenGcRef32,
+ "int32": INT32,
}
fakeaddress.signed = property(_signed_fakeaccessor)
fakeaddress.float = property(_float_fakeaccessor)
fakeaddress.char = property(_char_fakeaccessor)
fakeaddress.address = property(_address_fakeaccessor)
-fakeaddress.hiddengcref32 = property(_hiddengcref32_fakeaccessor)
+fakeaddress.int32 = property(_int32_fakeaccessor)
fakeaddress._TYPE = Address
# the obtained address will not keep the object alive. e.g. if the object is
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py Tue Oct 5 15:41:29 2010
@@ -426,9 +426,6 @@
'gc_gettypeptr_group': LLOp(canfold=True),
'get_member_index': LLOp(canfold=True),
- 'hide_into_adr32': LLOp(canrun=True),
- 'show_from_adr32': LLOp(canrun=True),
-
# __________ used by the JIT ________
'jit_marker': LLOp(),
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py Tue Oct 5 15:41:29 2010
@@ -3,16 +3,14 @@
from pypy.config.translationoption import IS_64_BITS
from pypy.rpython.rmodel import Repr, inputconst
from pypy.rpython.lltypesystem import lltype, llmemory, rffi
-from pypy.rpython.lltypesystem.lloperation import llop
from pypy.rpython.error import TyperError
def get_compressed_gcref_repr(rtyper, baserepr):
# Return either the original baserepr, or another repr standing for
# a HiddenGcRef32. The idea is that we only get a HiddenGcRef32 for
- # fixed-sized structures (XXX that are not too big); thus this is only
- # for structures that gets allocated by the minimarkpage2 mmap()-
- # within-32GB-of-RAM.
+ # fixed-sized structures; they get allocated by the minimarkpage2
+ # mmap()-within-32GB-of-RAM.
if baserepr.lowleveltype.TO._is_varsize():
return baserepr
try:
@@ -51,7 +49,7 @@
ptr = self.baserepr.convert_const(value)
T = lltype.typeOf(ptr)
assert T == self.BASETYPE
- return llmemory._hiddengcref32(llmemory.cast_ptr_to_adr(ptr))
+ return lltype.cast_opaque_ptr(self.lowleveltype, ptr)
def get_ll_eq_function(self):
if self.baserepr.get_ll_eq_function() is not None:
@@ -64,9 +62,10 @@
BASETYPE = self.BASETYPE
#
def ll_hiddengcref32_hash(x):
- x = llop.show_from_adr32(BASETYPE, x)
+ x = lltype.cast_opaque_ptr(BASETYPE, x)
return basefunc(x)
#
+ self.ll_hash_function = ll_hiddengcref32_hash
return self.ll_hash_function
def get_ll_fasthash_function(self):
@@ -77,27 +76,33 @@
BASETYPE = self.BASETYPE
#
def ll_hiddengcref32_fasthash(x):
- x = llop.show_from_adr32(BASETYPE, x)
+ x = lltype.cast_opaque_ptr(BASETYPE, x)
return basefunc(x)
#
+ self.ll_fasthash_function = ll_hiddengcref32_hash
return self.ll_fasthash_function
def get_ll_dummyval_obj(self, rtyper, s_value):
- DummyVal()
+ return DummyVal()
class DummyVal(object):
- ll_dummy_value = llop.hide_into_adr32xxx
+ TYPE = llmemory.HiddenGcRef32.TO
+ ll_dummy_value = lltype.opaqueptr(TYPE, "dummy_value", dummy_value=True)
+ def _freeze_(self):
+ return True
class __extend__(pairtype(Repr, CompressedGcRefRepr)):
def convert_from_to((r_from, r_to), v, llops):
assert r_from.lowleveltype.TO._gckind == 'gc'
- return llops.genop('hide_into_adr32', [v],
+ assert not isinstance(r_from.lowleveltype.TO, lltype.GcOpaqueType)
+ return llops.genop('cast_opaque_ptr', [v],
resulttype=llmemory.HiddenGcRef32)
class __extend__(pairtype(CompressedGcRefRepr, Repr)):
def convert_from_to((r_from, r_to), v, llops):
assert r_to.lowleveltype.TO._gckind == 'gc'
- return llops.genop('show_from_adr32', [v],
+ assert not isinstance(r_to.lowleveltype.TO, lltype.GcOpaqueType)
+ return llops.genop('cast_opaque_ptr', [v],
resulttype=r_to.lowleveltype)
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/test/test_llmemory.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/test/test_llmemory.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/test/test_llmemory.py Tue Oct 5 15:41:29 2010
@@ -645,3 +645,13 @@
#assert cast_int_to_adr(i) == adr -- depends on ll2ctypes details
i = cast_adr_to_int(NULL, mode="forced")
assert type(i) is int and i == 0
+
+def test_HiddenGcRef32():
+ from pypy.config.translationoption import IS_64_BITS
+ if not IS_64_BITS:
+ py.test.skip("only on 64-bits")
+ S = lltype.GcStruct('S')
+ p = lltype.malloc(S)
+ q = lltype.cast_opaque_ptr(HiddenGcRef32, p)
+ r = lltype.cast_opaque_ptr(lltype.Ptr(S), q)
+ assert r == p
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py Tue Oct 5 15:41:29 2010
@@ -366,10 +366,6 @@
#
# Helpers to discover GC pointers inside structures
-def is_gc_pointer_or_hidden(TYPE):
- return ((isinstance(TYPE, lltype.Ptr) and TYPE.TO._gckind == 'gc')
- or TYPE == llmemory.HiddenGcRef32)
-
def offsets_to_gc_pointers(TYPE):
offsets = []
if isinstance(TYPE, lltype.Struct):
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/raddress.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/raddress.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/raddress.py Tue Oct 5 15:41:29 2010
@@ -2,7 +2,7 @@
from pypy.tool.pairtype import pairtype
from pypy.annotation import model as annmodel
from pypy.rpython.lltypesystem.llmemory import NULL, Address, \
- cast_adr_to_int, fakeaddress, HiddenGcRef32, _hiddengcref32
+ cast_adr_to_int, fakeaddress
from pypy.rpython.rmodel import Repr, IntegerRepr
from pypy.rpython.rptr import PtrRepr
from pypy.rpython.lltypesystem import lltype
@@ -24,13 +24,6 @@
def rtyper_makekey(self):
return self.__class__, self.type
-class __extend__(annmodel.SomeHiddenGcRef32):
- def rtyper_makerepr(self, rtyper):
- return hiddengcref32_repr
-
- def rtyper_makekey(self):
- return self.__class__,
-
class AddressRepr(Repr):
lowleveltype = Address
@@ -145,14 +138,3 @@
def convert_from_to((r_ptr, r_addr), v, llops):
return llops.genop('cast_ptr_to_adr', [v], resulttype=Address)
-
-# ____________________________________________________________
-
-class HiddenGcRef32Repr(Repr):
- lowleveltype = HiddenGcRef32
-
- def convert_const(self, value):
- assert isinstance(value, _hiddengcref32)
- return value
-
-hiddengcref32_repr = HiddenGcRef32Repr()
From antocuni at codespeak.net Tue Oct 5 16:21:43 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Tue, 5 Oct 2010 16:21:43 +0200 (CEST)
Subject: [pypy-svn] r77608 - in pypy/branch/jitffi/pypy: jit/metainterp/test
rlib rlib/test
Message-ID: <20101005142143.5C8B0282B90@codespeak.net>
Author: antocuni
Date: Tue Oct 5 16:21:41 2010
New Revision: 77608
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py
pypy/branch/jitffi/pypy/rlib/libffi.py
pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
Log:
always cast the integer input arguments into a long
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py Tue Oct 5 16:21:41 2010
@@ -10,6 +10,8 @@
class TestFfiCall(LLJitMixin, _TestLibffiCall):
+ # ===> ../../../rlib/test/test_libffi.py
+
def call(self, funcspec, args, RESULT, init_result=0):
"""
Call the function specified by funcspec in a loop, and let the jit to
Modified: pypy/branch/jitffi/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/libffi.py Tue Oct 5 16:21:41 2010
@@ -33,10 +33,11 @@
types._import()
+ at specialize.arg(0)
def _fits_into_long(TYPE):
- if not isinstance(TYPE, lltype.Number):
+ if not isinstance(TYPE, lltype.Primitive):
return False
- if TYPE is rffi.FLOAT or TYPE is rffi.DOUBLE:
+ if TYPE is lltype.Void or TYPE is rffi.FLOAT or TYPE is rffi.DOUBLE:
return False
sz = rffi.sizeof(TYPE)
return sz <= rffi.sizeof(rffi.LONG)
@@ -51,8 +52,9 @@
@specialize.argtype(1)
def arg(self, val):
TYPE = lltype.typeOf(val)
- if TYPE is rffi.LONG:
+ if _fits_into_long(TYPE):
cls = IntArg
+ val = rffi.cast(rffi.LONG, val)
elif TYPE is rffi.DOUBLE:
cls = FloatArg
else:
Modified: pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_libffi.py Tue Oct 5 16:21:41 2010
@@ -63,17 +63,17 @@
# prepare C code as an example, so we can load it and call
# it via rlib.libffi
c_file = udir.ensure("test_libffi", dir=1).join("foolib.c")
- c_file.write(py.code.Source('''
- int sum_xy(int x, double y)
- {
- return (x + (int)y);
- }
-
- unsigned char cast_to_uchar_and_ovf(int x)
- {
- return 200+(unsigned char)x;
- }
- '''))
+ # automatically collect the C source from the docstrings of the tests
+ snippets = []
+ for name in dir(cls):
+ if name.startswith('test_'):
+ meth = getattr(cls, name)
+ # the heuristic to determine it it's really C code could be
+ # improved: so far we just check that there is a '{' :-)
+ if meth.__doc__ is not None and '{' in meth.__doc__:
+ snippets.append(meth.__doc__)
+ #
+ c_file.write(py.code.Source('\n'.join(snippets)))
eci = ExternalCompilationInfo(export_symbols=[])
cls.libfoo_name = str(platform.compile([c_file], eci, 'x',
standalone=False))
@@ -110,6 +110,12 @@
# ------------------------------------------------------------------------
def test_simple(self):
+ """
+ int sum_xy(int x, double y)
+ {
+ return (x + (int)y);
+ }
+ """
libfoo = self.get_libfoo()
func = (libfoo, 'sum_xy', [types.sint, types.double], types.sint)
res = self.call(func, [38, 4.2], rffi.LONG)
@@ -130,8 +136,26 @@
self.check_loops(call=1)
def test_cast_result(self):
+ """
+ unsigned char cast_to_uchar_and_ovf(int x)
+ {
+ return 200+(unsigned char)x;
+ }
+ """
libfoo = self.get_libfoo()
func = (libfoo, 'cast_to_uchar_and_ovf', [types.sint], types.uchar)
res = self.call(func, [0], rffi.UCHAR)
assert res == 200
self.check_loops(call=1)
+
+ def test_cast_argument(self):
+ """
+ int many_args(char a, int b)
+ {
+ return a+b;
+ }
+ """
+ libfoo = self.get_libfoo()
+ func = (libfoo, 'many_args', [types.uchar, types.sint], types.sint)
+ res = self.call(func, [chr(20), 22], rffi.LONG)
+ assert res == 42
From arigo at codespeak.net Tue Oct 5 16:28:54 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 16:28:54 +0200 (CEST)
Subject: [pypy-svn] r77609 - in pypy/branch/32ptr-on-64bit/pypy:
rpython/lltypesystem rpython/memory rpython/memory/gc
rpython/memory/gctransform translator/c translator/c/src
translator/c/test
Message-ID: <20101005142854.AF50C282B90@codespeak.net>
Author: arigo
Date: Tue Oct 5 16:28:52 2010
New Revision: 77609
Added:
pypy/branch/32ptr-on-64bit/pypy/translator/c/src/hiddengcref32.h
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gcwrapper.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/database.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/funcgen.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/src/commondefs.h
pypy/branch/32ptr-on-64bit/pypy/translator/c/src/g_prerequisite.h
pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_boehm.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_lladdresses.py
Log:
Still random progress.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py Tue Oct 5 16:28:52 2010
@@ -9,7 +9,6 @@
from pypy.rpython.lltypesystem import lltype
from pypy.tool.uid import uid
from pypy.rlib.objectmodel import we_are_translated
-from pypy.rlib import rarithmetic
class AddressOffset(Symbolic):
@@ -643,12 +642,6 @@
class _char_fakeaccessor(_fakeaccessor):
TYPE = lltype.Char
-r_int32 = rarithmetic.build_int("r_int", True, 32)
-INT32 = lltype.build_number("INT", r_int32)
-
-class _int32_fakeaccessor(_fakeaccessor):
- TYPE = INT32
-
class _address_fakeaccessor(_fakeaccessor):
TYPE = Address
@@ -671,19 +664,22 @@
raise TypeError(TARGETTYPE)
ptr[0] = value
+class _address32_fakeaccessor(_address_fakeaccessor):
+ pass
+
supported_access_types = {"signed": lltype.Signed,
"unsigned": lltype.Unsigned,
"char": lltype.Char,
"address": Address,
"float": lltype.Float,
- "int32": INT32,
+ "address32": Address,
}
fakeaddress.signed = property(_signed_fakeaccessor)
fakeaddress.float = property(_float_fakeaccessor)
fakeaddress.char = property(_char_fakeaccessor)
fakeaddress.address = property(_address_fakeaccessor)
-fakeaddress.int32 = property(_int32_fakeaccessor)
+fakeaddress.address32 = property(_address32_fakeaccessor)
fakeaddress._TYPE = Address
# the obtained address will not keep the object alive. e.g. if the object is
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py Tue Oct 5 16:28:52 2010
@@ -1,5 +1,4 @@
from pypy.rpython.lltypesystem import lltype, llmemory, llarena
-from pypy.rpython.lltypesystem.lloperation import llop
from pypy.rlib.debug import ll_assert
from pypy.rpython.memory.gcheader import GCHeaderBuilder
from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
@@ -212,14 +211,11 @@
# special handling of HiddenGcRef32 on 64-bit platforms
ofs = llmemory.remove_odd_value_marker(ofs)
item = obj + ofs
- address = llop.show_from_adr32(llmemory.Address,
- item.hiddengcref32[0])
+ address = item.address32[0]
if self.is_valid_gc_object(address):
newaddr = callback(address, arg)
if newaddr is not None:
- newaddr32 = llop.hide_into_adr32(llmemory.HiddenGcRef32,
- newaddr)
- item.hiddengcref32[0] = newaddr32
+ item.address32[0] = newaddr
else:
# common case
item = obj + ofs
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py Tue Oct 5 16:28:52 2010
@@ -16,7 +16,6 @@
from pypy.rpython.rbuiltin import gen_cast
from pypy.rpython.memory.gctypelayout import ll_weakref_deref, WEAKREF
from pypy.rpython.memory.gctypelayout import convert_weakref_to, WEAKREFPTR
-from pypy.rpython.memory.gctypelayout import is_gc_pointer_or_hidden
from pypy.rpython.memory.gctransform.log import log
from pypy.tool.sourcetools import func_with_new_name
from pypy.rpython.lltypesystem.lloperation import llop, LL_OPERATIONS
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py Tue Oct 5 16:28:52 2010
@@ -389,10 +389,10 @@
#adr = llmemory.cast_ptr_to_adr(ex)
#for off in offsets:
# (adr + off)
- elif isinstance(TYPE, lltype.Ptr) and TYPE.TO._gckind == 'gc':
- offsets.append(0)
elif TYPE == llmemory.HiddenGcRef32:
offsets.append(llmemory.OddValueMarker())
+ elif isinstance(TYPE, lltype.Ptr) and TYPE.TO._gckind == 'gc':
+ offsets.append(0)
return offsets
def gc_pointers_inside(v, adr, mutable_only=False):
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gcwrapper.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gcwrapper.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gcwrapper.py Tue Oct 5 16:28:52 2010
@@ -89,12 +89,7 @@
def setinterior(self, toplevelcontainer, inneraddr, INNERTYPE, newvalue,
offsets=()):
if (lltype.typeOf(toplevelcontainer).TO._gckind == 'gc' and
- is_gc_pointer_or_hidden(INNERTYPE)):
- #
- if INNERTYPE == llmemory.HiddenGcRef32:
- newvalueaddr = llop.show_from_adr32(llmemory.Address, newvalue)
- else:
- newvalueaddr = llmemory.cast_ptr_to_adr(newvalue)
+ isinstance(INNERTYPE, lltype.Ptr) and INNERTYPE.TO._gckind == 'gc'):
#
wb = True
if self.has_write_barrier_from_array:
@@ -103,7 +98,7 @@
assert (type(index) is int # <- fast path
or lltype.typeOf(index) == lltype.Signed)
self.gc.write_barrier_from_array(
- newvalueaddr,
+ llmemory.cast_ptr_to_adr(newvalue),
llmemory.cast_ptr_to_adr(toplevelcontainer),
index)
wb = False
@@ -111,7 +106,7 @@
#
if wb:
self.gc.write_barrier(
- newvalueaddr,
+ llmemory.cast_ptr_to_adr(newvalue),
llmemory.cast_ptr_to_adr(toplevelcontainer))
llheap.setinterior(toplevelcontainer, inneraddr, INNERTYPE, newvalue)
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/database.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/database.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/database.py Tue Oct 5 16:28:52 2010
@@ -4,6 +4,7 @@
ContainerType, OpaqueType, FixedSizeArray, _uninitialized
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.rpython.lltypesystem.llmemory import WeakRef, _WeakRefType, GCREF
+from pypy.rpython.lltypesystem.llmemory import HiddenGcRef32
from pypy.rpython.lltypesystem.rffi import CConstant
from pypy.rpython.lltypesystem import llgroup
from pypy.tool.sourcetools import valid_identifier
@@ -99,7 +100,7 @@
return node
def gettype(self, T, varlength=1, who_asks=None, argnames=[]):
- if isinstance(T, Primitive) or T == GCREF:
+ if isinstance(T, Primitive) or T == GCREF or T == HiddenGcRef32:
return PrimitiveType[T]
elif isinstance(T, Ptr):
if (isinstance(T.TO, OpaqueType) and
@@ -181,7 +182,7 @@
if isinstance(obj, CConstant):
return obj.c_name # without further checks
T = typeOf(obj)
- if isinstance(T, Primitive) or T == GCREF:
+ if isinstance(T, Primitive) or T == GCREF or T == HiddenGcRef32:
return PrimitiveName[T](obj, self)
elif isinstance(T, Ptr):
if (isinstance(T.TO, OpaqueType) and
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/funcgen.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/funcgen.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/funcgen.py Tue Oct 5 16:28:52 2010
@@ -10,7 +10,7 @@
from pypy.rpython.lltypesystem.lltype import pyobjectptr, ContainerType
from pypy.rpython.lltypesystem.lltype import Struct, Array, FixedSizeArray
from pypy.rpython.lltypesystem.lltype import ForwardReference, FuncType
-from pypy.rpython.lltypesystem.llmemory import Address
+from pypy.rpython.lltypesystem.llmemory import Address, HiddenGcRef32
from pypy.translator.backendopt.ssa import SSI_to_SSA
from pypy.translator.backendopt.innerloop import find_inner_loops
from pypy.tool.identity_dict import identity_dict
@@ -649,7 +649,15 @@
OP_CAST_PTR_TO_ADR = OP_CAST_POINTER
OP_CAST_ADR_TO_PTR = OP_CAST_POINTER
- OP_CAST_OPAQUE_PTR = OP_CAST_POINTER
+
+ def OP_CAST_OPAQUE_PTR(self, op):
+ if op.result.concretetype == HiddenGcRef32:
+ return 'OP_HIDE_INTO_ADR32(%s, %s);' % (self.expr(op.args[0]),
+ self.expr(op.result))
+ if op.args[0].concretetype == HiddenGcRef32:
+ return 'OP_SHOW_FROM_ADR32(%s, %s);' % (self.expr(op.args[0]),
+ self.expr(op.result))
+ return self.OP_CAST_OPAQUE_PTR(op)
def OP_CAST_INT_TO_PTR(self, op):
TYPE = self.lltypemap(op.result)
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py Tue Oct 5 16:28:52 2010
@@ -910,6 +910,7 @@
print >> f, '}'
def gen_late_initializations_hiddengcref32(f, database):
+ print >> f
print >> f, 'static void* pypy_hiddengcref32[] = {'
for access_expr, name in database.late_initializations_hiddengcref32:
print >> f, '\t&%s, %s,' % (access_expr, name)
@@ -922,7 +923,7 @@
\tfor (p = pypy_hiddengcref32; p[0] != NULL; p += 2)
\t{
\t\thiddengcref32_t h;
-\t\tOP_HIDE_INTO_ADR32((p[1]), h);
+\t\tOP_HIDE_INTO_ADR32_CHECK((p[1]), h);
\t\t*(hiddengcref32_t*)(p[0]) = h;
\t}
}
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py Tue Oct 5 16:28:52 2010
@@ -2,7 +2,8 @@
Struct, Array, FixedSizeArray, FuncType, PyObjectType, typeOf, \
GcStruct, GcArray, RttiStruct, ContainerType, \
parentlink, Ptr, PyObject, Void, OpaqueType, Float, \
- RuntimeTypeInfo, getRuntimeTypeInfo, Char, _subarray
+ RuntimeTypeInfo, getRuntimeTypeInfo, Char, _subarray, \
+ cast_opaque_ptr
from pypy.rpython.lltypesystem import llmemory, llgroup
from pypy.translator.c.funcgen import FunctionCodeGenerator
from pypy.translator.c.external import CExternalFunctionCodeGenerator
@@ -788,17 +789,15 @@
db.late_initializations.append((access_expr, db.get(value)))
expr = '0.0 /* patched later by %sinfinity */' % (
'-+'[value > 0])
- elif TYPE == llmemory.HiddenGcRef32:
- if value.special_value is None:
- name = db.get(value.adr64.ptr)
- db.late_initializations_hiddengcref32.append((access_expr,
- name))
- if not name.startswith('('):
- name = '(%s)' % name
- expr = '0 /*HIDE_INTO_ADR32%s*/' % name
- else:
- assert isinstance(value.special_value, int)
- expr = str(value.special_value)
+ elif (TYPE == llmemory.HiddenGcRef32
+ and value and not hasattr(value, 'dummy_value')):
+ realobj = value._obj.container
+ realvalue = cast_opaque_ptr(Ptr(typeOf(realobj)), value)
+ name = db.get(realvalue)
+ db.late_initializations_hiddengcref32.append((access_expr, name))
+ if not name.startswith('('):
+ name = '(%s)' % name
+ expr = '0 /*HIDE_INTO_ADR32%s*/' % name
else:
expr = db.get(value)
if TYPE is Void:
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py Tue Oct 5 16:28:52 2010
@@ -148,10 +148,16 @@
return 'NULL'
def name_hiddengcref32(value, db):
+ if hasattr(value, 'dummy_value'):
+ return '1 /* dummy value */'
+ elif not value:
+ return '0' # NULL
# The only prebuilt HiddenGcRef32 that should occur in a translated C
- # program occur as fields or items of a GcStruct or GcArray.
- if not value.special_value:
- db.get(value.adr64)
+ # program occur as fields or items of a GcStruct or GcArray. They are
+ # handled by late_initializations_hiddengcref32.
+ realobj = value._obj.container
+ realvalue = cast_opaque_ptr(Ptr(typeOf(realobj)), value)
+ db.get(realvalue) # force following dependencies
return 'HIDE_INTO_ADR32(???) /* see primitive.py, name_hiddengcref32() */'
def name_small_integer(value, db):
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/src/commondefs.h
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/src/commondefs.h (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/src/commondefs.h Tue Oct 5 16:28:52 2010
@@ -65,15 +65,6 @@
# define SIZEOF_LONG 8
# define SIZEOF_LONG_LONG 8
- /* HiddenGcRef32 support (on 64-bits only) */
- typedef unsigned int hiddengcref32_t;
-# define uint32_t hiddengcref32_t
-# define OP_SHOW_FROM_ADR32(x, r) r = (void*)(((unsigned long)(x)) << 3)
-# define OP_HIDE_INTO_ADR32(x, r) \
- r = (hiddengcref32_t)(((unsigned long)(x)) >> 3); \
- RPyAssert((void*)(((unsigned long)(r)) << 3) == (x), \
- "pointer too big or misaligned!")
-
#endif
/********************************************************/
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/src/g_prerequisite.h
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/src/g_prerequisite.h (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/src/g_prerequisite.h Tue Oct 5 16:28:52 2010
@@ -29,3 +29,8 @@
#include "src/align.h"
+
+
+#if PYPY_LONG_BIT == 64
+# include "src/hiddengcref32.h"
+#endif
Added: pypy/branch/32ptr-on-64bit/pypy/translator/c/src/hiddengcref32.h
==============================================================================
--- (empty file)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/src/hiddengcref32.h Tue Oct 5 16:28:52 2010
@@ -0,0 +1,29 @@
+
+/******************** 64-bit hiddengcref32 support ********************/
+
+typedef unsigned int hiddengcref32_t;
+
+
+void RPyPointerTooBig(void);
+
+#ifndef PYPY_NOT_MAIN_FILE
+void RPyPointerTooBig(void) {
+ fprintf(stderr, "Fatal error: Pointer too big or misaligned. "
+ "This can occur if your C\n"
+ "compiler puts static data after the first 32GB "
+ "of virtual address space.\n");
+ abort();
+}
+#endif
+
+
+#define OP_SHOW_FROM_ADR32(x, r) r = (void*)(((unsigned long)(x)) << 3)
+
+#define OP_HIDE_INTO_ADR32_CHECK(x, r) \
+ r = (hiddengcref32_t)(((unsigned long)(x)) >> 3); \
+ if ((void*)(((unsigned long)(r)) << 3) != (x)) \
+ RPyPointerTooBig()
+
+#define OP_HIDE_INTO_ADR32(x, r) \
+ RPyAssert(!(((long)(x)) & 0x7FFFFFFF8), "Pointer too big or misaligned"); \
+ r = (hiddengcref32_t)(((unsigned long)(x)) >> 3)
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_boehm.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_boehm.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_boehm.py Tue Oct 5 16:28:52 2010
@@ -469,3 +469,41 @@
assert res[2] != compute_hash(c) # likely
assert res[3] == compute_hash(d)
assert res[4] == compute_hash(("Hi", None, (7.5, 2, d)))
+
+ def test_hiddengcref32(self):
+ from pypy.config.translationoption import IS_64_BITS
+ if not IS_64_BITS:
+ py.test.skip("only for 64-bits")
+ py.test.skip("not a stand-alone executable: the static data is "
+ "after the limit of 32GB")
+ #
+ HiddenGcRef32 = llmemory.HiddenGcRef32
+ S = lltype.GcStruct('S', ('x', lltype.Signed),
+ ('y', HiddenGcRef32))
+ prebuilt = lltype.malloc(S, immortal=True)
+ prebuilt2 = lltype.malloc(S, immortal=True)
+ prebuilt.x = 42
+ prebuilt2.x = 53
+ prebuilt.y = lltype.cast_opaque_ptr(HiddenGcRef32, prebuilt2)
+ prebuilt2.y = lltype.cast_opaque_ptr(HiddenGcRef32, prebuilt)
+
+ def check(a, b):
+ p = lltype.cast_opaque_ptr(lltype.Ptr(S), a.y)
+ assert p == b
+
+ def fn():
+ assert prebuilt.x == 42
+ assert prebuilt2.x == 53
+ check(prebuilt, prebuilt2)
+ check(prebuilt2, prebuilt)
+ p = lltype.malloc(S)
+ p.y = lltype.cast_opaque_ptr(HiddenGcRef32, prebuilt)
+ prebuilt2.y = lltype.cast_opaque_ptr(HiddenGcRef32, p)
+ check(p, prebuilt)
+ check(prebuilt2, p)
+ check(prebuilt, prebuilt2)
+ return 0
+
+ f = self.getcompiled(fn)
+ res = fn()
+ assert res == 0
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_lladdresses.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_lladdresses.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/test/test_lladdresses.py Tue Oct 5 16:28:52 2010
@@ -232,43 +232,3 @@
assert res == 456
res = fc(77)
assert res == 123
-
-
-##class TestHiddenGcRef32(StandaloneTests):
-
-## def setup_class(cls):
-## from pypy.config.translationoption import IS_64_BITS
-## if 0:# not IS_64_BITS:
-## py.test.skip("only for 64-bits")
-
-## def test_hiddengcref32(self):
-## from pypy.rpython.lltypesystem.lloperation import llop
-## S = lltype.GcStruct('S', ('x', lltype.Signed),
-## ('y', HiddenGcRef32))
-## prebuilt = lltype.malloc(S, immortal=True)
-## prebuilt2 = lltype.malloc(S, immortal=True)
-## prebuilt.x = 42
-## prebuilt2.x = 53
-## prebuilt.y = llop.hide_into_adr32(HiddenGcRef32, prebuilt2)
-## prebuilt2.y = llop.hide_into_adr32(HiddenGcRef32, prebuilt)
-
-## def check(a, b):
-## p = llop.show_from_adr32(lltype.Ptr(S), a.y)
-## assert p == b
-
-## def f(argv):
-## assert prebuilt.x == 42
-## assert prebuilt2.x == 53
-## check(prebuilt, prebuilt2)
-## check(prebuilt2, prebuilt)
-## p = lltype.malloc(S)
-## p.y = llop.hide_into_adr32(HiddenGcRef32, prebuilt)
-## prebuilt2.y = llop.hide_into_adr32(HiddenGcRef32, p)
-## check(p, prebuilt)
-## check(prebuilt2, p)
-## check(prebuilt, prebuilt2)
-## return 0
-
-## fc = self.compile(f)
-## res = fc([])
-## assert res == 0
From arigo at codespeak.net Tue Oct 5 16:34:18 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 16:34:18 +0200 (CEST)
Subject: [pypy-svn] r77610 -
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc
Message-ID: <20101005143418.112C8282B90@codespeak.net>
Author: arigo
Date: Tue Oct 5 16:34:16 2010
New Revision: 77610
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py
Log:
Fix test_gc.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/minimark.py Tue Oct 5 16:34:16 2010
@@ -516,7 +516,8 @@
#
# We get here for large fixed-size objects. Complain if
# self.config.compressptr is set.
- if self.config.compressptr and not self.is_varsize(typeid):
+ if (self.translated_to_c and self.config.compressptr
+ and not self.is_varsize(typeid)):
raise FixedSizeObjectTooLarge
#
else:
From arigo at codespeak.net Tue Oct 5 16:46:41 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 16:46:41 +0200 (CEST)
Subject: [pypy-svn] r77611 - in pypy/branch/32ptr-on-64bit/pypy/rpython:
lltypesystem memory/gctransform memory/test
Message-ID: <20101005144641.CDC2C282B90@codespeak.net>
Author: arigo
Date: Tue Oct 5 16:46:40 2010
New Revision: 77611
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_transformed_gc.py
Log:
Whack whack whack until test_transformed_gc starts to run.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py Tue Oct 5 16:46:40 2010
@@ -5,10 +5,11 @@
# sizeof, offsetof
import weakref
-from pypy.rlib.objectmodel import Symbolic
+from pypy.rlib.objectmodel import Symbolic, we_are_translated
+from pypy.rlib.objectmodel import running_on_llinterp
+from pypy.rlib.debug import llinterpcall
from pypy.rpython.lltypesystem import lltype
from pypy.tool.uid import uid
-from pypy.rlib.objectmodel import we_are_translated
class AddressOffset(Symbolic):
@@ -580,9 +581,11 @@
" from a HiddenGcRef32")
def has_odd_value_marker(addressofs):
- if we_are_translated():
+ if not we_are_translated():
+ return _has_odd_value_marker(addressofs)
+ if not running_on_llinterp:
return bool(addressofs & 1)
- return _has_odd_value_marker(addressofs)
+ return llinterpcall(lltype.Bool, _has_odd_value_marker, addressofs)
def _has_odd_value_marker(addressofs):
while isinstance(addressofs, CompositeOffset):
@@ -590,9 +593,11 @@
return isinstance(addressofs, OddValueMarker)
def remove_odd_value_marker(addressofs):
- if we_are_translated():
+ if not we_are_translated():
+ return _remove_odd_value_marker(addressofs)
+ if not running_on_llinterp:
return addressofs - 1
- return _remove_odd_value_marker(addressofs)
+ return llinterpcall(lltype.Signed, _remove_odd_value_marker, addressofs)
def _remove_odd_value_marker(addressofs):
if isinstance(addressofs, CompositeOffset):
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py Tue Oct 5 16:46:40 2010
@@ -10,6 +10,7 @@
from pypy.rlib import rstack, rgc
from pypy.rlib.debug import ll_assert
from pypy.translator.backendopt import graphanalyze
+from pypy.translator.backendopt.support import var_needsgc
from pypy.annotation import model as annmodel
from pypy.objspace.flow.model import Constant
from pypy.rpython import annlowlevel
@@ -68,7 +69,9 @@
mallocvars[op.result] = True
elif op.opname in ("setfield", "setarrayitem", "setinteriorfield"):
TYPE = op.args[-1].concretetype
- if op.args[0] in mallocvars and is_gc_pointer_or_hidden(TYPE):
+ if (op.args[0] in mallocvars and
+ isinstance(TYPE, lltype.Ptr) and
+ TYPE.TO._gckind == "gc"):
result.add(op)
else:
if collect_analyzer.analyze(op):
@@ -1014,7 +1017,7 @@
v_struct = hop.spaceop.args[0]
v_newvalue = hop.spaceop.args[-1]
assert opname in ('setfield', 'setarrayitem', 'setinteriorfield')
- assert is_gc_pointer_or_hidden(v_newvalue.concretetype)
+ assert var_needsgc(v_newvalue)
if (self.write_barrier_ptr is not None
and v_struct.concretetype.TO._gckind == "gc"
and hop.spaceop not in self.clean_sets):
@@ -1056,12 +1059,12 @@
def _fetch_unpacked_pointer(self, hop, v_value):
# optimization for the common case where this setfield is preceded
- # by hide_into_adr32()
+ # by v_value = cast_opaque_ptr(v_normal_pointer)
for op in hop.llops[::-1]:
- if op.opname == 'hide_into_adr32' and op.result == v_value:
+ if op.opname == 'cast_opaque_ptr' and op.result == v_value:
return op.args[0]
else:
- return hop.genop("show_from_adr32", [v_value],
+ return hop.genop("cast_opaque_ptr", [v_value],
resulttype = llmemory.Address)
def transform_getfield_typeptr(self, hop):
@@ -1107,7 +1110,7 @@
GCTransformer.gct_setfield(self, hop)
def var_needs_set_transform(self, var):
- return is_gc_pointer_or_hidden(var.concretetype)
+ return var_needsgc(var)
def push_alive_nopyobj(self, var, llops):
pass
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_transformed_gc.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_transformed_gc.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_transformed_gc.py Tue Oct 5 16:46:40 2010
@@ -49,7 +49,8 @@
GC_CAN_MOVE = False
GC_CAN_MALLOC_NONMOVABLE = True
taggedpointers = False
-
+ compressptr = False
+
def setup_class(cls):
funcs0 = []
funcs2 = []
@@ -105,7 +106,8 @@
s_args = annmodel.SomePtr(lltype.Ptr(ARGS))
t = rtype(entrypoint, [s_args], gcname=cls.gcname,
stacklessgc=cls.stacklessgc,
- taggedpointers=cls.taggedpointers)
+ taggedpointers=cls.taggedpointers,
+ compressptr=cls.compressptr)
for fixup in mixlevelstuff:
if fixup:
@@ -1513,6 +1515,15 @@
res = run([])
assert res == 123
+class TestMiniMarkGCCompressPtr(TestMiniMarkGC):
+ compressptr = True
+
+ def setup_class(cls):
+ from pypy.config.translationoption import IS_64_BITS
+ if not IS_64_BITS:
+ py.test.skip("only for 64-bits")
+ TestMiniMarkGC.setup_class.im_func(cls)
+
# ________________________________________________________________
# tagged pointers
From afa at codespeak.net Tue Oct 5 18:06:39 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 5 Oct 2010 18:06:39 +0200 (CEST)
Subject: [pypy-svn] r77613 - pypy/branch/fast-forward/pypy/doc/config
Message-ID: <20101005160639.603C6282B90@codespeak.net>
Author: afa
Date: Tue Oct 5 18:06:37 2010
New Revision: 77613
Modified:
pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._multiprocessing.txt
Log:
typo
Modified: pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._multiprocessing.txt
==============================================================================
--- pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._multiprocessing.txt (original)
+++ pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._multiprocessing.txt Tue Oct 5 18:06:37 2010
@@ -1,2 +1,2 @@
Use the '_multiprocessing' module.
-Used by the 'multiprocessing standard lib module. This module is expected to be working and is included by default.
+Used by the 'multiprocessing' standard lib module. This module is expected to be working and is included by default.
From arigo at codespeak.net Tue Oct 5 18:23:15 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 18:23:15 +0200 (CEST)
Subject: [pypy-svn] r77614 - in pypy/branch/32ptr-on-64bit/pypy/rpython: .
lltypesystem lltypesystem/test memory/gc memory/gctransform
Message-ID: <20101005162315.B9DAF282B9C@codespeak.net>
Author: arigo
Date: Tue Oct 5 18:23:13 2010
New Revision: 77614
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/llinterp.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/opimpl.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/test/test_llmemory.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
pypy/branch/32ptr-on-64bit/pypy/rpython/rtyper.py
Log:
Intermediate check-in.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/llinterp.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/llinterp.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/llinterp.py Tue Oct 5 18:23:13 2010
@@ -1025,14 +1025,22 @@
def op_raw_load(self, addr, typ, offset):
checkadr(addr)
- value = getattr(addr, str(typ).lower())[offset]
+ if typ == llmemory.HiddenGcRef32:
+ name = 'hiddengcref32'
+ else:
+ name = str(typ).lower()
+ value = getattr(addr, name)[offset]
assert lltype.typeOf(value) == typ
return value
def op_raw_store(self, addr, typ, offset, value):
checkadr(addr)
+ if typ == llmemory.HiddenGcRef32:
+ name = 'hiddengcref32'
+ else:
+ name = str(typ).lower()
assert lltype.typeOf(value) == typ
- getattr(addr, str(typ).lower())[offset] = value
+ getattr(addr, name)[offset] = value
def op_stack_malloc(self, size): # mmh
raise NotImplementedError("backend only")
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py Tue Oct 5 18:23:13 2010
@@ -669,22 +669,22 @@
raise TypeError(TARGETTYPE)
ptr[0] = value
-class _address32_fakeaccessor(_address_fakeaccessor):
- pass
+class _hiddengcref32_fakeaccessor(_fakeaccessor):
+ TYPE = HiddenGcRef32
supported_access_types = {"signed": lltype.Signed,
"unsigned": lltype.Unsigned,
"char": lltype.Char,
"address": Address,
"float": lltype.Float,
- "address32": Address,
+ "hiddengcref32": HiddenGcRef32,
}
fakeaddress.signed = property(_signed_fakeaccessor)
fakeaddress.float = property(_float_fakeaccessor)
fakeaddress.char = property(_char_fakeaccessor)
fakeaddress.address = property(_address_fakeaccessor)
-fakeaddress.address32 = property(_address32_fakeaccessor)
+fakeaddress.hiddengcref32 = property(_hiddengcref32_fakeaccessor)
fakeaddress._TYPE = Address
# the obtained address will not keep the object alive. e.g. if the object is
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py Tue Oct 5 18:23:13 2010
@@ -426,6 +426,9 @@
'gc_gettypeptr_group': LLOp(canfold=True),
'get_member_index': LLOp(canfold=True),
+ 'hide_into_adr32': LLOp(canrun=True),
+ 'show_from_adr32': LLOp(canrun=True),
+
# __________ used by the JIT ________
'jit_marker': LLOp(),
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/opimpl.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/opimpl.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/opimpl.py Tue Oct 5 18:23:13 2010
@@ -526,16 +526,21 @@
def op_shrink_array(array, smallersize):
return False
-def op_hide_into_adr32(adr):
- if lltype.typeOf(adr) != llmemory.Address:
- adr = llmemory.cast_ptr_to_adr(adr)
- return llmemory._hiddengcref32(adr)
+def op_hide_into_adr32(ptr):
+ if lltype.typeOf(ptr) == llmemory.Address:
+ if not ptr:
+ return lltype.nullptr(llmemory.HiddenGcRef32.TO)
+ ptr = ptr.ptr
+ return lltype.cast_opaque_ptr(llmemory.HiddenGcRef32, ptr)
-def op_show_from_adr32(RESTYPE, adr32):
+def op_show_from_adr32(RESTYPE, ptr32):
if RESTYPE == llmemory.Address:
- return adr32.adr64
- else:
- return llmemory.cast_adr_to_ptr(adr32.adr64, RESTYPE)
+ if not ptr32:
+ return llmemory.NULL
+ PTRTYPE = lltype.Ptr(ptr32._obj.container._TYPE)
+ ptr = lltype.cast_opaque_ptr(PTRTYPE, ptr32)
+ return llmemory.cast_ptr_to_adr(ptr)
+ return lltype.cast_opaque_ptr(RESTYPE, ptr32)
op_show_from_adr32.need_result_type = True
# ____________________________________________________________
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py Tue Oct 5 18:23:13 2010
@@ -3,6 +3,7 @@
from pypy.config.translationoption import IS_64_BITS
from pypy.rpython.rmodel import Repr, inputconst
from pypy.rpython.lltypesystem import lltype, llmemory, rffi
+from pypy.rpython.lltypesystem.lloperation import llop
from pypy.rpython.error import TyperError
@@ -49,7 +50,7 @@
ptr = self.baserepr.convert_const(value)
T = lltype.typeOf(ptr)
assert T == self.BASETYPE
- return lltype.cast_opaque_ptr(self.lowleveltype, ptr)
+ return llop.hide_into_adr32(self.lowleveltype, ptr)
def get_ll_eq_function(self):
if self.baserepr.get_ll_eq_function() is not None:
@@ -62,7 +63,7 @@
BASETYPE = self.BASETYPE
#
def ll_hiddengcref32_hash(x):
- x = lltype.cast_opaque_ptr(BASETYPE, x)
+ x = llop.show_from_adr32(BASETYPE, x)
return basefunc(x)
#
self.ll_hash_function = ll_hiddengcref32_hash
@@ -76,7 +77,7 @@
BASETYPE = self.BASETYPE
#
def ll_hiddengcref32_fasthash(x):
- x = lltype.cast_opaque_ptr(BASETYPE, x)
+ x = llop.show_from_adr32(BASETYPE, x)
return basefunc(x)
#
self.ll_fasthash_function = ll_hiddengcref32_hash
@@ -97,12 +98,12 @@
def convert_from_to((r_from, r_to), v, llops):
assert r_from.lowleveltype.TO._gckind == 'gc'
assert not isinstance(r_from.lowleveltype.TO, lltype.GcOpaqueType)
- return llops.genop('cast_opaque_ptr', [v],
+ return llops.genop('hide_into_adr32', [v],
resulttype=llmemory.HiddenGcRef32)
class __extend__(pairtype(CompressedGcRefRepr, Repr)):
def convert_from_to((r_from, r_to), v, llops):
assert r_to.lowleveltype.TO._gckind == 'gc'
assert not isinstance(r_to.lowleveltype.TO, lltype.GcOpaqueType)
- return llops.genop('cast_opaque_ptr', [v],
+ return llops.genop('show_from_adr32', [v],
resulttype=r_to.lowleveltype)
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/test/test_llmemory.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/test/test_llmemory.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/test/test_llmemory.py Tue Oct 5 18:23:13 2010
@@ -650,8 +650,10 @@
from pypy.config.translationoption import IS_64_BITS
if not IS_64_BITS:
py.test.skip("only on 64-bits")
+ #
+ from pypy.rpython.lltypesystem.lloperation import llop
S = lltype.GcStruct('S')
p = lltype.malloc(S)
- q = lltype.cast_opaque_ptr(HiddenGcRef32, p)
- r = lltype.cast_opaque_ptr(lltype.Ptr(S), q)
+ q = llop.hide_into_adr32(HiddenGcRef32, p)
+ r = llop.show_from_adr32(lltype.Ptr(S), q)
assert r == p
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py Tue Oct 5 18:23:13 2010
@@ -1,4 +1,5 @@
from pypy.rpython.lltypesystem import lltype, llmemory, llarena
+from pypy.rpython.lltypesystem.lloperation import llop
from pypy.rlib.debug import ll_assert
from pypy.rpython.memory.gcheader import GCHeaderBuilder
from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
@@ -211,11 +212,13 @@
# special handling of HiddenGcRef32 on 64-bit platforms
ofs = llmemory.remove_odd_value_marker(ofs)
item = obj + ofs
- address = item.address32[0]
+ address = llop.show_from_adr32(
+ llmemory.Address, item.hiddengcref32[0])
if self.is_valid_gc_object(address):
newaddr = callback(address, arg)
if newaddr is not None:
- item.address32[0] = newaddr
+ item.hiddengcref32[0] = llop.hide_into_adr32(
+ llmemory.HiddenGcRef32, newaddr)
else:
# common case
item = obj + ofs
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py Tue Oct 5 18:23:13 2010
@@ -1059,12 +1059,12 @@
def _fetch_unpacked_pointer(self, hop, v_value):
# optimization for the common case where this setfield is preceded
- # by v_value = cast_opaque_ptr(v_normal_pointer)
+ # by v_value = hide_into_adr32(v_normal_pointer)
for op in hop.llops[::-1]:
- if op.opname == 'cast_opaque_ptr' and op.result == v_value:
+ if op.opname == 'hide_into_adr32' and op.result == v_value:
return op.args[0]
else:
- return hop.genop("cast_opaque_ptr", [v_value],
+ return hop.genop("show_from_adr32", [v_value],
resulttype = llmemory.Address)
def transform_getfield_typeptr(self, hop):
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/rtyper.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/rtyper.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/rtyper.py Tue Oct 5 18:23:13 2010
@@ -100,7 +100,8 @@
return self.primitive_to_repr[lltype]
except KeyError:
pass
- if isinstance(lltype, Primitive):
+ from pypy.rpython.lltypesystem import llmemory
+ if isinstance(lltype, Primitive) or lltype == llmemory.HiddenGcRef32:
repr = self.primitive_to_repr[lltype] = self.getrepr(annmodel.lltype_to_annotation(lltype))
return repr
raise TyperError('There is no primitive repr for %r'%(lltype,))
From arigo at codespeak.net Tue Oct 5 18:26:52 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 18:26:52 +0200 (CEST)
Subject: [pypy-svn] r77615 - in pypy/branch/32ptr-on-64bit/pypy/rpython: .
lltypesystem memory/gc
Message-ID: <20101005162652.4B84F282B9C@codespeak.net>
Author: arigo
Date: Tue Oct 5 18:26:50 2010
New Revision: 77615
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/llinterp.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py
pypy/branch/32ptr-on-64bit/pypy/rpython/rtyper.py
Log:
Figured that I don't need "addr.hiddengcref32[0]" after all.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/llinterp.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/llinterp.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/llinterp.py Tue Oct 5 18:26:50 2010
@@ -1025,22 +1025,14 @@
def op_raw_load(self, addr, typ, offset):
checkadr(addr)
- if typ == llmemory.HiddenGcRef32:
- name = 'hiddengcref32'
- else:
- name = str(typ).lower()
- value = getattr(addr, name)[offset]
+ value = getattr(addr, str(typ).lower())[offset]
assert lltype.typeOf(value) == typ
return value
def op_raw_store(self, addr, typ, offset, value):
checkadr(addr)
- if typ == llmemory.HiddenGcRef32:
- name = 'hiddengcref32'
- else:
- name = str(typ).lower()
assert lltype.typeOf(value) == typ
- getattr(addr, name)[offset] = value
+ getattr(addr, str(typ).lower())[offset] = value
def op_stack_malloc(self, size): # mmh
raise NotImplementedError("backend only")
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/llmemory.py Tue Oct 5 18:26:50 2010
@@ -669,22 +669,17 @@
raise TypeError(TARGETTYPE)
ptr[0] = value
-class _hiddengcref32_fakeaccessor(_fakeaccessor):
- TYPE = HiddenGcRef32
-
supported_access_types = {"signed": lltype.Signed,
"unsigned": lltype.Unsigned,
"char": lltype.Char,
"address": Address,
"float": lltype.Float,
- "hiddengcref32": HiddenGcRef32,
}
fakeaddress.signed = property(_signed_fakeaccessor)
fakeaddress.float = property(_float_fakeaccessor)
fakeaddress.char = property(_char_fakeaccessor)
fakeaddress.address = property(_address_fakeaccessor)
-fakeaddress.hiddengcref32 = property(_hiddengcref32_fakeaccessor)
fakeaddress._TYPE = Address
# the obtained address will not keep the object alive. e.g. if the object is
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py Tue Oct 5 18:26:50 2010
@@ -13,6 +13,8 @@
('links', lltype.Array(lltype.Signed)))
ARRAY_TYPEID_MAP = lltype.GcArray(lltype.Ptr(TYPEID_MAP))
+HIDDENGCREFFIELD = lltype.FixedSizeArray(llmemory.HiddenGcRef32, 1)
+
class GCBase(object):
_alloc_flavor_ = "raw"
moving_gc = False
@@ -212,13 +214,13 @@
# special handling of HiddenGcRef32 on 64-bit platforms
ofs = llmemory.remove_odd_value_marker(ofs)
item = obj + ofs
- address = llop.show_from_adr32(
- llmemory.Address, item.hiddengcref32[0])
+ item = llmemory.cast_adr_to_ptr(item, lltype.Ptr(HIDDENGCREFFIELD))
+ address = llop.show_from_adr32(llmemory.Address, item[0])
if self.is_valid_gc_object(address):
newaddr = callback(address, arg)
if newaddr is not None:
- item.hiddengcref32[0] = llop.hide_into_adr32(
- llmemory.HiddenGcRef32, newaddr)
+ item[0] = llop.hide_into_adr32(llmemory.HiddenGcRef32,
+ newaddr)
else:
# common case
item = obj + ofs
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/rtyper.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/rtyper.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/rtyper.py Tue Oct 5 18:26:50 2010
@@ -100,8 +100,7 @@
return self.primitive_to_repr[lltype]
except KeyError:
pass
- from pypy.rpython.lltypesystem import llmemory
- if isinstance(lltype, Primitive) or lltype == llmemory.HiddenGcRef32:
+ if isinstance(lltype, Primitive):
repr = self.primitive_to_repr[lltype] = self.getrepr(annmodel.lltype_to_annotation(lltype))
return repr
raise TyperError('There is no primitive repr for %r'%(lltype,))
From afa at codespeak.net Tue Oct 5 18:36:39 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 5 Oct 2010 18:36:39 +0200 (CEST)
Subject: [pypy-svn] r77616 -
pypy/branch/fast-forward/pypy/interpreter/pyparser
Message-ID: <20101005163639.9DF4B282B9C@codespeak.net>
Author: afa
Date: Tue Oct 5 18:36:38 2010
New Revision: 77616
Added:
pypy/branch/fast-forward/pypy/interpreter/pyparser/autopath.py
- copied unchanged from r77594, pypy/branch/fast-forward/pypy/bin/autopath.py
pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py (contents, props changed)
pypy/branch/fast-forward/pypy/interpreter/pyparser/pylexer.py (contents, props changed)
Modified:
pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py
Log:
I finally found how pytokenize.py is generated.
Tweak the generator until the result looks similar enough to the previous arrays.
the only notable difference is that state #2 and #3 have been swapped.
Added: pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py Tue Oct 5 18:36:38 2010
@@ -0,0 +1,312 @@
+#! /usr/bin/env python
+"""Module genPytokenize
+
+Generates finite state automata for recognizing Python tokens. These are hand
+coded versions of the regular expressions originally appearing in Ping's
+tokenize module in the Python standard library.
+
+When run from the command line, this should pretty print the DFA machinery.
+
+$Id: genPytokenize.py,v 1.1 2003/10/02 17:37:17 jriehl Exp $
+"""
+
+import autopath
+from pypy.interpreter.pyparser.pylexer import *
+from pypy.interpreter.pyparser.automata import NonGreedyDFA, DFA, DEFAULT
+
+def makePyPseudoDFA ():
+ import string
+ states = []
+ # ____________________________________________________________
+ def makeLineCont ():
+ return chain(states,
+ newArcPair(states, "\\"),
+ maybe(states, newArcPair(states, "\r")),
+ newArcPair(states, "\n"))
+ # ____________________________________________________________
+ # Ignore stuff
+ def makeWhitespace ():
+ return any(states, groupStr(states, " \f\t"))
+ # ____________________________________________________________
+ def makeComment ():
+ return chain(states,
+ newArcPair(states, "#"),
+ any(states, notGroupStr(states, "\r\n")))
+ # ____________________________________________________________
+ #ignore = chain(states,
+ # makeWhitespace(),
+ # any(states, chain(states,
+ # makeLineCont(),
+ # makeWhitespace())),
+ # maybe(states, makeComment()))
+ # ____________________________________________________________
+ # Names
+ name = chain(states,
+ groupStr(states, string.letters + "_"),
+ any(states, groupStr(states,
+ string.letters + string.digits + "_")))
+ # ____________________________________________________________
+ # Digits
+ def makeDigits ():
+ return groupStr(states, "0123456789")
+ # ____________________________________________________________
+ # Integer numbers
+ hexNumber = chain(states,
+ newArcPair(states, "0"),
+ groupStr(states, "xX"),
+ any(states, groupStr(states, "0123456789abcdefABCDEF")),
+ maybe(states, groupStr(states, "lL")))
+ octNumber = chain(states,
+ newArcPair(states, "0"),
+ any(states, groupStr(states, "01234567")),
+ maybe(states, groupStr(states, "lL")))
+ decNumber = chain(states,
+ groupStr(states, "123456789"),
+ any(states, makeDigits()),
+ maybe(states, groupStr(states, "lL")))
+ intNumber = group(states, hexNumber, octNumber, decNumber)
+ # ____________________________________________________________
+ # Exponents
+ def makeExp ():
+ return chain(states,
+ groupStr(states, "eE"),
+ maybe(states, groupStr(states, "+-")),
+ atleastonce(states, makeDigits()))
+ # ____________________________________________________________
+ # Floating point numbers
+ def makeFloat ():
+ pointFloat = chain(states,
+ group(states,
+ chain(states,
+ atleastonce(states, makeDigits()),
+ newArcPair(states, "."),
+ any(states, makeDigits())),
+ chain(states,
+ newArcPair(states, "."),
+ atleastonce(states, makeDigits()))),
+ maybe(states, makeExp()))
+ expFloat = chain(states,
+ atleastonce(states, makeDigits()),
+ makeExp())
+ return group(states, pointFloat, expFloat)
+ # ____________________________________________________________
+ # Imaginary numbers
+ imagNumber = group(states,
+ chain(states,
+ atleastonce(states, makeDigits()),
+ groupStr(states, "jJ")),
+ chain(states,
+ makeFloat(),
+ groupStr(states, "jJ")))
+ # ____________________________________________________________
+ # Any old number.
+ number = group(states, imagNumber, makeFloat(), intNumber)
+ # ____________________________________________________________
+ # Funny
+ operator = group(states,
+ chain(states,
+ chainStr(states, "**"),
+ maybe(states, newArcPair(states, "="))),
+ chain(states,
+ chainStr(states, ">>"),
+ maybe(states, newArcPair(states, "="))),
+ chain(states,
+ chainStr(states, "<<"),
+ maybe(states, newArcPair(states, "="))),
+ chainStr(states, "<>"),
+ chainStr(states, "!="),
+ chain(states,
+ chainStr(states, "//"),
+ maybe(states, newArcPair(states, "="))),
+ chain(states,
+ groupStr(states, "+-*/%&|^=<>"),
+ maybe(states, newArcPair(states, "="))),
+ newArcPair(states, "~"))
+ bracket = groupStr(states, "[](){}")
+ special = group(states,
+ chain(states,
+ maybe(states, newArcPair(states, "\r")),
+ newArcPair(states, "\n")),
+ groupStr(states, "@:;.,`"))
+ funny = group(states, operator, bracket, special)
+ # ____________________________________________________________
+ def makeStrPrefix ():
+ return chain(states,
+ maybe(states, groupStr(states, "uUbB")),
+ maybe(states, groupStr(states, "rR")))
+ # ____________________________________________________________
+ contStr = group(states,
+ chain(states,
+ makeStrPrefix(),
+ newArcPair(states, "'"),
+ any(states,
+ notGroupStr(states, "\n'\\")),
+ any(states,
+ chain(states,
+ newArcPair(states, "\\"),
+ newArcPair(states, DEFAULT),
+ any(states,
+ notGroupStr(states, "\n'\\")))),
+ group(states,
+ newArcPair(states, "'"),
+ makeLineCont())),
+ chain(states,
+ makeStrPrefix(),
+ newArcPair(states, '"'),
+ any(states,
+ notGroupStr(states, '\n"\\')),
+ any(states,
+ chain(states,
+ newArcPair(states, "\\"),
+ newArcPair(states, DEFAULT),
+ any(states,
+ notGroupStr(states, '\n"\\')))),
+ group(states,
+ newArcPair(states, '"'),
+ makeLineCont())))
+ triple = group(states,
+ makeStrPrefix(),
+ group(states,
+ chainStr(states, "'''"),
+ chainStr(states, '"""')))
+ pseudoExtras = group(states,
+ makeLineCont(),
+ makeComment(),
+ triple)
+ pseudoToken = chain(states,
+ makeWhitespace(),
+ group(states,
+ pseudoExtras, number, funny, contStr, name))
+ dfaStates, dfaAccepts = nfaToDfa(states, *pseudoToken)
+ return DFA(dfaStates, dfaAccepts)
+
+# ______________________________________________________________________
+
+def makePyEndDFAMap ():
+ states = []
+ single = chain(states,
+ any(states, notGroupStr(states, "'\\")),
+ any(states,
+ chain(states,
+ newArcPair(states, "\\"),
+ newArcPair(states, DEFAULT),
+ any(states, notGroupStr(states, "'\\")))),
+ newArcPair(states, "'"))
+ singleDFA = DFA(*nfaToDfa(states, *single))
+ states = []
+ double = chain(states,
+ any(states, notGroupStr(states, '"\\')),
+ any(states,
+ chain(states,
+ newArcPair(states, "\\"),
+ newArcPair(states, DEFAULT),
+ any(states, notGroupStr(states, '"\\')))),
+ newArcPair(states, '"'))
+ doubleDFA = DFA(*nfaToDfa(states, *double))
+ states = []
+ single3 = chain(states,
+ any(states, notGroupStr(states, "'\\")),
+ any(states,
+ chain(states,
+ group(states,
+ chain(states,
+ newArcPair(states, "\\"),
+ newArcPair(states, DEFAULT)),
+ chain(states,
+ newArcPair(states, "'"),
+ notChainStr(states, "''"))),
+ any(states, notGroupStr(states, "'\\")))),
+ chainStr(states, "'''"))
+ single3DFA = NonGreedyDFA(*nfaToDfa(states, *single3))
+ states = []
+ double3 = chain(states,
+ any(states, notGroupStr(states, '"\\')),
+ any(states,
+ chain(states,
+ group(states,
+ chain(states,
+ newArcPair(states, "\\"),
+ newArcPair(states, DEFAULT)),
+ chain(states,
+ newArcPair(states, '"'),
+ notChainStr(states, '""'))),
+ any(states, notGroupStr(states, '"\\')))),
+ chainStr(states, '"""'))
+ double3DFA = NonGreedyDFA(*nfaToDfa(states, *double3))
+ map = {"'" : singleDFA,
+ '"' : doubleDFA,
+ "r" : None,
+ "R" : None,
+ "u" : None,
+ "U" : None,
+ "b" : None,
+ "B" : None}
+ for uniPrefix in ("", "u", "U", "b", "B", ):
+ for rawPrefix in ("", "r", "R"):
+ prefix = uniPrefix + rawPrefix
+ map[prefix + "'''"] = single3DFA
+ map[prefix + '"""'] = double3DFA
+ return map
+
+# ______________________________________________________________________
+
+def output(name, dfa_class, dfa):
+ import textwrap
+ i = 0
+ for line in textwrap.wrap(repr(dfa.accepts), width = 50):
+ if i == 0:
+ print "accepts =", line
+ else:
+ print " ", line
+ i += 1
+ import StringIO
+ print "states = ["
+ for state in dfa.states:
+ s = StringIO.StringIO()
+ i = 0
+ for k, v in sorted(state.items()):
+ i += 1
+ if k == '\x00default':
+ k = "automata.DEFAULT"
+ else:
+ k = repr(k)
+ s.write(k)
+ s.write('::')
+ s.write(repr(v))
+ if i < len(state):
+ s.write(', ')
+ s.write('},')
+ i = 0
+ for line in textwrap.wrap(s.getvalue(), width=35):
+ line = line.replace('::', ': ')
+ if i == 0:
+ print ' {' + line
+ else:
+ print ' ' + line
+ i += 1
+ print
+ print " ]"
+ print "%s = automata.%s(states, accepts)" % (name, dfa_class)
+ print
+
+def main ():
+ pseudoDFA = makePyPseudoDFA()
+ output("pseudoDFA", "DFA", pseudoDFA)
+ endDFAMap = makePyEndDFAMap()
+ output("double3DFA", "NonGreedyDFA", endDFAMap['"""'])
+ output("single3DFA", "NonGreedyDFA", endDFAMap["'''"])
+ output("doubleDFA", "DFA", endDFAMap['"'])
+ output("singleDFA", "DFA", endDFAMap["'"])
+ print "endDFAs = {\"'\" : singleDFA,"
+ print " '\"' : doubleDFA,"
+ print " 'r' : None,"
+ print " 'R' : None,"
+ print " 'u' : None,"
+ print " 'U' : None,"
+ print " 'b' : None,"
+ print " 'B' : None}"
+
+# ______________________________________________________________________
+
+if __name__ == "__main__":
+ main()
Added: pypy/branch/fast-forward/pypy/interpreter/pyparser/pylexer.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/pylexer.py Tue Oct 5 18:36:38 2010
@@ -0,0 +1,240 @@
+# Used by genpytokenize.py to generate the parser in pytokenize.py
+from pypy.interpreter.pyparser.automata import DFA, DEFAULT
+
+class EMPTY: pass
+
+def newArcPair (states, transitionLabel):
+ s1Index = len(states)
+ s2Index = s1Index + 1
+ states.append([(transitionLabel, s2Index)])
+ states.append([])
+ return s1Index, s2Index
+
+# ______________________________________________________________________
+
+def chain (states, *stateIndexPairs):
+ if len(stateIndexPairs) > 1:
+ start, lastFinish = stateIndexPairs[0]
+ for nStart, nFinish in stateIndexPairs[1:]:
+ states[lastFinish].append((EMPTY, nStart))
+ lastFinish = nFinish
+ return start, nFinish
+ else:
+ return stateIndexPairs[0]
+
+
+# ______________________________________________________________________
+
+def chainStr (states, str):
+ return chain(states, *map(lambda x : newArcPair(states, x), str))
+
+# ______________________________________________________________________
+
+def notChainStr (states, str):
+ """XXX I'm not sure this is how it should be done, but I'm going to
+ try it anyway. Note that for this case, I require only single character
+ arcs, since I would have to basically invert all accepting states and
+ non-accepting states of any sub-NFA's.
+ """
+ assert len(str) > 0
+ arcs = map(lambda x : newArcPair(states, x), str)
+ finish = len(states)
+ states.append([])
+ start, lastFinish = arcs[0]
+ states[start].append((EMPTY, finish))
+ for crntStart, crntFinish in arcs[1:]:
+ states[lastFinish].append((EMPTY, crntStart))
+ states[crntStart].append((EMPTY, finish))
+ return start, finish
+
+# ______________________________________________________________________
+
+def group (states, *stateIndexPairs):
+ if len(stateIndexPairs) > 1:
+ start = len(states)
+ finish = start + 1
+ startList = []
+ states.append(startList)
+ states.append([])
+ for eStart, eFinish in stateIndexPairs:
+ startList.append((EMPTY, eStart))
+ states[eFinish].append((EMPTY, finish))
+ return start, finish
+ else:
+ return stateIndexPairs[0]
+
+# ______________________________________________________________________
+
+def groupStr (states, str):
+ return group(states, *map(lambda x : newArcPair(states, x), str))
+
+# ______________________________________________________________________
+
+def notGroup (states, *stateIndexPairs):
+ """Like group, but will add a DEFAULT transition to a new end state,
+ causing anything in the group to not match by going to a dead state.
+ XXX I think this is right...
+ """
+ start, dead = group(states, *stateIndexPairs)
+ finish = len(states)
+ states.append([])
+ states[start].append((DEFAULT, finish))
+ return start, finish
+
+# ______________________________________________________________________
+
+def notGroupStr (states, str):
+ return notGroup(states, *map(lambda x : newArcPair(states, x), str))
+# ______________________________________________________________________
+
+def any (states, *stateIndexPairs):
+ start, finish = group(states, *stateIndexPairs)
+ states[finish].append((EMPTY, start))
+ return start, start
+
+# ______________________________________________________________________
+
+def maybe (states, *stateIndexPairs):
+ start, finish = group(states, *stateIndexPairs)
+ states[start].append((EMPTY, finish))
+ return start, finish
+
+# ______________________________________________________________________
+
+def atleastonce (states, *stateIndexPairs):
+ start, finish = group(states, *stateIndexPairs)
+ states[finish].append((EMPTY, start))
+ return start, finish
+
+# ______________________________________________________________________
+
+def closure (states, start, result = 0L):
+ if None == result:
+ result = 0L
+ if 0 == (result & (1L << start)):
+ result |= (1L << start)
+ for label, arrow in states[start]:
+ if label == EMPTY:
+ result |= closure(states, arrow, result)
+ return result
+
+# ______________________________________________________________________
+
+def nfaToDfa (states, start, finish):
+ tempStates = []
+ startClosure = closure(states, start)
+ crntTempState = [startClosure, [], 0 != (startClosure & (1L << finish))]
+ tempStates.append(crntTempState)
+ index = 0
+ while index < len(tempStates):
+ crntTempState = tempStates[index]
+ crntClosure, crntArcs, crntAccept = crntTempState
+ for index2 in range(0, len(states)):
+ if 0 != (crntClosure & (1L << index2)):
+ for label, nfaArrow in states[index2]:
+ if label == EMPTY:
+ continue
+ foundTempArc = False
+ for tempArc in crntArcs:
+ if tempArc[0] == label:
+ foundTempArc = True
+ break
+ if not foundTempArc:
+ tempArc = [label, -1, 0L]
+ crntArcs.append(tempArc)
+ tempArc[2] = closure(states, nfaArrow, tempArc[2])
+ for arcIndex in range(0, len(crntArcs)):
+ label, arrow, targetStates = crntArcs[arcIndex]
+ targetFound = False
+ arrow = 0
+ for destTempState in tempStates:
+ if destTempState[0] == targetStates:
+ targetFound = True
+ break
+ arrow += 1
+ if not targetFound:
+ assert arrow == len(tempStates)
+ newState = [targetStates, [], 0 != (targetStates &
+ (1L << finish))]
+ tempStates.append(newState)
+ crntArcs[arcIndex][1] = arrow
+ index += 1
+ tempStates = simplifyTempDfa(tempStates)
+ states = finalizeTempDfa(tempStates)
+ return states
+
+# ______________________________________________________________________
+
+def sameState (s1, s2):
+ """sameState(s1, s2)
+ Note:
+ state := [ nfaclosure : Long, [ arc ], accept : Boolean ]
+ arc := [ label, arrow : Int, nfaClosure : Long ]
+ """
+ if (len(s1[1]) != len(s2[1])) or (s1[2] != s2[2]):
+ return False
+ for arcIndex in range(0, len(s1[1])):
+ arc1 = s1[1][arcIndex]
+ arc2 = s2[1][arcIndex]
+ if arc1[:-1] != arc2[:-1]:
+ return False
+ return True
+
+# ______________________________________________________________________
+
+def simplifyTempDfa (tempStates):
+ """simplifyTempDfa (tempStates)
+ """
+ changes = True
+ deletedStates = []
+ while changes:
+ changes = False
+ for i in range(1, len(tempStates)):
+ if i in deletedStates:
+ continue
+ for j in range(0, i):
+ if j in deletedStates:
+ continue
+ if sameState(tempStates[i], tempStates[j]):
+ deletedStates.append(i)
+ for k in range(0, len(tempStates)):
+ if k in deletedStates:
+ continue
+ for arc in tempStates[k][1]:
+ if arc[1] == i:
+ arc[1] = j
+ changes = True
+ break
+ for stateIndex in deletedStates:
+ tempStates[stateIndex] = None
+ return tempStates
+# ______________________________________________________________________
+
+def finalizeTempDfa (tempStates):
+ """finalizeTempDfa (tempStates)
+
+ Input domain:
+ tempState := [ nfaClosure : Long, [ tempArc ], accept : Boolean ]
+ tempArc := [ label, arrow, nfaClosure ]
+
+ Output domain:
+ state := [ arcMap, accept : Boolean ]
+ """
+ states = []
+ accepts = []
+ stateMap = {}
+ tempIndex = 0
+ for tempIndex in range(0, len(tempStates)):
+ tempState = tempStates[tempIndex]
+ if None != tempState:
+ stateMap[tempIndex] = len(states)
+ states.append({})
+ accepts.append(tempState[2])
+ for tempIndex in stateMap.keys():
+ stateBitset, tempArcs, accepting = tempStates[tempIndex]
+ newIndex = stateMap[tempIndex]
+ arcMap = states[newIndex]
+ for tempArc in tempArcs:
+ arcMap[tempArc[0]] = stateMap[tempArc[1]]
+ return states, accepts
+
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py Tue Oct 5 18:36:38 2010
@@ -21,39 +21,40 @@
__all__ = [ "tokenize" ]
# ______________________________________________________________________
-# Automatically generated DFA's (with one or two hand tweeks):
-pseudoStatesAccepts = [True, True, True, True, True, True, True, True,
- True, True, False, True, True, True, False, False,
- False, False, True, False, False, True, True, False,
- True, False, True, False, True, False, True, False,
- False, False, True, False, False, False, True]
-
-pseudoStates = [
- {'\t': 0, '\n': 13, '\x0c': 0, '\r': 14, ' ': 0, '!': 10,
- '"': 16, '#': 18, '%': 12, '&': 12,
- "'": 15, '(': 13, ')': 13, '*': 7,
- '+': 12, ',': 13, '-': 12, '.': 6,
- '/': 11, '0': 4, '1': 5, '2': 5,
- '3': 5, '4': 5, '5': 5, '6': 5,
- '7': 5, '8': 5, '9': 5, ':': 13,
- ';': 13, '<': 9, '=': 12, '>': 8,
- '@': 13, 'A': 1,
- 'B': 3, 'C': 1, 'D': 1, 'E': 1,
- 'F': 1, 'G': 1, 'H': 1, 'I': 1,
- 'J': 1, 'K': 1, 'L': 1, 'M': 1,
- 'N': 1, 'O': 1, 'P': 1, 'Q': 1,
- 'R': 2, 'S': 1, 'T': 1, 'U': 3,
- 'V': 1, 'W': 1, 'X': 1, 'Y': 1,
- 'Z': 1, '[': 13, '\\': 17, ']': 13,
- '^': 12, '_': 1, '`': 13, 'a': 1,
- 'b': 3, 'c': 1, 'd': 1, 'e': 1,
- 'f': 1, 'g': 1, 'h': 1, 'i': 1,
- 'j': 1, 'k': 1, 'l': 1, 'm': 1,
- 'n': 1, 'o': 1, 'p': 1, 'q': 1,
- 'r': 2, 's': 1, 't': 1, 'u': 3,
- 'v': 1, 'w': 1, 'x': 1, 'y': 1,
- 'z': 1, '{': 13, '|': 12, '}': 13,
- '~': 13},
+# Automatically generated DFA's
+
+accepts = [True, True, True, True, True, True, True, True,
+ True, True, False, True, True, True, False, False,
+ False, False, True, False, False, True, True,
+ False, True, False, True, False, True, False,
+ True, False, False, False, True, False, False,
+ False, True]
+states = [
+ {'\t': 0, '\n': 13, '\x0c': 0,
+ '\r': 14, ' ': 0, '!': 10, '"': 16,
+ '#': 18, '%': 12, '&': 12, "'": 15,
+ '(': 13, ')': 13, '*': 7, '+': 12,
+ ',': 13, '-': 12, '.': 6, '/': 11,
+ '0': 4, '1': 5, '2': 5, '3': 5,
+ '4': 5, '5': 5, '6': 5, '7': 5,
+ '8': 5, '9': 5, ':': 13, ';': 13,
+ '<': 9, '=': 12, '>': 8, '@': 13,
+ 'A': 1, 'B': 2, 'C': 1, 'D': 1,
+ 'E': 1, 'F': 1, 'G': 1, 'H': 1,
+ 'I': 1, 'J': 1, 'K': 1, 'L': 1,
+ 'M': 1, 'N': 1, 'O': 1, 'P': 1,
+ 'Q': 1, 'R': 3, 'S': 1, 'T': 1,
+ 'U': 2, 'V': 1, 'W': 1, 'X': 1,
+ 'Y': 1, 'Z': 1, '[': 13, '\\': 17,
+ ']': 13, '^': 12, '_': 1, '`': 13,
+ 'a': 1, 'b': 2, 'c': 1, 'd': 1,
+ 'e': 1, 'f': 1, 'g': 1, 'h': 1,
+ 'i': 1, 'j': 1, 'k': 1, 'l': 1,
+ 'm': 1, 'n': 1, 'o': 1, 'p': 1,
+ 'q': 1, 'r': 3, 's': 1, 't': 1,
+ 'u': 2, 'v': 1, 'w': 1, 'x': 1,
+ 'y': 1, 'z': 1, '{': 13, '|': 12,
+ '}': 13, '~': 13},
{'0': 1, '1': 1, '2': 1, '3': 1,
'4': 1, '5': 1, '6': 1, '7': 1,
@@ -72,39 +73,39 @@
't': 1, 'u': 1, 'v': 1, 'w': 1,
'x': 1, 'y': 1, 'z': 1},
- {'"': 16, "'": 15, '0': 1, '1': 1,
+ {'"': 20, "'": 19, '0': 1, '1': 1,
'2': 1, '3': 1, '4': 1, '5': 1,
'6': 1, '7': 1, '8': 1, '9': 1,
'A': 1, 'B': 1, 'C': 1, 'D': 1,
'E': 1, 'F': 1, 'G': 1, 'H': 1,
'I': 1, 'J': 1, 'K': 1, 'L': 1,
'M': 1, 'N': 1, 'O': 1, 'P': 1,
- 'Q': 1, 'R': 1, 'S': 1, 'T': 1,
+ 'Q': 1, 'R': 3, 'S': 1, 'T': 1,
'U': 1, 'V': 1, 'W': 1, 'X': 1,
'Y': 1, 'Z': 1, '_': 1, 'a': 1,
'b': 1, 'c': 1, 'd': 1, 'e': 1,
'f': 1, 'g': 1, 'h': 1, 'i': 1,
'j': 1, 'k': 1, 'l': 1, 'm': 1,
'n': 1, 'o': 1, 'p': 1, 'q': 1,
- 'r': 1, 's': 1, 't': 1, 'u': 1,
+ 'r': 3, 's': 1, 't': 1, 'u': 1,
'v': 1, 'w': 1, 'x': 1, 'y': 1,
'z': 1},
- {'"': 16, "'": 15, '0': 1, '1': 1,
+ {'"': 20, "'": 19, '0': 1, '1': 1,
'2': 1, '3': 1, '4': 1, '5': 1,
'6': 1, '7': 1, '8': 1, '9': 1,
'A': 1, 'B': 1, 'C': 1, 'D': 1,
'E': 1, 'F': 1, 'G': 1, 'H': 1,
'I': 1, 'J': 1, 'K': 1, 'L': 1,
'M': 1, 'N': 1, 'O': 1, 'P': 1,
- 'Q': 1, 'R': 2, 'S': 1, 'T': 1,
+ 'Q': 1, 'R': 1, 'S': 1, 'T': 1,
'U': 1, 'V': 1, 'W': 1, 'X': 1,
'Y': 1, 'Z': 1, '_': 1, 'a': 1,
'b': 1, 'c': 1, 'd': 1, 'e': 1,
'f': 1, 'g': 1, 'h': 1, 'i': 1,
'j': 1, 'k': 1, 'l': 1, 'm': 1,
'n': 1, 'o': 1, 'p': 1, 'q': 1,
- 'r': 2, 's': 1, 't': 1, 'u': 1,
+ 'r': 1, 's': 1, 't': 1, 'u': 1,
'v': 1, 'w': 1, 'x': 1, 'y': 1,
'z': 1},
@@ -128,11 +129,11 @@
{'=': 13, '>': 12},
- {'=': 13, '<': 12, '>': 13},
+ {'<': 12, '=': 13, '>': 13},
{'=': 13},
- {'=': 13, '/': 12},
+ {'/': 12, '=': 13},
{'=': 13},
@@ -140,77 +141,85 @@
{'\n': 13},
- {automata.DEFAULT: 19, '\n': 27, '\\': 29, "'": 28},
+ {automata.DEFAULT: 19, '\n': 27,
+ "'": 28, '\\': 29},
- {automata.DEFAULT: 20, '"': 30, '\n': 27, '\\': 31},
+ {automata.DEFAULT: 20, '\n': 27,
+ '"': 30, '\\': 31},
{'\n': 13, '\r': 14},
- {automata.DEFAULT: 18, '\n': 27, '\r': 27},
+ {automata.DEFAULT: 18, '\n': 27,
+ '\r': 27},
+
+ {automata.DEFAULT: 19, '\n': 27,
+ "'": 13, '\\': 29},
- {automata.DEFAULT: 19, '\n': 27, '\\': 29, "'": 13},
+ {automata.DEFAULT: 20, '\n': 27,
+ '"': 13, '\\': 31},
- {automata.DEFAULT: 20, '"': 13, '\n': 27, '\\': 31},
-
{'0': 21, '1': 21, '2': 21, '3': 21,
'4': 21, '5': 21, '6': 21, '7': 21,
'8': 21, '9': 21, 'A': 21, 'B': 21,
'C': 21, 'D': 21, 'E': 21, 'F': 21,
'L': 13, 'a': 21, 'b': 21, 'c': 21,
'd': 21, 'e': 21, 'f': 21, 'l': 13},
-
+
{'.': 24, '0': 22, '1': 22, '2': 22,
'3': 22, '4': 22, '5': 22, '6': 22,
'7': 22, '8': 23, '9': 23, 'E': 25,
'J': 13, 'L': 13, 'e': 25, 'j': 13,
'l': 13},
-
+
{'.': 24, '0': 23, '1': 23, '2': 23,
'3': 23, '4': 23, '5': 23, '6': 23,
'7': 23, '8': 23, '9': 23, 'E': 25,
'J': 13, 'e': 25, 'j': 13},
-
+
{'0': 24, '1': 24, '2': 24, '3': 24,
'4': 24, '5': 24, '6': 24, '7': 24,
'8': 24, '9': 24, 'E': 32, 'J': 13,
'e': 32, 'j': 13},
-
+
{'+': 33, '-': 33, '0': 34, '1': 34,
'2': 34, '3': 34, '4': 34, '5': 34,
'6': 34, '7': 34, '8': 34, '9': 34},
-
+
{'0': 26, '1': 26, '2': 26, '3': 26,
'4': 26, '5': 26, '6': 26, '7': 26,
'8': 26, '9': 26, 'E': 32, 'J': 13,
'e': 32, 'j': 13},
-
+
{},
{"'": 13},
- {automata.DEFAULT: 35, '\n': 13, '\r': 14},
+ {automata.DEFAULT: 35, '\n': 13,
+ '\r': 14},
{'"': 13},
- {automata.DEFAULT: 36, '\n': 13, '\r': 14},
+ {automata.DEFAULT: 36, '\n': 13,
+ '\r': 14},
{'+': 37, '-': 37, '0': 38, '1': 38,
'2': 38, '3': 38, '4': 38, '5': 38,
'6': 38, '7': 38, '8': 38, '9': 38},
-
-
+
{'0': 34, '1': 34, '2': 34, '3': 34,
'4': 34, '5': 34, '6': 34, '7': 34,
'8': 34, '9': 34},
-
+
{'0': 34, '1': 34, '2': 34, '3': 34,
'4': 34, '5': 34, '6': 34, '7': 34,
'8': 34, '9': 34, 'J': 13, 'j': 13},
-
- {automata.DEFAULT: 35, '\n': 27, '\\': 29, "'": 13},
-
- {automata.DEFAULT: 36, '"': 13, '\n': 27, '\\': 31},
-
+
+ {automata.DEFAULT: 35, '\n': 27,
+ "'": 13, '\\': 29},
+
+ {automata.DEFAULT: 36, '\n': 27,
+ '"': 13, '\\': 31},
+
{'0': 38, '1': 38, '2': 38, '3': 38,
'4': 38, '5': 38, '6': 38, '7': 38,
'8': 38, '9': 38},
@@ -218,56 +227,95 @@
{'0': 38, '1': 38, '2': 38, '3': 38,
'4': 38, '5': 38, '6': 38, '7': 38,
'8': 38, '9': 38, 'J': 13, 'j': 13},
+
]
+pseudoDFA = automata.DFA(states, accepts)
-pseudoDFA = automata.DFA(pseudoStates, pseudoStatesAccepts)
+accepts = [False, False, False, False, False, True]
+states = [
+ {automata.DEFAULT: 0, '"': 1,
+ '\\': 2},
+
+ {automata.DEFAULT: 4, '"': 3,
+ '\\': 2},
-double3StatesAccepts = [False, False, False, False, False, True]
-double3States = [
- {automata.DEFAULT: 0, '"': 1, '\\': 2},
- {automata.DEFAULT: 4, '"': 3, '\\': 2},
{automata.DEFAULT: 4},
- {automata.DEFAULT: 4, '"': 5, '\\': 2},
- {automata.DEFAULT: 4, '"': 1, '\\': 2},
- {automata.DEFAULT: 4, '"': 5, '\\': 2},
+
+ {automata.DEFAULT: 4, '"': 5,
+ '\\': 2},
+
+ {automata.DEFAULT: 4, '"': 1,
+ '\\': 2},
+
+ {automata.DEFAULT: 4, '"': 5,
+ '\\': 2},
+
]
-double3DFA = automata.NonGreedyDFA(double3States, double3StatesAccepts)
+double3DFA = automata.NonGreedyDFA(states, accepts)
+
+accepts = [False, False, False, False, False, True]
+states = [
+ {automata.DEFAULT: 0, "'": 1,
+ '\\': 2},
+
+ {automata.DEFAULT: 4, "'": 3,
+ '\\': 2},
-single3StatesAccepts = [False, False, False, False, False, True]
-single3States = [
- {automata.DEFAULT: 0, '\\': 2, "'": 1},
- {automata.DEFAULT: 4, '\\': 2, "'": 3},
{automata.DEFAULT: 4},
- {automata.DEFAULT: 4, '\\': 2, "'": 5},
- {automata.DEFAULT: 4, '\\': 2, "'": 1},
- {automata.DEFAULT: 4, '\\': 2, "'": 5},
+
+ {automata.DEFAULT: 4, "'": 5,
+ '\\': 2},
+
+ {automata.DEFAULT: 4, "'": 1,
+ '\\': 2},
+
+ {automata.DEFAULT: 4, "'": 5,
+ '\\': 2},
+
]
-single3DFA = automata.NonGreedyDFA(single3States, single3StatesAccepts)
+single3DFA = automata.NonGreedyDFA(states, accepts)
+
+accepts = [False, True, False, False]
+states = [
+ {automata.DEFAULT: 0, '"': 1,
+ '\\': 2},
-singleStatesAccepts = [False, True, False]
-singleStates = [
- {automata.DEFAULT: 0, '\\': 2, "'": 1},
{},
- {automata.DEFAULT: 0},
+
+ {automata.DEFAULT: 3},
+
+ {automata.DEFAULT: 3, '"': 1,
+ '\\': 2},
+
]
-singleDFA = automata.DFA(singleStates, singleStatesAccepts)
+doubleDFA = automata.DFA(states, accepts)
+
+accepts = [False, True, False, False]
+states = [
+ {automata.DEFAULT: 0, "'": 1,
+ '\\': 2},
-doubleStatesAccepts = [False, True, False]
-doubleStates = [
- {automata.DEFAULT: 0, '"': 1, '\\': 2},
{},
- {automata.DEFAULT: 0},
+
+ {automata.DEFAULT: 3},
+
+ {automata.DEFAULT: 3, "'": 1,
+ '\\': 2},
+
]
-doubleDFA = automata.DFA(doubleStates, doubleStatesAccepts)
+singleDFA = automata.DFA(states, accepts)
endDFAs = {"'" : singleDFA,
'"' : doubleDFA,
- "r" : None,
- "R" : None,
- "u" : None,
- "U" : None,
- "b" : None,
- "B" : None}
+ 'r' : None,
+ 'R' : None,
+ 'u' : None,
+ 'U' : None,
+ 'b' : None,
+ 'B' : None}
+
+#_______________________________________________________________________
+# End of automatically generated DFA's
for uniPrefix in ("", "u", "U", "b", "B"):
for rawPrefix in ("", "r", "R"):
From afa at codespeak.net Tue Oct 5 18:53:43 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 5 Oct 2010 18:53:43 +0200 (CEST)
Subject: [pypy-svn] r77617 - in pypy/branch/fast-forward/pypy/interpreter:
astcompiler astcompiler/test pyparser pyparser/test
Message-ID: <20101005165343.7CD09282B9C@codespeak.net>
Author: afa
Date: Tue Oct 5 18:53:41 2010
New Revision: 77617
Modified:
pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py
pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py
pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py
pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py
pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py
Log:
Add new octal literal syntax 0o777 to the tokenizer.
Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py Tue Oct 5 18:53:41 2010
@@ -1037,13 +1037,15 @@
if raw.startswith("0"):
if len(raw) > 2 and raw[1] in "Xx":
base = 16
+ ## elif len(raw) > 2 and raw[1] in "Oo": # Fallback below is enough
+ ## base = 8
elif len(raw) > 1:
base = 8
# strip leading characters
i = 0
limit = len(raw) - 1
while i < limit:
- if raw[i] not in "0xX":
+ if raw[i] not in "0xXoO":
break
i += 1
raw = raw[i:]
Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py Tue Oct 5 18:53:41 2010
@@ -1137,6 +1137,8 @@
assert space.eq_w(get_num("-0xAAAAAAL"), space.wrap(-0xAAAAAAL))
n = get_num(str(-sys.maxint - 1))
assert space.is_true(space.isinstance(n, space.w_int))
+ for num in ("0o53", "0O53", "0o0000053", "0O00053"):
+ assert space.eq_w(get_num(num), space.wrap(053))
def check_comprehension(self, brackets, ast_type):
def brack(s):
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py Tue Oct 5 18:53:41 2010
@@ -58,6 +58,7 @@
maybe(states, groupStr(states, "lL")))
octNumber = chain(states,
newArcPair(states, "0"),
+ maybe(states, groupStr(states, "oO")),
any(states, groupStr(states, "01234567")),
maybe(states, groupStr(states, "lL")))
decNumber = chain(states,
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py Tue Oct 5 18:53:41 2010
@@ -26,9 +26,9 @@
accepts = [True, True, True, True, True, True, True, True,
True, True, False, True, True, True, False, False,
False, False, True, False, False, True, True,
- False, True, False, True, False, True, False,
- True, False, False, False, True, False, False,
- False, True]
+ True, False, True, False, True, False, True,
+ False, True, False, False, False, True, False,
+ False, False, True]
states = [
{'\t': 0, '\n': 13, '\x0c': 0,
'\r': 14, ' ': 0, '!': 10, '"': 16,
@@ -109,21 +109,22 @@
'v': 1, 'w': 1, 'x': 1, 'y': 1,
'z': 1},
- {'.': 24, '0': 22, '1': 22, '2': 22,
- '3': 22, '4': 22, '5': 22, '6': 22,
- '7': 22, '8': 23, '9': 23, 'E': 25,
- 'J': 13, 'L': 13, 'X': 21, 'e': 25,
- 'j': 13, 'l': 13, 'x': 21},
+ {'.': 25, '0': 23, '1': 23, '2': 23,
+ '3': 23, '4': 23, '5': 23, '6': 23,
+ '7': 23, '8': 24, '9': 24, 'E': 26,
+ 'J': 13, 'L': 13, 'O': 22, 'X': 21,
+ 'e': 26, 'j': 13, 'l': 13, 'o': 22,
+ 'x': 21},
- {'.': 24, '0': 5, '1': 5, '2': 5,
+ {'.': 25, '0': 5, '1': 5, '2': 5,
'3': 5, '4': 5, '5': 5, '6': 5,
- '7': 5, '8': 5, '9': 5, 'E': 25,
- 'J': 13, 'L': 13, 'e': 25, 'j': 13,
+ '7': 5, '8': 5, '9': 5, 'E': 26,
+ 'J': 13, 'L': 13, 'e': 26, 'j': 13,
'l': 13},
- {'0': 26, '1': 26, '2': 26, '3': 26,
- '4': 26, '5': 26, '6': 26, '7': 26,
- '8': 26, '9': 26},
+ {'0': 27, '1': 27, '2': 27, '3': 27,
+ '4': 27, '5': 27, '6': 27, '7': 27,
+ '8': 27, '9': 27},
{'*': 12, '=': 13},
@@ -141,92 +142,101 @@
{'\n': 13},
- {automata.DEFAULT: 19, '\n': 27,
- "'": 28, '\\': 29},
+ {automata.DEFAULT: 19, '\n': 28,
+ "'": 29, '\\': 30},
- {automata.DEFAULT: 20, '\n': 27,
- '"': 30, '\\': 31},
+ {automata.DEFAULT: 20, '\n': 28,
+ '"': 31, '\\': 32},
{'\n': 13, '\r': 14},
- {automata.DEFAULT: 18, '\n': 27,
- '\r': 27},
+ {automata.DEFAULT: 18, '\n': 28,
+ '\r': 28},
- {automata.DEFAULT: 19, '\n': 27,
- "'": 13, '\\': 29},
+ {automata.DEFAULT: 19, '\n': 28,
+ "'": 13, '\\': 30},
- {automata.DEFAULT: 20, '\n': 27,
- '"': 13, '\\': 31},
+ {automata.DEFAULT: 20, '\n': 28,
+ '"': 13, '\\': 32},
{'0': 21, '1': 21, '2': 21, '3': 21,
'4': 21, '5': 21, '6': 21, '7': 21,
'8': 21, '9': 21, 'A': 21, 'B': 21,
'C': 21, 'D': 21, 'E': 21, 'F': 21,
'L': 13, 'a': 21, 'b': 21, 'c': 21,
- 'd': 21, 'e': 21, 'f': 21, 'l': 13},
-
- {'.': 24, '0': 22, '1': 22, '2': 22,
- '3': 22, '4': 22, '5': 22, '6': 22,
- '7': 22, '8': 23, '9': 23, 'E': 25,
- 'J': 13, 'L': 13, 'e': 25, 'j': 13,
+ 'd': 21, 'e': 21, 'f': 21,
'l': 13},
- {'.': 24, '0': 23, '1': 23, '2': 23,
+ {'0': 22, '1': 22, '2': 22, '3': 22,
+ '4': 22, '5': 22, '6': 22, '7': 22,
+ 'L': 13, 'l': 13},
+
+ {'.': 25, '0': 23, '1': 23, '2': 23,
'3': 23, '4': 23, '5': 23, '6': 23,
- '7': 23, '8': 23, '9': 23, 'E': 25,
- 'J': 13, 'e': 25, 'j': 13},
+ '7': 23, '8': 24, '9': 24, 'E': 26,
+ 'J': 13, 'L': 13, 'e': 26, 'j': 13,
+ 'l': 13},
- {'0': 24, '1': 24, '2': 24, '3': 24,
- '4': 24, '5': 24, '6': 24, '7': 24,
- '8': 24, '9': 24, 'E': 32, 'J': 13,
- 'e': 32, 'j': 13},
-
- {'+': 33, '-': 33, '0': 34, '1': 34,
- '2': 34, '3': 34, '4': 34, '5': 34,
- '6': 34, '7': 34, '8': 34, '9': 34},
-
- {'0': 26, '1': 26, '2': 26, '3': 26,
- '4': 26, '5': 26, '6': 26, '7': 26,
- '8': 26, '9': 26, 'E': 32, 'J': 13,
- 'e': 32, 'j': 13},
+ {'.': 25, '0': 24, '1': 24, '2': 24,
+ '3': 24, '4': 24, '5': 24, '6': 24,
+ '7': 24, '8': 24, '9': 24, 'E': 26,
+ 'J': 13, 'e': 26, 'j': 13},
+
+ {'0': 25, '1': 25, '2': 25, '3': 25,
+ '4': 25, '5': 25, '6': 25, '7': 25,
+ '8': 25, '9': 25, 'E': 33, 'J': 13,
+ 'e': 33, 'j': 13},
+
+ {'+': 34, '-': 34, '0': 35, '1': 35,
+ '2': 35, '3': 35, '4': 35, '5': 35,
+ '6': 35, '7': 35, '8': 35,
+ '9': 35},
+
+ {'0': 27, '1': 27, '2': 27, '3': 27,
+ '4': 27, '5': 27, '6': 27, '7': 27,
+ '8': 27, '9': 27, 'E': 33, 'J': 13,
+ 'e': 33, 'j': 13},
{},
{"'": 13},
- {automata.DEFAULT: 35, '\n': 13,
+ {automata.DEFAULT: 36, '\n': 13,
'\r': 14},
{'"': 13},
- {automata.DEFAULT: 36, '\n': 13,
+ {automata.DEFAULT: 37, '\n': 13,
'\r': 14},
- {'+': 37, '-': 37, '0': 38, '1': 38,
- '2': 38, '3': 38, '4': 38, '5': 38,
- '6': 38, '7': 38, '8': 38, '9': 38},
-
- {'0': 34, '1': 34, '2': 34, '3': 34,
- '4': 34, '5': 34, '6': 34, '7': 34,
- '8': 34, '9': 34},
-
- {'0': 34, '1': 34, '2': 34, '3': 34,
- '4': 34, '5': 34, '6': 34, '7': 34,
- '8': 34, '9': 34, 'J': 13, 'j': 13},
-
- {automata.DEFAULT: 35, '\n': 27,
- "'": 13, '\\': 29},
-
- {automata.DEFAULT: 36, '\n': 27,
- '"': 13, '\\': 31},
-
- {'0': 38, '1': 38, '2': 38, '3': 38,
- '4': 38, '5': 38, '6': 38, '7': 38,
- '8': 38, '9': 38},
-
- {'0': 38, '1': 38, '2': 38, '3': 38,
- '4': 38, '5': 38, '6': 38, '7': 38,
- '8': 38, '9': 38, 'J': 13, 'j': 13},
+ {'+': 38, '-': 38, '0': 39, '1': 39,
+ '2': 39, '3': 39, '4': 39, '5': 39,
+ '6': 39, '7': 39, '8': 39,
+ '9': 39},
+
+ {'0': 35, '1': 35, '2': 35, '3': 35,
+ '4': 35, '5': 35, '6': 35, '7': 35,
+ '8': 35, '9': 35},
+
+ {'0': 35, '1': 35, '2': 35, '3': 35,
+ '4': 35, '5': 35, '6': 35, '7': 35,
+ '8': 35, '9': 35, 'J': 13,
+ 'j': 13},
+
+ {automata.DEFAULT: 36, '\n': 28,
+ "'": 13, '\\': 30},
+
+ {automata.DEFAULT: 37, '\n': 28,
+ '"': 13, '\\': 32},
+
+ {'0': 39, '1': 39, '2': 39, '3': 39,
+ '4': 39, '5': 39, '6': 39, '7': 39,
+ '8': 39, '9': 39},
+
+ {'0': 39, '1': 39, '2': 39, '3': 39,
+ '4': 39, '5': 39, '6': 39, '7': 39,
+ '8': 39, '9': 39, 'J': 13,
+ 'j': 13},
]
pseudoDFA = automata.DFA(states, accepts)
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py Tue Oct 5 18:53:41 2010
@@ -108,3 +108,8 @@
self.parse("br'\\\n'")
py.test.raises(SyntaxError, self.parse, "b'a\\n")
+
+ def test_new_octal_literal(self):
+ self.parse('0777')
+ self.parse('0o777')
+ py.test.raises(SyntaxError, self.parse, "0o778")
From arigo at codespeak.net Tue Oct 5 18:53:58 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 18:53:58 +0200 (CEST)
Subject: [pypy-svn] r77618 -
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test
Message-ID: <20101005165358.A78AF282B9C@codespeak.net>
Author: arigo
Date: Tue Oct 5 18:53:57 2010
New Revision: 77618
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_transformed_gc.py
Log:
Fix this test, now that we can (slowly) run test_transformed_gc with
the gc flag DEBUG=True.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_transformed_gc.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_transformed_gc.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_transformed_gc.py Tue Oct 5 18:53:57 2010
@@ -1315,7 +1315,8 @@
def test_immutable_to_old_promotion(self):
run, transformer = self.runner("immutable_to_old_promotion", transformer=True)
run([1, 4])
- if not transformer.GCClass.prebuilt_gc_objects_are_static_roots:
+ if (not transformer.GCClass.prebuilt_gc_objects_are_static_roots
+ and not transformer.GCClass.DEBUG):
assert len(transformer.layoutbuilder.addresses_of_static_ptrs) == 0
else:
assert len(transformer.layoutbuilder.addresses_of_static_ptrs) >= 4
From arigo at codespeak.net Tue Oct 5 18:54:25 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 18:54:25 +0200 (CEST)
Subject: [pypy-svn] r77619 -
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform
Message-ID: <20101005165425.2A314282B9C@codespeak.net>
Author: arigo
Date: Tue Oct 5 18:54:23 2010
New Revision: 77619
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
Log:
Fix: confusion between addresses and pointers.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py Tue Oct 5 18:54:23 2010
@@ -1031,7 +1031,8 @@
# a constant won't be ok
return
if v_newvalue.concretetype == llmemory.HiddenGcRef32:
- v_newvalue = self._fetch_unpacked_pointer(hop, v_newvalue)
+ v_newvalue = self._fetch_unpacked_address(hop, v_newvalue)
+ assert v_newvalue.concretetype == llmemory.Address
if isinstance(v_newvalue, Constant):
# comes from a Constant -- skip
return
@@ -1057,12 +1058,16 @@
v_newvalue,
v_structaddr])
- def _fetch_unpacked_pointer(self, hop, v_value):
+ def _fetch_unpacked_address(self, hop, v_value):
# optimization for the common case where this setfield is preceded
# by v_value = hide_into_adr32(v_normal_pointer)
for op in hop.llops[::-1]:
if op.opname == 'hide_into_adr32' and op.result == v_value:
- return op.args[0]
+ v_value = op.args[0]
+ if v_value.concretetype != llmemory.Address:
+ v_value = hop.genop("cast_ptr_to_adr", [v_value],
+ resulttype = llmemory.Address)
+ return v_value
else:
return hop.genop("show_from_adr32", [v_value],
resulttype = llmemory.Address)
From arigo at codespeak.net Tue Oct 5 19:02:35 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 19:02:35 +0200 (CEST)
Subject: [pypy-svn] r77620 - in pypy/branch/32ptr-on-64bit/pypy:
rpython/lltypesystem rpython/lltypesystem/test
rpython/memory/gc rpython/memory/gctransform translator/c
translator/c/src
Message-ID: <20101005170235.DCF62282B9C@codespeak.net>
Author: arigo
Date: Tue Oct 5 19:02:33 2010
New Revision: 77620
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/opimpl.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/test/test_llmemory.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/funcgen.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py
pypy/branch/32ptr-on-64bit/pypy/translator/c/src/hiddengcref32.h
Log:
Rename {hide_into,show_from}_adr32 into *_ptr32,
which matches the current implementation better.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/lloperation.py Tue Oct 5 19:02:33 2010
@@ -426,8 +426,8 @@
'gc_gettypeptr_group': LLOp(canfold=True),
'get_member_index': LLOp(canfold=True),
- 'hide_into_adr32': LLOp(canrun=True),
- 'show_from_adr32': LLOp(canrun=True),
+ 'hide_into_ptr32': LLOp(canrun=True),
+ 'show_from_ptr32': LLOp(canrun=True),
# __________ used by the JIT ________
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/opimpl.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/opimpl.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/opimpl.py Tue Oct 5 19:02:33 2010
@@ -526,14 +526,14 @@
def op_shrink_array(array, smallersize):
return False
-def op_hide_into_adr32(ptr):
+def op_hide_into_ptr32(ptr):
if lltype.typeOf(ptr) == llmemory.Address:
if not ptr:
return lltype.nullptr(llmemory.HiddenGcRef32.TO)
ptr = ptr.ptr
return lltype.cast_opaque_ptr(llmemory.HiddenGcRef32, ptr)
-def op_show_from_adr32(RESTYPE, ptr32):
+def op_show_from_ptr32(RESTYPE, ptr32):
if RESTYPE == llmemory.Address:
if not ptr32:
return llmemory.NULL
@@ -541,7 +541,7 @@
ptr = lltype.cast_opaque_ptr(PTRTYPE, ptr32)
return llmemory.cast_ptr_to_adr(ptr)
return lltype.cast_opaque_ptr(RESTYPE, ptr32)
-op_show_from_adr32.need_result_type = True
+op_show_from_ptr32.need_result_type = True
# ____________________________________________________________
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py Tue Oct 5 19:02:33 2010
@@ -50,7 +50,7 @@
ptr = self.baserepr.convert_const(value)
T = lltype.typeOf(ptr)
assert T == self.BASETYPE
- return llop.hide_into_adr32(self.lowleveltype, ptr)
+ return llop.hide_into_ptr32(self.lowleveltype, ptr)
def get_ll_eq_function(self):
if self.baserepr.get_ll_eq_function() is not None:
@@ -63,7 +63,7 @@
BASETYPE = self.BASETYPE
#
def ll_hiddengcref32_hash(x):
- x = llop.show_from_adr32(BASETYPE, x)
+ x = llop.show_from_ptr32(BASETYPE, x)
return basefunc(x)
#
self.ll_hash_function = ll_hiddengcref32_hash
@@ -77,7 +77,7 @@
BASETYPE = self.BASETYPE
#
def ll_hiddengcref32_fasthash(x):
- x = llop.show_from_adr32(BASETYPE, x)
+ x = llop.show_from_ptr32(BASETYPE, x)
return basefunc(x)
#
self.ll_fasthash_function = ll_hiddengcref32_hash
@@ -98,12 +98,12 @@
def convert_from_to((r_from, r_to), v, llops):
assert r_from.lowleveltype.TO._gckind == 'gc'
assert not isinstance(r_from.lowleveltype.TO, lltype.GcOpaqueType)
- return llops.genop('hide_into_adr32', [v],
+ return llops.genop('hide_into_ptr32', [v],
resulttype=llmemory.HiddenGcRef32)
class __extend__(pairtype(CompressedGcRefRepr, Repr)):
def convert_from_to((r_from, r_to), v, llops):
assert r_to.lowleveltype.TO._gckind == 'gc'
assert not isinstance(r_to.lowleveltype.TO, lltype.GcOpaqueType)
- return llops.genop('show_from_adr32', [v],
+ return llops.genop('show_from_ptr32', [v],
resulttype=r_to.lowleveltype)
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/test/test_llmemory.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/test/test_llmemory.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/test/test_llmemory.py Tue Oct 5 19:02:33 2010
@@ -654,6 +654,6 @@
from pypy.rpython.lltypesystem.lloperation import llop
S = lltype.GcStruct('S')
p = lltype.malloc(S)
- q = llop.hide_into_adr32(HiddenGcRef32, p)
- r = llop.show_from_adr32(lltype.Ptr(S), q)
+ q = llop.hide_into_ptr32(HiddenGcRef32, p)
+ r = llop.show_from_ptr32(lltype.Ptr(S), q)
assert r == p
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py Tue Oct 5 19:02:33 2010
@@ -215,11 +215,11 @@
ofs = llmemory.remove_odd_value_marker(ofs)
item = obj + ofs
item = llmemory.cast_adr_to_ptr(item, lltype.Ptr(HIDDENGCREFFIELD))
- address = llop.show_from_adr32(llmemory.Address, item[0])
+ address = llop.show_from_ptr32(llmemory.Address, item[0])
if self.is_valid_gc_object(address):
newaddr = callback(address, arg)
if newaddr is not None:
- item[0] = llop.hide_into_adr32(llmemory.HiddenGcRef32,
+ item[0] = llop.hide_into_ptr32(llmemory.HiddenGcRef32,
newaddr)
else:
# common case
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py Tue Oct 5 19:02:33 2010
@@ -1060,16 +1060,16 @@
def _fetch_unpacked_address(self, hop, v_value):
# optimization for the common case where this setfield is preceded
- # by v_value = hide_into_adr32(v_normal_pointer)
+ # by v_value = hide_into_ptr32(v_normal_pointer)
for op in hop.llops[::-1]:
- if op.opname == 'hide_into_adr32' and op.result == v_value:
+ if op.opname == 'hide_into_ptr32' and op.result == v_value:
v_value = op.args[0]
if v_value.concretetype != llmemory.Address:
v_value = hop.genop("cast_ptr_to_adr", [v_value],
resulttype = llmemory.Address)
return v_value
else:
- return hop.genop("show_from_adr32", [v_value],
+ return hop.genop("show_from_ptr32", [v_value],
resulttype = llmemory.Address)
def transform_getfield_typeptr(self, hop):
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/funcgen.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/funcgen.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/funcgen.py Tue Oct 5 19:02:33 2010
@@ -647,17 +647,28 @@
self.expr(op.args[0])))
return '\t'.join(result)
- OP_CAST_PTR_TO_ADR = OP_CAST_POINTER
- OP_CAST_ADR_TO_PTR = OP_CAST_POINTER
+ def OP_CAST_PTR_TO_ADR(self, op):
+ # detect a case that is not supported and should be handled with
+ # show_from_ptr32.
+ if op.args[0].concretetype == HiddenGcRef32:
+ raise Exception("cast_ptr_to_adr(hiddengcref32): not supported")
+ return self.OP_CAST_POINTER(op)
+
+ def OP_CAST_ADR_TO_PTR(self, op):
+ # detect a case that is not supported and should be handled with
+ # hide_into_ptr32.
+ if op.result.concretetype == HiddenGcRef32:
+ raise Exception("hiddengcref32 = cast_adr_to_ptr(): not supported")
+ return self.OP_CAST_POINTER(op)
def OP_CAST_OPAQUE_PTR(self, op):
+ # detect cases that are not supported and should be handled with
+ # hide_into_ptr32 or show_from_ptr32.
if op.result.concretetype == HiddenGcRef32:
- return 'OP_HIDE_INTO_ADR32(%s, %s);' % (self.expr(op.args[0]),
- self.expr(op.result))
+ raise Exception("hiddengcref32 = cast_opaque_ptr(): not supported")
if op.args[0].concretetype == HiddenGcRef32:
- return 'OP_SHOW_FROM_ADR32(%s, %s);' % (self.expr(op.args[0]),
- self.expr(op.result))
- return self.OP_CAST_OPAQUE_PTR(op)
+ raise Exception("cast_opaque_ptr(hiddengcref32): not supported")
+ return self.OP_CAST_POINTER(op)
def OP_CAST_INT_TO_PTR(self, op):
TYPE = self.lltypemap(op.result)
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py Tue Oct 5 19:02:33 2010
@@ -923,7 +923,7 @@
\tfor (p = pypy_hiddengcref32; p[0] != NULL; p += 2)
\t{
\t\thiddengcref32_t h;
-\t\tOP_HIDE_INTO_ADR32_CHECK((p[1]), h);
+\t\tOP_HIDE_INTO_PTR32_CHECK((p[1]), h);
\t\t*(hiddengcref32_t*)(p[0]) = h;
\t}
}
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/node.py Tue Oct 5 19:02:33 2010
@@ -790,14 +790,14 @@
expr = '0.0 /* patched later by %sinfinity */' % (
'-+'[value > 0])
elif (TYPE == llmemory.HiddenGcRef32
- and value and not hasattr(value, 'dummy_value')):
+ and value and not hasattr(value._obj, 'dummy_value')):
realobj = value._obj.container
realvalue = cast_opaque_ptr(Ptr(typeOf(realobj)), value)
name = db.get(realvalue)
db.late_initializations_hiddengcref32.append((access_expr, name))
if not name.startswith('('):
name = '(%s)' % name
- expr = '0 /*HIDE_INTO_ADR32%s*/' % name
+ expr = '0 /*HIDE_INTO_PTR32%s*/' % name
else:
expr = db.get(value)
if TYPE is Void:
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/primitive.py Tue Oct 5 19:02:33 2010
@@ -148,17 +148,17 @@
return 'NULL'
def name_hiddengcref32(value, db):
- if hasattr(value, 'dummy_value'):
- return '1 /* dummy value */'
- elif not value:
+ if not value:
return '0' # NULL
+ elif hasattr(value._obj, 'dummy_value'):
+ return '1 /* dummy value */'
# The only prebuilt HiddenGcRef32 that should occur in a translated C
# program occur as fields or items of a GcStruct or GcArray. They are
# handled by late_initializations_hiddengcref32.
realobj = value._obj.container
realvalue = cast_opaque_ptr(Ptr(typeOf(realobj)), value)
db.get(realvalue) # force following dependencies
- return 'HIDE_INTO_ADR32(???) /* see primitive.py, name_hiddengcref32() */'
+ return 'HIDE_INTO_PTR32(???) /* see primitive.py, name_hiddengcref32() */'
def name_small_integer(value, db):
"""Works for integers of size at most INT or UINT."""
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/src/hiddengcref32.h
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/src/hiddengcref32.h (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/src/hiddengcref32.h Tue Oct 5 19:02:33 2010
@@ -17,13 +17,13 @@
#endif
-#define OP_SHOW_FROM_ADR32(x, r) r = (void*)(((unsigned long)(x)) << 3)
+#define OP_SHOW_FROM_PTR32(x, r) r = (void*)(((unsigned long)(x)) << 3)
-#define OP_HIDE_INTO_ADR32_CHECK(x, r) \
+#define OP_HIDE_INTO_PTR32_CHECK(x, r) \
r = (hiddengcref32_t)(((unsigned long)(x)) >> 3); \
if ((void*)(((unsigned long)(r)) << 3) != (x)) \
RPyPointerTooBig()
-#define OP_HIDE_INTO_ADR32(x, r) \
+#define OP_HIDE_INTO_PTR32(x, r) \
RPyAssert(!(((long)(x)) & 0x7FFFFFFF8), "Pointer too big or misaligned"); \
r = (hiddengcref32_t)(((unsigned long)(x)) >> 3)
From arigo at codespeak.net Tue Oct 5 19:10:30 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 19:10:30 +0200 (CEST)
Subject: [pypy-svn] r77621 - pypy/branch/32ptr-on-64bit/pypy/translator/c/src
Message-ID: <20101005171030.8EDE836C22E@codespeak.net>
Author: arigo
Date: Tue Oct 5 19:10:29 2010
New Revision: 77621
Modified:
pypy/branch/32ptr-on-64bit/pypy/translator/c/src/hiddengcref32.h
Log:
Bah.
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/src/hiddengcref32.h
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/src/hiddengcref32.h (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/src/hiddengcref32.h Tue Oct 5 19:10:29 2010
@@ -25,5 +25,5 @@
RPyPointerTooBig()
#define OP_HIDE_INTO_PTR32(x, r) \
- RPyAssert(!(((long)(x)) & 0x7FFFFFFF8), "Pointer too big or misaligned"); \
+ RPyAssert(!(((long)(x)) & ~0x7FFFFFFF8), "Pointer too big or misaligned"); \
r = (hiddengcref32_t)(((unsigned long)(x)) >> 3)
From arigo at codespeak.net Tue Oct 5 19:10:48 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 19:10:48 +0200 (CEST)
Subject: [pypy-svn] r77622 - in pypy/branch/32ptr-on-64bit/pypy: config
rpython rpython/memory/gctransform
Message-ID: <20101005171048.39D8A282B9C@codespeak.net>
Author: arigo
Date: Tue Oct 5 19:10:46 2010
New Revision: 77622
Modified:
pypy/branch/32ptr-on-64bit/pypy/config/translationoption.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
pypy/branch/32ptr-on-64bit/pypy/rpython/rbuiltin.py
Log:
Fix fix fix...
Modified: pypy/branch/32ptr-on-64bit/pypy/config/translationoption.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/config/translationoption.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/config/translationoption.py Tue Oct 5 19:10:46 2010
@@ -197,7 +197,8 @@
default=False, cmdline="--compressptr",
requires=[("translation.type_system", "lltype"),
("translation.taggedpointers", False),
- ("translation.gc", "minimark")] # for now
+ ("translation.gc", "minimark"), # for now
+ ("translation.gcrootfinder", "shadowstack")] # xxx
+ [("compressptr (64-bit only)", True)]*(not IS_64_BITS)),
# options for ootype
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctransform/framework.py Tue Oct 5 19:10:46 2010
@@ -1166,6 +1166,9 @@
c_k = rmodel.inputconst(lltype.Signed, k)
v_newaddr = hop.genop("raw_load", [base_addr, c_type, c_k],
resulttype=llmemory.Address)
+ if var.concretetype == llmemory.HiddenGcRef32:
+ v_newaddr = gen_cast(hop.llops, llmemory.HiddenGcRef32,
+ v_newaddr)
hop.genop("gc_reload_possibly_moved", [v_newaddr, var])
def compute_borrowed_vars(self, graph):
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/rbuiltin.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/rbuiltin.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/rbuiltin.py Tue Oct 5 19:10:46 2010
@@ -464,12 +464,20 @@
else:
return llops.genop('cast_pointer', [v_value], resulttype = TGT)
elif ORIG == llmemory.Address:
- return llops.genop('cast_adr_to_ptr', [v_value], resulttype = TGT)
+ if TGT == llmemory.HiddenGcRef32:
+ return llops.genop('hide_into_ptr32', [v_value],
+ resulttype = TGT)
+ else:
+ return llops.genop('cast_adr_to_ptr', [v_value],
+ resulttype = TGT)
elif isinstance(ORIG, lltype.Primitive):
v_value = gen_cast(llops, lltype.Signed, v_value)
return llops.genop('cast_int_to_ptr', [v_value], resulttype=TGT)
elif TGT == llmemory.Address and isinstance(ORIG, lltype.Ptr):
- return llops.genop('cast_ptr_to_adr', [v_value], resulttype = TGT)
+ if ORIG == llmemory.HiddenGcRef32:
+ return llops.genop('show_from_ptr32', [v_value], resulttype = TGT)
+ else:
+ return llops.genop('cast_ptr_to_adr', [v_value], resulttype = TGT)
elif isinstance(TGT, lltype.Primitive):
if isinstance(ORIG, lltype.Ptr):
v_value = llops.genop('cast_ptr_to_int', [v_value],
From afa at codespeak.net Tue Oct 5 19:31:41 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 5 Oct 2010 19:31:41 +0200 (CEST)
Subject: [pypy-svn] r77623 - in pypy/branch/fast-forward/pypy/objspace/std:
. test
Message-ID: <20101005173141.783B4282B90@codespeak.net>
Author: afa
Date: Tue Oct 5 19:31:39 2010
New Revision: 77623
Modified:
pypy/branch/fast-forward/pypy/objspace/std/strutil.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_floatobject.py
Log:
Add float('infinity')
Modified: pypy/branch/fast-forward/pypy/objspace/std/strutil.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/strutil.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/strutil.py Tue Oct 5 19:31:39 2010
@@ -178,10 +178,12 @@
low = s.lower()
- if low == "-inf":
+ if low == "-inf" or low == "-infinity":
return -INFINITY
elif low == "inf" or low == "+inf":
return INFINITY
+ elif low == "infinity" or low == "+infinity":
+ return INFINITY
elif low == "nan" or low == "-nan" or low == "+nan":
return NAN
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_floatobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_floatobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_floatobject.py Tue Oct 5 19:31:39 2010
@@ -98,6 +98,9 @@
assert float("-INf") == -inf
assert str(inf) == "inf"
assert str(-inf) == "-inf"
+ assert str(float("infinity")) == 'inf'
+ assert str(float("+infinity")) == 'inf'
+ assert str(float("-infinity")) == '-inf'
assert str(float("nan")) == "nan"
assert str(float("-nAn")) == "nan"
assert repr(inf) == "inf"
From arigo at codespeak.net Tue Oct 5 19:32:19 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 5 Oct 2010 19:32:19 +0200 (CEST)
Subject: [pypy-svn] r77624 - in
pypy/branch/32ptr-on-64bit/pypy/rpython/memory: . gc
Message-ID: <20101005173219.83129282B90@codespeak.net>
Author: arigo
Date: Tue Oct 5 19:32:18 2010
New Revision: 77624
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py
Log:
Hack goes on.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gc/base.py Tue Oct 5 19:32:18 2010
@@ -178,6 +178,7 @@
Typically, 'callback' is a bound method and 'arg' can be None.
"""
typeid = self.get_type_id(obj)
+ # XXX missing performance shortcut for GcArray(HiddenGcRef32)
if self.is_gcarrayofgcptr(typeid):
# a performance shortcut for GcArray(gcptr)
length = (obj + llmemory.gcarrayofptr_lengthoffset).signed[0]
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/gctypelayout.py Tue Oct 5 19:32:18 2010
@@ -131,6 +131,7 @@
T_IS_GCARRAY_OF_GCPTR = 0x40000
T_IS_WEAKREF = 0x80000
T_IS_RPYTHON_INSTANCE = 0x100000 # the type is a subclass of OBJECT
+T_IS_GCARRAY_OF_HIDDENGCREF32 = 0x200000
T_KEY_MASK = intmask(0xFF000000)
T_KEY_VALUE = intmask(0x7A000000) # bug detection only
@@ -175,7 +176,10 @@
ARRAY = TYPE
if (isinstance(ARRAY.OF, lltype.Ptr)
and ARRAY.OF.TO._gckind == 'gc'):
- infobits |= T_IS_GCARRAY_OF_GCPTR
+ if ARRAY.OF == llmemory.HiddenGcRef32:
+ pass # infobits |= T_IS_GCARRAY_OF_HIDDENGCREF32
+ else:
+ infobits |= T_IS_GCARRAY_OF_GCPTR
varinfo.ofstolength = llmemory.ArrayLengthOffset(ARRAY)
varinfo.ofstovar = llmemory.itemoffsetof(TYPE, 0)
assert isinstance(ARRAY, lltype.Array)
From afa at codespeak.net Tue Oct 5 19:54:44 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 5 Oct 2010 19:54:44 +0200 (CEST)
Subject: [pypy-svn] r77625 - in pypy/branch/fast-forward/pypy: objspace/std
objspace/std/test rlib
Message-ID: <20101005175444.8CDAB282B90@codespeak.net>
Author: afa
Date: Tue Oct 5 19:54:42 2010
New Revision: 77625
Modified:
pypy/branch/fast-forward/pypy/objspace/std/floatobject.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_floatobject.py
pypy/branch/fast-forward/pypy/rlib/rarithmetic.py
pypy/branch/fast-forward/pypy/rlib/rbigint.py
Log:
Ensure that int(float('nan')) raises a ValueError
Modified: pypy/branch/fast-forward/pypy/objspace/std/floatobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/floatobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/floatobject.py Tue Oct 5 19:54:42 2010
@@ -74,21 +74,20 @@
try:
return W_LongObject.fromfloat(w_floatobj.floatval)
except OverflowError:
+ if isnan(w_floatobj.floatval):
+ raise OperationError(
+ space.w_ValueError,
+ space.wrap("cannot convert float NaN to integer"))
raise OperationError(space.w_OverflowError,
space.wrap("cannot convert float infinity to long"))
def trunc__Float(space, w_floatobj):
whole = math.modf(w_floatobj.floatval)[1]
try:
- i = ovfcheck_float_to_int(whole)
+ value = ovfcheck_float_to_int(whole)
except OverflowError:
- pass
+ return long__Float(space, w_floatobj)
else:
- return space.wrap(i)
- try:
- return W_LongObject.fromfloat(w_floatobj.floatval)
- except OverflowError:
- raise OperationError(space.w_OverflowError,
- space.wrap("cannot convert infinity to long"))
+ return space.newint(value)
def float_w__Float(space, w_float):
return w_float.floatval
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_floatobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_floatobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_floatobject.py Tue Oct 5 19:54:42 2010
@@ -89,6 +89,11 @@
assert int(42.1234) == 42
assert int(4e10) == 40000000000L
+ raises(OverflowError, int, float('inf'))
+ raises(OverflowError, long, float('inf'))
+ raises(ValueError, int, float('nan'))
+ raises(ValueError, long, float('nan'))
+
def test_float_string(self):
assert 42 == float("42")
assert 42.25 == float("42.25")
Modified: pypy/branch/fast-forward/pypy/rlib/rarithmetic.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rarithmetic.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rarithmetic.py Tue Oct 5 19:54:42 2010
@@ -217,6 +217,8 @@
# successfully be casted to an int.
if sys.maxint == 2147483647:
def ovfcheck_float_to_int(x):
+ if isnan(x):
+ raise OverflowError
if -2147483649.0 < x < 2147483648.0:
return int(x)
raise OverflowError
@@ -225,6 +227,8 @@
# Note the "<= x <" here, as opposed to "< x <" above.
# This is justified by test_typed in translator/c/test.
def ovfcheck_float_to_int(x):
+ if isnan(x):
+ raise OverflowError
if -9223372036854776832.0 <= x < 9223372036854775296.0:
return int(x)
raise OverflowError
Modified: pypy/branch/fast-forward/pypy/rlib/rbigint.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rbigint.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rbigint.py Tue Oct 5 19:54:42 2010
@@ -1,5 +1,5 @@
from pypy.rlib.rarithmetic import LONG_BIT, intmask, r_uint, r_ulonglong
-from pypy.rlib.rarithmetic import ovfcheck, r_longlong, widen
+from pypy.rlib.rarithmetic import ovfcheck, r_longlong, widen, isinf, isnan
from pypy.rlib.debug import make_sure_not_resized
import math, sys
@@ -109,7 +109,7 @@
def fromfloat(dval):
""" Create a new bigint object from a float """
neg = 0
- if isinf(dval):
+ if isinf(dval) or isnan(dval):
raise OverflowError
if dval < 0.0:
neg = 1
@@ -1259,9 +1259,6 @@
assert x > 0.0
return x * sign, exponent
-def isinf(x):
- return x != 0.0 and x / 2 == x
-
##def ldexp(x, exp):
## assert type(x) is float
## lb1 = LONG_BIT - 1
From afa at codespeak.net Tue Oct 5 20:00:58 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 5 Oct 2010 20:00:58 +0200 (CEST)
Subject: [pypy-svn] r77626 - in pypy/branch/fast-forward/pypy/interpreter:
astcompiler astcompiler/test pyparser pyparser/test
Message-ID: <20101005180058.4F1BC282B90@codespeak.net>
Author: afa
Date: Tue Oct 5 20:00:56 2010
New Revision: 77626
Modified:
pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py
pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py
pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py
pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py
pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py
Log:
Add Binary literals: 0b11101
Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py Tue Oct 5 20:00:56 2010
@@ -1037,6 +1037,8 @@
if raw.startswith("0"):
if len(raw) > 2 and raw[1] in "Xx":
base = 16
+ elif len(raw) > 2 and raw[1] in "Bb":
+ base = 2
## elif len(raw) > 2 and raw[1] in "Oo": # Fallback below is enough
## base = 8
elif len(raw) > 1:
@@ -1045,7 +1047,7 @@
i = 0
limit = len(raw) - 1
while i < limit:
- if raw[i] not in "0xXoO":
+ if raw[i] not in "0xXoOBb":
break
i += 1
raw = raw[i:]
Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py Tue Oct 5 20:00:56 2010
@@ -1139,6 +1139,8 @@
assert space.is_true(space.isinstance(n, space.w_int))
for num in ("0o53", "0O53", "0o0000053", "0O00053"):
assert space.eq_w(get_num(num), space.wrap(053))
+ for num in ("0b00101", "0B00101", "0b101", "0B101"):
+ assert space.eq_w(get_num(num), space.wrap(5))
def check_comprehension(self, brackets, ast_type):
def brack(s):
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py Tue Oct 5 20:00:56 2010
@@ -61,11 +61,16 @@
maybe(states, groupStr(states, "oO")),
any(states, groupStr(states, "01234567")),
maybe(states, groupStr(states, "lL")))
+ binNumber = chain(states,
+ newArcPair(states, "0"),
+ groupStr(states, "bB"),
+ any(states, groupStr(states, "01")),
+ maybe(states, groupStr(states, "lL")))
decNumber = chain(states,
groupStr(states, "123456789"),
any(states, makeDigits()),
maybe(states, groupStr(states, "lL")))
- intNumber = group(states, hexNumber, octNumber, decNumber)
+ intNumber = group(states, hexNumber, octNumber, binNumber, decNumber)
# ____________________________________________________________
# Exponents
def makeExp ():
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py Tue Oct 5 20:00:56 2010
@@ -26,7 +26,7 @@
accepts = [True, True, True, True, True, True, True, True,
True, True, False, True, True, True, False, False,
False, False, True, False, False, True, True,
- True, False, True, False, True, False, True,
+ True, True, False, True, False, True, False, True,
False, True, False, False, False, True, False,
False, False, True]
states = [
@@ -109,22 +109,22 @@
'v': 1, 'w': 1, 'x': 1, 'y': 1,
'z': 1},
- {'.': 25, '0': 23, '1': 23, '2': 23,
+ {'.': 26, '0': 23, '1': 23, '2': 23,
'3': 23, '4': 23, '5': 23, '6': 23,
- '7': 23, '8': 24, '9': 24, 'E': 26,
- 'J': 13, 'L': 13, 'O': 22, 'X': 21,
- 'e': 26, 'j': 13, 'l': 13, 'o': 22,
- 'x': 21},
+ '7': 23, '8': 25, '9': 25, 'B': 24,
+ 'E': 27, 'J': 13, 'L': 13, 'O': 22,
+ 'X': 21, 'b': 24, 'e': 27, 'j': 13,
+ 'l': 13, 'o': 22, 'x': 21},
- {'.': 25, '0': 5, '1': 5, '2': 5,
+ {'.': 26, '0': 5, '1': 5, '2': 5,
'3': 5, '4': 5, '5': 5, '6': 5,
- '7': 5, '8': 5, '9': 5, 'E': 26,
- 'J': 13, 'L': 13, 'e': 26, 'j': 13,
+ '7': 5, '8': 5, '9': 5, 'E': 27,
+ 'J': 13, 'L': 13, 'e': 27, 'j': 13,
'l': 13},
- {'0': 27, '1': 27, '2': 27, '3': 27,
- '4': 27, '5': 27, '6': 27, '7': 27,
- '8': 27, '9': 27},
+ {'0': 28, '1': 28, '2': 28, '3': 28,
+ '4': 28, '5': 28, '6': 28, '7': 28,
+ '8': 28, '9': 28},
{'*': 12, '=': 13},
@@ -142,22 +142,22 @@
{'\n': 13},
- {automata.DEFAULT: 19, '\n': 28,
- "'": 29, '\\': 30},
+ {automata.DEFAULT: 19, '\n': 29,
+ "'": 30, '\\': 31},
- {automata.DEFAULT: 20, '\n': 28,
- '"': 31, '\\': 32},
+ {automata.DEFAULT: 20, '\n': 29,
+ '"': 32, '\\': 33},
{'\n': 13, '\r': 14},
- {automata.DEFAULT: 18, '\n': 28,
- '\r': 28},
+ {automata.DEFAULT: 18, '\n': 29,
+ '\r': 29},
- {automata.DEFAULT: 19, '\n': 28,
- "'": 13, '\\': 30},
+ {automata.DEFAULT: 19, '\n': 29,
+ "'": 13, '\\': 31},
- {automata.DEFAULT: 20, '\n': 28,
- '"': 13, '\\': 32},
+ {automata.DEFAULT: 20, '\n': 29,
+ '"': 13, '\\': 33},
{'0': 21, '1': 21, '2': 21, '3': 21,
'4': 21, '5': 21, '6': 21, '7': 21,
@@ -171,71 +171,74 @@
'4': 22, '5': 22, '6': 22, '7': 22,
'L': 13, 'l': 13},
- {'.': 25, '0': 23, '1': 23, '2': 23,
+ {'.': 26, '0': 23, '1': 23, '2': 23,
'3': 23, '4': 23, '5': 23, '6': 23,
- '7': 23, '8': 24, '9': 24, 'E': 26,
- 'J': 13, 'L': 13, 'e': 26, 'j': 13,
+ '7': 23, '8': 25, '9': 25, 'E': 27,
+ 'J': 13, 'L': 13, 'e': 27, 'j': 13,
'l': 13},
- {'.': 25, '0': 24, '1': 24, '2': 24,
- '3': 24, '4': 24, '5': 24, '6': 24,
- '7': 24, '8': 24, '9': 24, 'E': 26,
- 'J': 13, 'e': 26, 'j': 13},
-
- {'0': 25, '1': 25, '2': 25, '3': 25,
- '4': 25, '5': 25, '6': 25, '7': 25,
- '8': 25, '9': 25, 'E': 33, 'J': 13,
- 'e': 33, 'j': 13},
-
- {'+': 34, '-': 34, '0': 35, '1': 35,
- '2': 35, '3': 35, '4': 35, '5': 35,
- '6': 35, '7': 35, '8': 35,
- '9': 35},
-
- {'0': 27, '1': 27, '2': 27, '3': 27,
- '4': 27, '5': 27, '6': 27, '7': 27,
- '8': 27, '9': 27, 'E': 33, 'J': 13,
- 'e': 33, 'j': 13},
+ {'0': 24, '1': 24, 'L': 13,
+ 'l': 13},
+
+ {'.': 26, '0': 25, '1': 25, '2': 25,
+ '3': 25, '4': 25, '5': 25, '6': 25,
+ '7': 25, '8': 25, '9': 25, 'E': 27,
+ 'J': 13, 'e': 27, 'j': 13},
+
+ {'0': 26, '1': 26, '2': 26, '3': 26,
+ '4': 26, '5': 26, '6': 26, '7': 26,
+ '8': 26, '9': 26, 'E': 34, 'J': 13,
+ 'e': 34, 'j': 13},
+
+ {'+': 35, '-': 35, '0': 36, '1': 36,
+ '2': 36, '3': 36, '4': 36, '5': 36,
+ '6': 36, '7': 36, '8': 36,
+ '9': 36},
+
+ {'0': 28, '1': 28, '2': 28, '3': 28,
+ '4': 28, '5': 28, '6': 28, '7': 28,
+ '8': 28, '9': 28, 'E': 34, 'J': 13,
+ 'e': 34, 'j': 13},
{},
{"'": 13},
- {automata.DEFAULT: 36, '\n': 13,
+ {automata.DEFAULT: 37, '\n': 13,
'\r': 14},
{'"': 13},
- {automata.DEFAULT: 37, '\n': 13,
+ {automata.DEFAULT: 38, '\n': 13,
'\r': 14},
- {'+': 38, '-': 38, '0': 39, '1': 39,
- '2': 39, '3': 39, '4': 39, '5': 39,
- '6': 39, '7': 39, '8': 39,
- '9': 39},
-
- {'0': 35, '1': 35, '2': 35, '3': 35,
- '4': 35, '5': 35, '6': 35, '7': 35,
- '8': 35, '9': 35},
-
- {'0': 35, '1': 35, '2': 35, '3': 35,
- '4': 35, '5': 35, '6': 35, '7': 35,
- '8': 35, '9': 35, 'J': 13,
+ {'+': 39, '-': 39, '0': 40, '1': 40,
+ '2': 40, '3': 40, '4': 40, '5': 40,
+ '6': 40, '7': 40, '8': 40,
+ '9': 40},
+
+ {'0': 36, '1': 36, '2': 36, '3': 36,
+ '4': 36, '5': 36, '6': 36, '7': 36,
+ '8': 36, '9': 36},
+
+ {'0': 36, '1': 36, '2': 36, '3': 36,
+ '4': 36, '5': 36, '6': 36, '7': 36,
+ '8': 36, '9': 36, 'J': 13,
'j': 13},
- {automata.DEFAULT: 36, '\n': 28,
- "'": 13, '\\': 30},
+ {automata.DEFAULT: 37, '\n': 29,
+ "'": 13, '\\': 31},
- {automata.DEFAULT: 37, '\n': 28,
- '"': 13, '\\': 32},
+ {automata.DEFAULT: 38, '\n': 29,
+ '"': 13, '\\': 33},
- {'0': 39, '1': 39, '2': 39, '3': 39,
- '4': 39, '5': 39, '6': 39, '7': 39,
- '8': 39, '9': 39},
-
- {'0': 39, '1': 39, '2': 39, '3': 39,
- '4': 39, '5': 39, '6': 39, '7': 39,
- '8': 39, '9': 39, 'J': 13,
+ {'0': 40, '1': 40, '2': 40, '3': 40,
+ '4': 40, '5': 40, '6': 40, '7': 40,
+ '8': 40, '9': 40},
+
+ {'0': 40, '1': 40, '2': 40, '3': 40,
+ '4': 40, '5': 40, '6': 40, '7': 40,
+ '8': 40, '9': 40, 'J': 13,
'j': 13},
]
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py Tue Oct 5 20:00:56 2010
@@ -112,4 +112,10 @@
def test_new_octal_literal(self):
self.parse('0777')
self.parse('0o777')
+ self.parse('0o777L')
py.test.raises(SyntaxError, self.parse, "0o778")
+
+ def test_new_binary_literal(self):
+ self.parse('0b1101')
+ self.parse('0b0l')
+ py.test.raises(SyntaxError, self.parse, "0b112")
From afa at codespeak.net Tue Oct 5 20:12:51 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 5 Oct 2010 20:12:51 +0200 (CEST)
Subject: [pypy-svn] r77627 - in pypy/branch/fast-forward/pypy/objspace/std:
. test
Message-ID: <20101005181251.B9D7F282B90@codespeak.net>
Author: afa
Date: Tue Oct 5 20:12:50 2010
New Revision: 77627
Modified:
pypy/branch/fast-forward/pypy/objspace/std/strutil.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py
Log:
Fix int("0b10", 2) and long("0o77L", 8)
Modified: pypy/branch/fast-forward/pypy/objspace/std/strutil.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/strutil.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/strutil.py Tue Oct 5 20:12:50 2010
@@ -52,7 +52,9 @@
if base == 0:
if s.startswith('0x') or s.startswith('0X'):
base = 16
- elif s.startswith('0'):
+ elif s.startswith('0b') or s.startswith('0B'):
+ base = 2
+ elif s.startswith('0'): # also covers the '0o' case
base = 8
else:
base = 10
@@ -64,6 +66,10 @@
self.error()
if base == 16 and (s.startswith('0x') or s.startswith('0X')):
s = s[2:]
+ if base == 8 and (s.startswith('0o') or s.startswith('0O')):
+ s = s[2:]
+ if base == 2 and (s.startswith('0b') or s.startswith('0B')):
+ s = s[2:]
self.s = s
self.n = len(s)
self.i = 0
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py Tue Oct 5 20:12:50 2010
@@ -52,6 +52,9 @@
('0X', 0, 0), # " "
('0x', 16, 0), # " "
('0X', 16, 0), # " "
+ ('0b11', 2, 3),
+ ('0B10', 2, 2),
+ ('0o77', 8, 63),
]
for s, base, expected in cases:
assert string_to_int(s, base) == expected
@@ -65,6 +68,8 @@
space = self.space
cases = ['0x123', # must use base 0 or 16
' 0X12 ',
+ '0b01',
+ '0o01',
'',
'++12',
'+-12',
From afa at codespeak.net Tue Oct 5 20:13:29 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 5 Oct 2010 20:13:29 +0200 (CEST)
Subject: [pypy-svn] r77628 - pypy/branch/fast-forward/lib-python
Message-ID: <20101005181329.07D05282B90@codespeak.net>
Author: afa
Date: Tue Oct 5 20:13:28 2010
New Revision: 77628
Modified:
pypy/branch/fast-forward/lib-python/TODO
Log:
2 tasks finished
Modified: pypy/branch/fast-forward/lib-python/TODO
==============================================================================
--- pypy/branch/fast-forward/lib-python/TODO (original)
+++ pypy/branch/fast-forward/lib-python/TODO Tue Oct 5 20:13:28 2010
@@ -12,10 +12,6 @@
- Missing builtin: bytearray (possibly reuse module.__pypy__.bytebuffer)
-- Octal literals: 0o777
-
-- float('infinity'), float('nan')
-
- Seen in test_inspect, this has never worked in pypy::
assert eval('a', None, dict(a=42)) == 42
From agaynor at codespeak.net Tue Oct 5 20:35:34 2010
From: agaynor at codespeak.net (agaynor at codespeak.net)
Date: Tue, 5 Oct 2010 20:35:34 +0200 (CEST)
Subject: [pypy-svn] r77629 - pypy/branch/fast-forward/pypy/module/math
Message-ID: <20101005183534.976E5282B90@codespeak.net>
Author: agaynor
Date: Tue Oct 5 20:35:33 2010
New Revision: 77629
Modified:
pypy/branch/fast-forward/pypy/module/math/__init__.py
pypy/branch/fast-forward/pypy/module/math/interp_math.py
Log:
Added exmp1 to math module.
Modified: pypy/branch/fast-forward/pypy/module/math/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/math/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/math/__init__.py Tue Oct 5 20:35:33 2010
@@ -34,6 +34,7 @@
'atan2' : 'interp_math.atan2',
'modf' : 'interp_math.modf',
'exp' : 'interp_math.exp',
+ 'expm1' : 'interp_math.expm1',
'acos' : 'interp_math.acos',
'isinf' : 'interp_math.isinf',
'isnan' : 'interp_math.isnan',
Modified: pypy/branch/fast-forward/pypy/module/math/interp_math.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/math/interp_math.py (original)
+++ pypy/branch/fast-forward/pypy/module/math/interp_math.py Tue Oct 5 20:35:33 2010
@@ -328,6 +328,10 @@
return math1(space, math.exp, w_x)
exp.unwrap_spec = [ObjSpace, W_Root]
+def expm1(space, w_x):
+ return math1(space, math.expm1, w_x)
+expm1.unwrap_spec = [ObjSpace, W_Root]
+
def acos(space, w_x):
"""acos(x)
From david at codespeak.net Tue Oct 5 21:03:51 2010
From: david at codespeak.net (david at codespeak.net)
Date: Tue, 5 Oct 2010 21:03:51 +0200 (CEST)
Subject: [pypy-svn] r77630 - in pypy/branch/arm-backend/pypy/jit/backend:
llsupport x86
Message-ID: <20101005190351.2AF14282B90@codespeak.net>
Author: david
Date: Tue Oct 5 21:03:38 2010
New Revision: 77630
Modified:
pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py
pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py
Log:
move _compute_vars_longevity to llsupport and rename it to compute_vars_longevity
Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py Tue Oct 5 21:03:38 2010
@@ -44,7 +44,7 @@
all_regs = []
no_lower_byte_regs = []
save_around_call_regs = []
-
+
def __init__(self, longevity, frame_manager=None, assembler=None):
self.free_regs = self.all_regs[:]
self.longevity = longevity
@@ -244,7 +244,7 @@
if isinstance(v, Const):
return self.return_constant(v, forbidden_vars, selected_reg,
imm_fine)
-
+
prev_loc = self.loc(v)
loc = self.force_allocate_reg(v, forbidden_vars, selected_reg,
need_lower_byte=need_lower_byte)
@@ -349,3 +349,37 @@
be stored by the cpu, according to the variable type
"""
raise NotImplementedError("Abstract")
+
+def compute_vars_longevity(inputargs, operations):
+ # compute a dictionary that maps variables to index in
+ # operations that is a "last-time-seen"
+ longevity = {}
+ start_live = {}
+ for inputarg in inputargs:
+ start_live[inputarg] = 0
+ for i in range(len(operations)):
+ op = operations[i]
+ if op.result is not None:
+ start_live[op.result] = i
+ for j in range(op.numargs()):
+ arg = op.getarg(j)
+ if isinstance(arg, Box):
+ if arg not in start_live:
+ print "Bogus arg in operation %d at %d" % (op.getopnum(), i)
+ raise AssertionError
+ longevity[arg] = (start_live[arg], i)
+ if op.is_guard():
+ for arg in op.getfailargs():
+ if arg is None: # hole
+ continue
+ assert isinstance(arg, Box)
+ if arg not in start_live:
+ print "Bogus arg in guard %d at %d" % (op.getopnum(), i)
+ raise AssertionError
+ longevity[arg] = (start_live[arg], i)
+ for arg in inputargs:
+ if arg not in longevity:
+ longevity[arg] = (-1, -1)
+ for arg in longevity:
+ assert isinstance(arg, Box)
+ return longevity
Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py Tue Oct 5 21:03:38 2010
@@ -16,7 +16,7 @@
from pypy.jit.backend.llsupport.descr import BaseFieldDescr, BaseArrayDescr
from pypy.jit.backend.llsupport.descr import BaseCallDescr, BaseSizeDescr
from pypy.jit.backend.llsupport.regalloc import FrameManager, RegisterManager,\
- TempBox
+ TempBox, compute_vars_longevity
from pypy.jit.backend.x86.arch import WORD, FRAME_FIXED_SIZE, IS_X86_32, IS_X86_64
class X86RegisterManager(RegisterManager):
@@ -106,7 +106,7 @@
def convert_to_imm(self, c):
const_id, adr = self.float_constants.record_float(c.getfloat())
return ConstFloatLoc(adr, const_id)
-
+
def after_call(self, v):
# the result is stored in st0, but we don't have this around,
# so genop_call will move it to some frame location immediately
@@ -152,7 +152,7 @@
cpu = self.assembler.cpu
cpu.gc_ll_descr.rewrite_assembler(cpu, operations)
# compute longevity of variables
- longevity = self._compute_vars_longevity(inputargs, operations)
+ longevity = compute_vars_longevity(inputargs, operations)
self.longevity = longevity
# XXX
if cpu.WORD == 4:
@@ -163,7 +163,7 @@
xmm_reg_mgr_cls = X86_64_XMMRegisterManager
else:
raise AssertionError("Word size should be 4 or 8")
-
+
self.rm = gpr_reg_mgr_cls(longevity,
frame_manager = self.fm,
assembler = self.assembler)
@@ -173,7 +173,7 @@
def prepare_loop(self, inputargs, operations, looptoken):
self._prepare(inputargs, operations)
jump = operations[-1]
- loop_consts = self._compute_loop_consts(inputargs, jump, looptoken)
+ loop_consts = compute_loop_consts(inputargs, jump, looptoken)
self.loop_consts = loop_consts
return self._process_inputargs(inputargs)
@@ -340,11 +340,11 @@
arglocs))
else:
self.assembler.dump('%s(%s)' % (guard_op, arglocs))
- current_depths = (self.fm.frame_depth, self.param_depth)
+ current_depths = (self.fm.frame_depth, self.param_depth)
self.assembler.regalloc_perform_guard(guard_op, faillocs, arglocs,
result_loc,
current_depths)
- self.possibly_free_vars(guard_op.getfailargs())
+ self.possibly_free_vars(guard_op.getfailargs())
def PerformDiscard(self, op, arglocs):
if not we_are_translated():
@@ -397,39 +397,6 @@
assert not self.rm.reg_bindings
assert not self.xrm.reg_bindings
- def _compute_vars_longevity(self, inputargs, operations):
- # compute a dictionary that maps variables to index in
- # operations that is a "last-time-seen"
- longevity = {}
- start_live = {}
- for inputarg in inputargs:
- start_live[inputarg] = 0
- for i in range(len(operations)):
- op = operations[i]
- if op.result is not None:
- start_live[op.result] = i
- for j in range(op.numargs()):
- arg = op.getarg(j)
- if isinstance(arg, Box):
- if arg not in start_live:
- print "Bogus arg in operation %d at %d" % (op.getopnum(), i)
- raise AssertionError
- longevity[arg] = (start_live[arg], i)
- if op.is_guard():
- for arg in op.getfailargs():
- if arg is None: # hole
- continue
- assert isinstance(arg, Box)
- if arg not in start_live:
- print "Bogus arg in guard %d at %d" % (op.getopnum(), i)
- raise AssertionError
- longevity[arg] = (start_live[arg], i)
- for arg in inputargs:
- if arg not in longevity:
- longevity[arg] = (-1, -1)
- for arg in longevity:
- assert isinstance(arg, Box)
- return longevity
def loc(self, v):
if v is None: # xxx kludgy
@@ -692,7 +659,7 @@
self._call(op, [imm(size), vable] +
[self.loc(op.getarg(i)) for i in range(op.numargs())],
guard_not_forced_op=guard_op)
-
+
def consider_cond_call_gc_wb(self, op):
assert op.result is None
args = op.getarglist()
@@ -1006,13 +973,13 @@
xmmtmploc = self.xrm.force_allocate_reg(box1, selected_reg=xmmtmp)
# Part about non-floats
# XXX we don't need a copy, we only just the original list
- src_locations = [self.loc(op.getarg(i)) for i in range(op.numargs())
+ src_locations = [self.loc(op.getarg(i)) for i in range(op.numargs())
if op.getarg(i).type != FLOAT]
assert tmploc not in nonfloatlocs
dst_locations = [loc for loc in nonfloatlocs if loc is not None]
remap_frame_layout(assembler, src_locations, dst_locations, tmploc)
# Part about floats
- src_locations = [self.loc(op.getarg(i)) for i in range(op.numargs())
+ src_locations = [self.loc(op.getarg(i)) for i in range(op.numargs())
if op.getarg(i).type == FLOAT]
dst_locations = [loc for loc in floatlocs if loc is not None]
remap_frame_layout(assembler, src_locations, dst_locations, xmmtmp)
From david at codespeak.net Tue Oct 5 21:25:14 2010
From: david at codespeak.net (david at codespeak.net)
Date: Tue, 5 Oct 2010 21:25:14 +0200 (CEST)
Subject: [pypy-svn] r77631 - in
pypy/branch/arm-backend/pypy/jit/backend/arm: . test
Message-ID: <20101005192514.395BC282B9C@codespeak.net>
Author: david
Date: Tue Oct 5 21:25:08 2010
New Revision: 77631
Added:
pypy/branch/arm-backend/pypy/jit/backend/arm/
pypy/branch/arm-backend/pypy/jit/backend/arm/__init__.py
pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/conditions.py
pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/
pypy/branch/arm-backend/pypy/jit/backend/arm/test/__init__.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/gen.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
Log:
basic encoding for some arm instructions and infrastructure for testing
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/__init__.py
==============================================================================
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Tue Oct 5 21:25:08 2010
@@ -0,0 +1,79 @@
+import conditions as cond
+from pypy.rlib.rmmap import alloc
+from pypy.rpython.lltypesystem import lltype, rffi
+
+class ARMv7Builder(object):
+
+ def __init__(self):
+ self._data = alloc(1024)
+ self._pos = 0
+
+ def LDR_ri(self, rt, rn, imm=0, cond=cond.AL):
+ # XXX U and P bits are not encoded yet
+ self.write32(cond << 28
+ | 5 << 24
+ | 9 << 20
+ | (rn & 0xF) << 16
+ | (rt & 0xF) << 12
+ | (imm & 0xFFF))
+
+ def ADD_ri(self, rt, rn, imm, cond=cond.AL):
+ # XXX S bit
+ self.write32(cond << 28
+ | 2 << 24
+ | 8 << 20
+ | (rn & 0xF) << 16
+ | (rt & 0xF) << 12
+ | (imm & 0xFFF))
+ def MOV_ri(self, rt, imm=0, cond=cond.AL):
+ # XXX Check the actual allowed size for imm
+ # XXX S bit
+ self.write32(cond << 28
+ | 0x3 << 24
+ | 0xA << 20
+ #| 0x0 << 16
+ | (rt & 0xF) << 12
+ | (imm & 0xFFF))
+
+ def STR_ri(self, rt, rn, imm=0, cond=cond.AL):
+ self.write32(cond << 28
+ | 0x5 << 24
+ | 0x8 << 20
+ | (rn & 0xF) << 16
+ | (rt & 0xF) << 12
+ | (imm & 0xFFF))
+
+ def ASR_ri(self, rd, rm, imm=0, cond=cond.AL, s=0):
+ self.write32(cond << 28
+ | 0xD << 21
+ | (s & 0x1) << 20
+ | (rd & 0xF) << 12
+ | (imm & 0x1F) << 7
+ | 0x4 << 4
+ | (rm & 0xF))
+
+ #XXX encode shifttype correctly
+ def ORR_rr(self, rd, rn, rm, imm=0, cond=cond.AL, s=0, shifttype=0):
+ self.write32(cond << 28
+ | 0x3 << 23
+ | (s & 0x1) << 20
+ | (rn & 0xFF) << 16
+ | (rd & 0xFF) << 12
+ | (imm & 0x1F) << 7
+ | (shifttype & 0x3) << 5
+ | (rm & 0xFF))
+
+ def write32(self, word):
+ self.writechar(chr(word & 0xFF))
+ self.writechar(chr((word >> 8) & 0xFF))
+ self.writechar(chr((word >> 16) & 0xFF))
+ self.writechar(chr((word >> 24) & 0xFF))
+
+ def writechar(self, char):
+ self._data[self._pos] = char
+ self._pos += 1
+
+ def baseaddr(self):
+ return rffi.cast(lltype.Signed, self._data)
+
+
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/conditions.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/conditions.py Tue Oct 5 21:25:08 2010
@@ -0,0 +1,15 @@
+EQ = 0x0
+NE = 0x1
+CS = 0x2
+CC = 0x3
+MI = 0x4
+PL = 0x5
+VS = 0x6
+VC = 0x7
+HI = 0x8
+LS = 0x9
+GE = 0xA
+LT = 0xB
+GT = 0xC
+LE = 0xD
+AL = 0xE
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py Tue Oct 5 21:25:08 2010
@@ -0,0 +1,6 @@
+r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11, r12, r13, r14, r15 = range(16)
+# aliases for registers
+ip = 12
+sp = 13
+lr = 14
+pc = 15
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/test/__init__.py
==============================================================================
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/test/gen.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/gen.py Tue Oct 5 21:25:08 2010
@@ -0,0 +1,52 @@
+import os
+import tempfile
+class ASMInstruction(object):
+
+ if os.uname()[0] == 'Darwin':
+ asm = '~/Code/arm-jit/android/android-ndk-r4b//build/prebuilt/darwin-x86/arm-eabi-4.4.0/arm-eabi/bin/as'
+ else:
+ asm = 'as'
+ asm_opts = '-mcpu=cortex-a8 -march=armv7'
+ body = """.section .text
+.arm
+.global main
+main:
+ .ascii "START"
+ %s
+ .ascii "END"
+"""
+ begin_tag = 'START'
+ end_tag = 'END'
+
+ def __init__(self, instr):
+ self.instr = instr
+ self.file = tempfile.NamedTemporaryFile(mode='w')
+ self.name = self.file.name
+ self.tmpdir = os.path.dirname(self.name)
+
+ def encode(self):
+ f = open("%s/a.out" % (self.tmpdir),'rb')
+ data = f.read()
+ f.close()
+ i = data.find(self.begin_tag)
+ assert i>=0
+ j = data.find(self.end_tag, i)
+ assert j>=0
+ as_code = data[i+len(self.begin_tag):j]
+ return as_code
+
+
+
+ def assemble(self, *args):
+ res = self.body % (self.instr)
+ self.file.write(res)
+ self.file.flush()
+ os.system("%s %s %s -o %s/a.out" % (self.asm, self.asm_opts, self.name, self.tmpdir))
+
+ def __del__(self):
+ self.file.close()
+
+def assemble(instr):
+ a = ASMInstruction(instr)
+ a.assemble(instr)
+ return a.encode()
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Tue Oct 5 21:25:08 2010
@@ -0,0 +1,58 @@
+from pypy.jit.backend.arm import registers as r
+from pypy.jit.backend.arm import codebuilder
+from gen import assemble
+import py
+class CodeBuilder(codebuilder.ARMv7Builder):
+ def __init__(self):
+ self.buffer = []
+
+ def writechar(self, char):
+ self.buffer.append(char)
+
+ def hexdump(self):
+ return ''.join(self.buffer)
+
+class TestInstrCodeBuilder(object):
+ def setup_method(self, ffuu_method):
+ self.cb = CodeBuilder()
+
+ def test_ldr(self):
+ self.cb.LDR_ri(r.r0, r.r1)
+ self.assert_equal("LDR r0, [r1]")
+
+ def test_add_ri(self):
+ self.cb.ADD_ri(r.r0, r.r1, 1)
+ self.assert_equal("ADD r0, r1, #1")
+
+ def test_mov_ri(self):
+ self.cb.MOV_ri(r.r9, 123)
+ self.assert_equal("MOV r9, #123")
+
+ def test_mov_ri2(self):
+ self.cb.MOV_ri(r.r9, 255)
+ self.assert_equal("MOV r9, #255")
+
+ def test_mov_ri_max(self):
+ py.test.skip("Check the actual largest thing")
+ self.cb.MOV_ri(r.r9, 0xFFF)
+ self.assert_equal("MOV r9, #4095")
+
+ def test_str_ri(self):
+ self.cb.STR_ri(r.r9, r.r14)
+ self.assert_equal("STR r9, [r14]")
+
+ def test_asr_ri(self):
+ self.cb.ASR_ri(r.r7, r.r5, 24)
+ self.assert_equal('ASR r7, r5, #24')
+
+ def test_orr_rr_no_shift(self):
+ self.cb.ORR_rr(r.r0, r.r7,r.r12)
+ self.assert_equal('ORR r0, r7, r12')
+
+ def test_orr_rr_lsl_8(self):
+ self.cb.ORR_rr(r.r0, r.r7,r.r12, 8)
+ self.assert_equal('ORR r0, r7, r12, lsl #8')
+
+ def assert_equal(self, asm):
+ assert self.cb.hexdump() == assemble(asm)
+
From david at codespeak.net Tue Oct 5 21:33:11 2010
From: david at codespeak.net (david at codespeak.net)
Date: Tue, 5 Oct 2010 21:33:11 +0200 (CEST)
Subject: [pypy-svn] r77632 - in
pypy/branch/arm-backend/pypy/jit/backend/arm: . test
Message-ID: <20101005193311.D3E16282B9C@codespeak.net>
Author: david
Date: Tue Oct 5 21:33:10 2010
New Revision: 77632
Added:
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py
Log:
simple implementation of cpu and assembler to pass test_compile_linear_loop in runner_test
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Tue Oct 5 21:33:10 2010
@@ -0,0 +1,52 @@
+from pypy.jit.backend.arm.codebuilder import ARMv7Builder
+from pypy.jit.backend.arm import registers as r
+from pypy.jit.backend.arm.regalloc import RegAlloc, ARMRegisterManager
+from pypy.jit.metainterp.resoperation import rop
+from pypy.rpython.lltypesystem import lltype
+# XXX Move to llsupport
+from pypy.jit.backend.x86.support import values_array
+
+
+class AssemblerARM(object):
+
+ def __init__(self, cpu, failargs_limit=1000):
+ self.mc = ARMv7Builder()
+ self.cpu = cpu
+ self.fail_boxes_int = values_array(lltype.Signed, failargs_limit)
+
+ def assemble_loop(self, inputargs, operations, looptoken):
+ assert len(inputargs) == 1
+ reg = 0
+ self.gen_preamble()
+ addr = self.fail_boxes_int.get_addr_for_num(0)
+ self.gen_load_int(r.r3, addr)
+ self.mc.LDR_ri(r.r2, r.r3)
+ for op in operations:
+ if op.getopnum() == rop.INT_ADD:
+ self.mc.ADD_ri(r.r1, r.r2, op.getarg(1).getint())
+ elif op.getopnum() == rop.FINISH:
+ n = self.cpu.get_fail_descr_number(op.getdescr())
+ self.mc.MOV_ri(r.r0, n)
+ self.mc.STR_ri(r.r1, r.r3)
+ self.gen_out()
+
+ def gen_out(self):
+ self.mc.write32(0xe50b3010) # str r3, [fp, #-16]
+ self.mc.write32(0xe51b3010) # ldr r3, [fp, #-16]
+ #self.mc.write32(0xe1a00003) # mov r0, r3
+ self.mc.write32(0xe24bd00c) # sub sp, fp, #12 ; 0xc
+ self.mc.write32(0xe89da800) # ldm sp, {fp, sp, pc}
+
+ def gen_preamble(self):
+ self.mc.write32(0xe1a0c00d) # mov ip, sp
+ self.mc.write32(0xe92dd800) #push {fp, ip, lr, pc}
+ self.mc.write32(0xe24cb004) # sub fp, ip, #4 ; 0x4
+ self.mc.write32(0xe24dd008) #sub sp, sp, #8 ; 0x8
+ self.mc.write32(0xe50b0014) # str r0, [fp, #-20]
+
+ def gen_load_int(self, reg, value):
+ self.mc.MOV_ri(reg, (value & 0xFF))
+
+ for offset in range(8, 25, 8):
+ self.mc.MOV_ri(r.ip, (value >> offset) & 0xFF)
+ self.mc.ORR_rr(reg, reg, r.ip, offset)
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Tue Oct 5 21:33:10 2010
@@ -0,0 +1,43 @@
+from pypy.jit.backend.arm.assembler import AssemblerARM
+from pypy.jit.backend.llsupport.llmodel import AbstractLLCPU
+from pypy.rpython.llinterp import LLInterpreter
+from pypy.rpython.lltypesystem import lltype, rffi
+
+
+class ArmCPU(AbstractLLCPU):
+
+ BOOTSTRAP_TP = lltype.FuncType([], lltype.Signed)
+ supports_floats = False
+
+ def __init__(self, rtyper, stats, opts=None, translate_support_code=False,
+ gcdescr=None):
+ AbstractLLCPU.__init__(self, rtyper, stats, opts,
+ translate_support_code, gcdescr)
+ self.assembler = AssemblerARM(self)
+
+ def compile_loop(self, inputargs, operations, looptoken):
+ self.assembler.assemble_loop(inputargs, operations, looptoken)
+
+ def set_future_value_int(self, index, intvalue):
+ self.assembler.fail_boxes_int.setitem(index, intvalue)
+
+ def get_latest_value_int(self, index):
+ return self.assembler.fail_boxes_int.getitem(index)
+
+ def execute_token(self, executable_token):
+ addr = self.assembler.mc.baseaddr()#executable_token._arm_bootstrap_code
+ assert addr % 8 == 0
+ func = rffi.cast(lltype.Ptr(self.BOOTSTRAP_TP), addr)
+ fail_index = self._execute_call(func)
+ return self.get_fail_descr_from_number(fail_index)
+
+ def _execute_call(self, func):
+ #prev_interpreter = LLInterpreter.current_interpreter
+ #LLInterpreter.current_interpreter = self.debug_ll_interpreter
+ res = 0
+ #try:
+ res = func()
+ #finally:
+ # LLInterpreter.current_interpreter = prev_interpreter
+ return res
+
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py Tue Oct 5 21:33:10 2010
@@ -0,0 +1,13 @@
+from pypy.jit.backend.arm.runner import ArmCPU
+from pypy.jit.backend.test.runner_test import LLtypeBackendTest
+
+class FakeStats(object):
+ pass
+
+class TestARM(LLtypeBackendTest):
+
+ # for the individual tests see
+ # ====> ../../test/runner_test.py
+
+ def setup_method(self, meth):
+ self.cpu = ArmCPU(rtyper=None, stats=FakeStats())
From david at codespeak.net Tue Oct 5 21:34:22 2010
From: david at codespeak.net (david at codespeak.net)
Date: Tue, 5 Oct 2010 21:34:22 +0200 (CEST)
Subject: [pypy-svn] r77633 -
pypy/branch/arm-backend/pypy/jit/backend/arm/test
Message-ID: <20101005193422.A84EE36C222@codespeak.net>
Author: david
Date: Tue Oct 5 21:34:21 2010
New Revision: 77633
Added:
pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py
Log:
integration tests to check generated instructions on arm
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py Tue Oct 5 21:34:21 2010
@@ -0,0 +1,21 @@
+from pypy.rpython.lltypesystem import lltype, rffi
+
+def run_asm(asm):
+ BOOTSTRAP_TP = lltype.FuncType([], lltype.Signed)
+ addr = asm.mc.baseaddr()#executable_token._arm_bootstrap_code
+ assert addr % 8 == 0
+ func = rffi.cast(lltype.Ptr(BOOTSTRAP_TP), addr)
+ return func()
+
+def skip_unless_arm(f):
+ import os
+ def skip_it(*args):
+ import py
+ py.test.skip('only for arm')
+
+ func = skip_it
+ try:
+ if os.uname()[4].index('arm') >= 0:
+ func=f
+ finally:
+ return func
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py Tue Oct 5 21:34:21 2010
@@ -0,0 +1,38 @@
+from pypy.jit.backend.arm import registers as r
+from pypy.jit.backend.arm.assembler import AssemblerARM
+from pypy.jit.backend.arm.test.support import skip_unless_arm, run_asm
+
+class TestRunningAssembler():
+ def setup_method(self, method):
+ self.a = AssemblerARM(None)
+
+ @skip_unless_arm
+ def test_load_small_int_to_reg(self):
+ self.a.gen_preamble()
+ self.a.gen_load_int(r.r0, 123)
+ self.a.gen_out()
+ assert run_asm(self.a) == 123
+
+ @skip_unless_arm
+ def test_load_medium_int_to_reg(self):
+ self.a.gen_preamble()
+ self.a.gen_load_int(r.r0, 0xBBD7)
+ self.a.gen_out()
+ assert run_asm(self.a) == 48087
+
+ @skip_unless_arm
+ def test_load_int_to_reg(self):
+ self.a.gen_preamble()
+ self.a.gen_load_int(r.r0, 0xFFFFFF85)
+ self.a.gen_out()
+ assert run_asm(self.a) == -123
+
+
+ @skip_unless_arm
+ def test_or(self):
+ self.a.gen_preamble()
+ self.a.mc.MOV_ri(r.r1, 8)
+ self.a.mc.MOV_ri(r.r2, 8)
+ self.a.mc.ORR_rr(r.r0, r.r1, r.r2, 4)
+ self.a.gen_out()
+ assert run_asm(self.a) == 0x88
From afa at codespeak.net Tue Oct 5 21:44:50 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 5 Oct 2010 21:44:50 +0200 (CEST)
Subject: [pypy-svn] r77634 -
pypy/branch/fast-forward/pypy/interpreter/pyparser/test
Message-ID: <20101005194450.0C351282B90@codespeak.net>
Author: afa
Date: Tue Oct 5 21:44:49 2010
New Revision: 77634
Modified:
pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py
Log:
Add a failing test
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py Tue Oct 5 21:44:49 2010
@@ -100,6 +100,10 @@
tree = self.parse("x = 43", "single")
assert tree.type == syms.single_input
+ def test_multiline_string(self):
+ self.parse("''' \n '''")
+ self.parse("r''' \n '''")
+
def test_bytes_literal(self):
self.parse('b" "')
self.parse('br" "')
From afa at codespeak.net Tue Oct 5 22:01:25 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 5 Oct 2010 22:01:25 +0200 (CEST)
Subject: [pypy-svn] r77635 -
pypy/branch/fast-forward/pypy/interpreter/pyparser
Message-ID: <20101005200125.4DD7D282B9C@codespeak.net>
Author: afa
Date: Tue Oct 5 22:01:23 2010
New Revision: 77635
Modified:
pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py
pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py
Log:
Fix an error in the tokenizer's grammar:
triple-quoted string with prefix were not correctly decoded
r""" """
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py Tue Oct 5 22:01:23 2010
@@ -170,7 +170,7 @@
group(states,
newArcPair(states, '"'),
makeLineCont())))
- triple = group(states,
+ triple = chain(states,
makeStrPrefix(),
group(states,
chainStr(states, "'''"),
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py Tue Oct 5 22:01:23 2010
@@ -23,11 +23,11 @@
# ______________________________________________________________________
# Automatically generated DFA's
-accepts = [True, True, True, True, True, True, True, True,
+accepts = [False, True, True, True, True, True, True, True,
True, True, False, True, True, True, False, False,
- False, False, True, False, False, True, True,
- True, True, False, True, False, True, False, True,
- False, True, False, False, False, True, False,
+ False, False, True, True, True, True, True, False,
+ True, False, True, False, True, False, False,
+ True, False, False, False, False, True, False,
False, False, True]
states = [
{'\t': 0, '\n': 13, '\x0c': 0,
@@ -73,7 +73,7 @@
't': 1, 'u': 1, 'v': 1, 'w': 1,
'x': 1, 'y': 1, 'z': 1},
- {'"': 20, "'": 19, '0': 1, '1': 1,
+ {'"': 16, "'": 15, '0': 1, '1': 1,
'2': 1, '3': 1, '4': 1, '5': 1,
'6': 1, '7': 1, '8': 1, '9': 1,
'A': 1, 'B': 1, 'C': 1, 'D': 1,
@@ -91,7 +91,7 @@
'v': 1, 'w': 1, 'x': 1, 'y': 1,
'z': 1},
- {'"': 20, "'": 19, '0': 1, '1': 1,
+ {'"': 16, "'": 15, '0': 1, '1': 1,
'2': 1, '3': 1, '4': 1, '5': 1,
'6': 1, '7': 1, '8': 1, '9': 1,
'A': 1, 'B': 1, 'C': 1, 'D': 1,
@@ -109,22 +109,22 @@
'v': 1, 'w': 1, 'x': 1, 'y': 1,
'z': 1},
- {'.': 26, '0': 23, '1': 23, '2': 23,
- '3': 23, '4': 23, '5': 23, '6': 23,
- '7': 23, '8': 25, '9': 25, 'B': 24,
- 'E': 27, 'J': 13, 'L': 13, 'O': 22,
- 'X': 21, 'b': 24, 'e': 27, 'j': 13,
- 'l': 13, 'o': 22, 'x': 21},
+ {'.': 24, '0': 21, '1': 21, '2': 21,
+ '3': 21, '4': 21, '5': 21, '6': 21,
+ '7': 21, '8': 23, '9': 23, 'B': 22,
+ 'E': 25, 'J': 13, 'L': 13, 'O': 20,
+ 'X': 19, 'b': 22, 'e': 25, 'j': 13,
+ 'l': 13, 'o': 20, 'x': 19},
- {'.': 26, '0': 5, '1': 5, '2': 5,
+ {'.': 24, '0': 5, '1': 5, '2': 5,
'3': 5, '4': 5, '5': 5, '6': 5,
- '7': 5, '8': 5, '9': 5, 'E': 27,
- 'J': 13, 'L': 13, 'e': 27, 'j': 13,
+ '7': 5, '8': 5, '9': 5, 'E': 25,
+ 'J': 13, 'L': 13, 'e': 25, 'j': 13,
'l': 13},
- {'0': 28, '1': 28, '2': 28, '3': 28,
- '4': 28, '5': 28, '6': 28, '7': 28,
- '8': 28, '9': 28},
+ {'0': 26, '1': 26, '2': 26, '3': 26,
+ '4': 26, '5': 26, '6': 26, '7': 26,
+ '8': 26, '9': 26},
{'*': 12, '=': 13},
@@ -142,52 +142,46 @@
{'\n': 13},
- {automata.DEFAULT: 19, '\n': 29,
- "'": 30, '\\': 31},
+ {automata.DEFAULT: 30, '\n': 27,
+ "'": 28, '\\': 29},
- {automata.DEFAULT: 20, '\n': 29,
- '"': 32, '\\': 33},
+ {automata.DEFAULT: 33, '\n': 27,
+ '"': 31, '\\': 32},
{'\n': 13, '\r': 14},
- {automata.DEFAULT: 18, '\n': 29,
- '\r': 29},
-
- {automata.DEFAULT: 19, '\n': 29,
- "'": 13, '\\': 31},
+ {automata.DEFAULT: 18, '\n': 27,
+ '\r': 27},
- {automata.DEFAULT: 20, '\n': 29,
- '"': 13, '\\': 33},
-
- {'0': 21, '1': 21, '2': 21, '3': 21,
- '4': 21, '5': 21, '6': 21, '7': 21,
- '8': 21, '9': 21, 'A': 21, 'B': 21,
- 'C': 21, 'D': 21, 'E': 21, 'F': 21,
- 'L': 13, 'a': 21, 'b': 21, 'c': 21,
- 'd': 21, 'e': 21, 'f': 21,
+ {'0': 19, '1': 19, '2': 19, '3': 19,
+ '4': 19, '5': 19, '6': 19, '7': 19,
+ '8': 19, '9': 19, 'A': 19, 'B': 19,
+ 'C': 19, 'D': 19, 'E': 19, 'F': 19,
+ 'L': 13, 'a': 19, 'b': 19, 'c': 19,
+ 'd': 19, 'e': 19, 'f': 19,
'l': 13},
- {'0': 22, '1': 22, '2': 22, '3': 22,
- '4': 22, '5': 22, '6': 22, '7': 22,
+ {'0': 20, '1': 20, '2': 20, '3': 20,
+ '4': 20, '5': 20, '6': 20, '7': 20,
'L': 13, 'l': 13},
- {'.': 26, '0': 23, '1': 23, '2': 23,
- '3': 23, '4': 23, '5': 23, '6': 23,
- '7': 23, '8': 25, '9': 25, 'E': 27,
- 'J': 13, 'L': 13, 'e': 27, 'j': 13,
+ {'.': 24, '0': 21, '1': 21, '2': 21,
+ '3': 21, '4': 21, '5': 21, '6': 21,
+ '7': 21, '8': 23, '9': 23, 'E': 25,
+ 'J': 13, 'L': 13, 'e': 25, 'j': 13,
'l': 13},
- {'0': 24, '1': 24, 'L': 13,
+ {'0': 22, '1': 22, 'L': 13,
'l': 13},
- {'.': 26, '0': 25, '1': 25, '2': 25,
- '3': 25, '4': 25, '5': 25, '6': 25,
- '7': 25, '8': 25, '9': 25, 'E': 27,
- 'J': 13, 'e': 27, 'j': 13},
+ {'.': 24, '0': 23, '1': 23, '2': 23,
+ '3': 23, '4': 23, '5': 23, '6': 23,
+ '7': 23, '8': 23, '9': 23, 'E': 25,
+ 'J': 13, 'e': 25, 'j': 13},
- {'0': 26, '1': 26, '2': 26, '3': 26,
- '4': 26, '5': 26, '6': 26, '7': 26,
- '8': 26, '9': 26, 'E': 34, 'J': 13,
+ {'0': 24, '1': 24, '2': 24, '3': 24,
+ '4': 24, '5': 24, '6': 24, '7': 24,
+ '8': 24, '9': 24, 'E': 34, 'J': 13,
'e': 34, 'j': 13},
{'+': 35, '-': 35, '0': 36, '1': 36,
@@ -195,9 +189,9 @@
'6': 36, '7': 36, '8': 36,
'9': 36},
- {'0': 28, '1': 28, '2': 28, '3': 28,
- '4': 28, '5': 28, '6': 28, '7': 28,
- '8': 28, '9': 28, 'E': 34, 'J': 13,
+ {'0': 26, '1': 26, '2': 26, '3': 26,
+ '4': 26, '5': 26, '6': 26, '7': 26,
+ '8': 26, '9': 26, 'E': 34, 'J': 13,
'e': 34, 'j': 13},
{},
@@ -207,11 +201,17 @@
{automata.DEFAULT: 37, '\n': 13,
'\r': 14},
+ {automata.DEFAULT: 30, '\n': 27,
+ "'": 13, '\\': 29},
+
{'"': 13},
{automata.DEFAULT: 38, '\n': 13,
'\r': 14},
+ {automata.DEFAULT: 33, '\n': 27,
+ '"': 13, '\\': 32},
+
{'+': 39, '-': 39, '0': 40, '1': 40,
'2': 40, '3': 40, '4': 40, '5': 40,
'6': 40, '7': 40, '8': 40,
@@ -226,11 +226,11 @@
'8': 36, '9': 36, 'J': 13,
'j': 13},
- {automata.DEFAULT: 37, '\n': 29,
- "'": 13, '\\': 31},
+ {automata.DEFAULT: 37, '\n': 27,
+ "'": 13, '\\': 29},
- {automata.DEFAULT: 38, '\n': 29,
- '"': 13, '\\': 33},
+ {automata.DEFAULT: 38, '\n': 27,
+ '"': 13, '\\': 32},
{'0': 40, '1': 40, '2': 40, '3': 40,
'4': 40, '5': 40, '6': 40, '7': 40,
From afa at codespeak.net Tue Oct 5 23:14:00 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 5 Oct 2010 23:14:00 +0200 (CEST)
Subject: [pypy-svn] r77636 - in
pypy/branch/fast-forward/pypy/interpreter/astcompiler: . test
Message-ID: <20101005211400.0E17D282B90@codespeak.net>
Author: afa
Date: Tue Oct 5 23:13:59 2010
New Revision: 77636
Modified:
pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py
pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py
Log:
Properly skip the number prefix: in base 16, don't strip the digit 'b'!
Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py Tue Oct 5 23:13:59 2010
@@ -1047,7 +1047,11 @@
i = 0
limit = len(raw) - 1
while i < limit:
- if raw[i] not in "0xXoOBb":
+ if base == 16 and raw[i] not in "0xX":
+ break
+ if base == 8 and raw[i] not in "0oO":
+ break
+ if base == 2 and raw[i] not in "0bB":
break
i += 1
raw = raw[i:]
Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py Tue Oct 5 23:13:59 2010
@@ -1129,6 +1129,7 @@
assert space.eq_w(get_num("00053"), space.wrap(053))
for num in ("0x53", "0X53", "0x0000053", "0X00053"):
assert space.eq_w(get_num(num), space.wrap(0x53))
+ assert space.eq_w(get_num("0Xb0d2"), space.wrap(0xb0d2))
assert space.eq_w(get_num("0X53"), space.wrap(0x53))
assert space.eq_w(get_num("0"), space.wrap(0))
assert space.eq_w(get_num("00000"), space.wrap(0))
From afa at codespeak.net Wed Oct 6 00:13:40 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 6 Oct 2010 00:13:40 +0200 (CEST)
Subject: [pypy-svn] r77637 - pypy/branch/fast-forward/lib_pypy
Message-ID: <20101005221340.E87E0282B9C@codespeak.net>
Author: afa
Date: Wed Oct 6 00:13:39 2010
New Revision: 77637
Modified:
pypy/branch/fast-forward/lib_pypy/hashlib.py
Log:
hashlib.algorithms should be a tuple
Modified: pypy/branch/fast-forward/lib_pypy/hashlib.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/hashlib.py (original)
+++ pypy/branch/fast-forward/lib_pypy/hashlib.py Wed Oct 6 00:13:39 2010
@@ -108,7 +108,7 @@
new = __hash_new
-algorithms = ['md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512']
+algorithms = ('md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512')
def __getfunc(name):
def new(string=''):
From antocuni at codespeak.net Wed Oct 6 11:19:42 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Wed, 6 Oct 2010 11:19:42 +0200 (CEST)
Subject: [pypy-svn] r77642 - in pypy/branch/jitffi/pypy: jit/backend/llgraph
rlib rlib/test rpython/lltypesystem
Message-ID: <20101006091942.A7BAD282B90@codespeak.net>
Author: antocuni
Date: Wed Oct 6 11:19:40 2010
New Revision: 77642
Modified:
pypy/branch/jitffi/pypy/jit/backend/llgraph/llimpl.py
pypy/branch/jitffi/pypy/rlib/libffi.py
pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
pypy/branch/jitffi/pypy/rpython/lltypesystem/lltype.py
Log:
add support for passing pointers as arguments to ffi calls
Modified: pypy/branch/jitffi/pypy/jit/backend/llgraph/llimpl.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/backend/llgraph/llimpl.py (original)
+++ pypy/branch/jitffi/pypy/jit/backend/llgraph/llimpl.py Wed Oct 6 11:19:40 2010
@@ -1018,6 +1018,9 @@
if isinstance(TYPE, lltype.Ptr):
if isinstance(x, (int, long, llmemory.AddressAsInt)):
x = llmemory.cast_int_to_adr(x)
+ if TYPE is rffi.VOIDP:
+ # assume that we want a "C-style" cast, without typechecking the value
+ return rffi.cast(TYPE, x)
return llmemory.cast_adr_to_ptr(x, TYPE)
elif TYPE == llmemory.Address:
if isinstance(x, (int, long, llmemory.AddressAsInt)):
Modified: pypy/branch/jitffi/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/libffi.py Wed Oct 6 11:19:40 2010
@@ -1,5 +1,5 @@
from pypy.rpython.lltypesystem import rffi, lltype
-from pypy.rlib.objectmodel import specialize, enforceargs
+from pypy.rlib.objectmodel import specialize, enforceargs, we_are_translated
from pypy.rlib.rarithmetic import intmask, r_uint
from pypy.rlib import jit
from pypy.rlib import clibffi
@@ -35,6 +35,8 @@
@specialize.arg(0)
def _fits_into_long(TYPE):
+ if isinstance(TYPE, lltype.Ptr):
+ return True # pointers always fits into longs
if not isinstance(TYPE, lltype.Primitive):
return False
if TYPE is lltype.Void or TYPE is rffi.FLOAT or TYPE is rffi.DOUBLE:
@@ -44,6 +46,17 @@
# ======================================================================
+ at specialize.memo()
+def _check_type(TYPE):
+ if isinstance(TYPE, lltype.Ptr):
+ if TYPE.TO._gckind != 'raw':
+ raise TypeError, "Can only push raw values to C, not 'gc'"
+ # XXX probably we should recursively check for struct fields here,
+ # lets just ignore that for now
+ if isinstance(TYPE.TO, lltype.Array) and 'nolength' not in TYPE.TO._hints:
+ raise TypeError, "Can only push to C arrays without length info"
+
+
class ArgChain(object):
first = None
last = None
@@ -52,6 +65,7 @@
@specialize.argtype(1)
def arg(self, val):
TYPE = lltype.typeOf(val)
+ _check_type(TYPE)
if _fits_into_long(TYPE):
cls = IntArg
val = rffi.cast(rffi.LONG, val)
Modified: pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_libffi.py Wed Oct 6 11:19:40 2010
@@ -25,6 +25,18 @@
floatarg = intarg.next
assert floatarg.floatval == 123.45
+ def test_wrong_args(self):
+ # so far the test passes but for the wrong reason :-), i.e. because
+ # .arg() only supports integers and floats
+ chain = ArgChain()
+ x = lltype.malloc(lltype.GcStruct('xxx'))
+ y = lltype.malloc(lltype.GcArray(rffi.LONG), 3)
+ z = lltype.malloc(lltype.Array(rffi.LONG), 4, flavor='raw')
+ py.test.raises(TypeError, "chain.arg(x)")
+ py.test.raises(TypeError, "chain.arg(y)")
+ py.test.raises(TypeError, "chain.arg(z)")
+ lltype.free(z, flavor='raw')
+
def test_library_open(self):
lib = self.get_libc()
del lib
@@ -159,3 +171,27 @@
func = (libfoo, 'many_args', [types.uchar, types.sint], types.sint)
res = self.call(func, [chr(20), 22], rffi.LONG)
assert res == 42
+
+ def test_call_time(self):
+ import time
+ libc = self.get_libc()
+ # XXX assume time_t is long
+ # XXX: on msvcr80 the name of the function is _time32, fix it in that case
+ func = (libc, 'time', [types.pointer], types.ulong)
+ LONGP = rffi.CArray(rffi.LONG)
+ null = lltype.nullptr(LONGP)
+ t0 = self.call(func, [null], rffi.LONG)
+ time.sleep(1)
+ t1 = self.call(func, [null], rffi.LONG)
+ assert t1 > t0
+ #
+ ptr_result = lltype.malloc(LONGP, 1, flavor='raw')
+ t2 = self.call(func, [ptr_result], rffi.LONG)
+ assert ptr_result[0] == t2
+ lltype.free(ptr_result, flavor='raw')
+ if self.__class__ is TestLibffiCall:
+ # the test does not make sense when run with the JIT through
+ # meta_interp, because the __del__ are not properly called (hence
+ # we "leak" memory)
+ del libc
+ assert not ALLOCATED
Modified: pypy/branch/jitffi/pypy/rpython/lltypesystem/lltype.py
==============================================================================
--- pypy/branch/jitffi/pypy/rpython/lltypesystem/lltype.py (original)
+++ pypy/branch/jitffi/pypy/rpython/lltypesystem/lltype.py Wed Oct 6 11:19:40 2010
@@ -794,6 +794,8 @@
return llmemory.cast_adr_to_ptr(value, TGT)
elif TGT == llmemory.Address and isinstance(ORIG, Ptr):
return llmemory.cast_ptr_to_adr(value)
+ elif TGT == Signed and isinstance(ORIG, Ptr) and ORIG.TO._gckind == 'raw':
+ return llmemory.cast_adr_to_int(llmemory.cast_ptr_to_adr(value), 'symbolic')
raise TypeError("don't know how to cast from %r to %r" % (ORIG, TGT))
From antocuni at codespeak.net Wed Oct 6 12:23:18 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Wed, 6 Oct 2010 12:23:18 +0200 (CEST)
Subject: [pypy-svn] r77643 - in pypy/branch/jitffi/pypy:
jit/backend/llsupport jit/metainterp rlib/test
Message-ID: <20101006102318.6148D36C22E@codespeak.net>
Author: antocuni
Date: Wed Oct 6 12:23:16 2010
New Revision: 77643
Modified:
pypy/branch/jitffi/pypy/jit/backend/llsupport/ffisupport.py
pypy/branch/jitffi/pypy/jit/metainterp/warmstate.py
pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
Log:
add support for returning pointers
Modified: pypy/branch/jitffi/pypy/jit/backend/llsupport/ffisupport.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/backend/llsupport/ffisupport.py (original)
+++ pypy/branch/jitffi/pypy/jit/backend/llsupport/ffisupport.py Wed Oct 6 12:23:16 2010
@@ -29,7 +29,7 @@
if ffi_type is types.void:
return history.VOID
elif ffi_type is types.pointer:
- return history.REF
+ return history.INT
elif ffi_type is types.double:
return history.FLOAT
elif ffi_type is types.uchar:
Modified: pypy/branch/jitffi/pypy/jit/metainterp/warmstate.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/warmstate.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/warmstate.py Wed Oct 6 12:23:16 2010
@@ -1,5 +1,5 @@
import sys
-from pypy.rpython.lltypesystem import lltype, llmemory, rstr
+from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.annlowlevel import hlstr, llstr, cast_base_ptr_to_instance
from pypy.rpython.annlowlevel import cast_object_to_ptr
@@ -23,7 +23,11 @@
"""
INPUT = lltype.typeOf(x)
if INPUT is lltype.Signed:
- return lltype.cast_primitive(TYPE, x) # XXX missing: Ptr(non-gc)
+ if isinstance(TYPE, lltype.Ptr) and TYPE.TO._gckind == 'raw':
+ # non-gc pointer
+ return rffi.cast(TYPE, x)
+ else:
+ return lltype.cast_primitive(TYPE, x)
elif INPUT is lltype.Float:
assert TYPE is lltype.Float
return x
Modified: pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_libffi.py Wed Oct 6 12:23:16 2010
@@ -195,3 +195,24 @@
# we "leak" memory)
del libc
assert not ALLOCATED
+
+ def test_return_pointer(self):
+ """
+ struct pair {
+ long a;
+ long b;
+ };
+
+ struct pair my_static_pair = {10, 20};
+
+ long* get_pointer_to_b()
+ {
+ return &my_static_pair.b;
+ }
+ """
+ libfoo = self.get_libfoo()
+ func = (libfoo, 'get_pointer_to_b', [], types.pointer)
+ LONGP = lltype.Ptr(rffi.CArray(rffi.LONG))
+ null = lltype.nullptr(LONGP.TO)
+ res = self.call(func, [], LONGP, init_result=null)
+ assert res[0] == 20
From david at codespeak.net Wed Oct 6 14:58:44 2010
From: david at codespeak.net (david at codespeak.net)
Date: Wed, 6 Oct 2010 14:58:44 +0200 (CEST)
Subject: [pypy-svn] r77645 - in pypy/branch/arm-backend/pypy/jit/backend:
llsupport x86
Message-ID: <20101006125844.B527B36C22E@codespeak.net>
Author: david
Date: Wed Oct 6 14:58:42 2010
New Revision: 77645
Modified:
pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py
pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py
Log:
finish moving compute_loop_consts to llsupport
Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py Wed Oct 6 14:58:42 2010
@@ -383,3 +383,14 @@
for arg in longevity:
assert isinstance(arg, Box)
return longevity
+
+def compute_loop_consts(self, inputargs, jump, looptoken):
+ if jump.getopnum() != rop.JUMP or jump.getdescr() is not looptoken:
+ loop_consts = {}
+ else:
+ loop_consts = {}
+ for i in range(len(inputargs)):
+ if inputargs[i] is jump.getarg(i):
+ loop_consts[inputargs[i]] = i
+ return loop_consts
+
Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py Wed Oct 6 14:58:42 2010
@@ -16,7 +16,7 @@
from pypy.jit.backend.llsupport.descr import BaseFieldDescr, BaseArrayDescr
from pypy.jit.backend.llsupport.descr import BaseCallDescr, BaseSizeDescr
from pypy.jit.backend.llsupport.regalloc import FrameManager, RegisterManager,\
- TempBox, compute_vars_longevity
+ TempBox, compute_vars_longevity, compute_loop_consts
from pypy.jit.backend.x86.arch import WORD, FRAME_FIXED_SIZE, IS_X86_32, IS_X86_64
class X86RegisterManager(RegisterManager):
@@ -267,16 +267,6 @@
return self.rm.force_allocate_reg(var, forbidden_vars,
selected_reg, need_lower_byte)
- def _compute_loop_consts(self, inputargs, jump, looptoken):
- if jump.getopnum() != rop.JUMP or jump.getdescr() is not looptoken:
- loop_consts = {}
- else:
- loop_consts = {}
- for i in range(len(inputargs)):
- if inputargs[i] is jump.getarg(i):
- loop_consts[inputargs[i]] = i
- return loop_consts
-
def _update_bindings(self, locs, inputargs):
# XXX this should probably go to llsupport/regalloc.py
used = {}
From david at codespeak.net Wed Oct 6 15:07:34 2010
From: david at codespeak.net (david at codespeak.net)
Date: Wed, 6 Oct 2010 15:07:34 +0200 (CEST)
Subject: [pypy-svn] r77646 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101006130734.D9CE436C22E@codespeak.net>
Author: david
Date: Wed Oct 6 15:07:33 2010
New Revision: 77646
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
Log:
remove currently unused import
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Wed Oct 6 15:07:33 2010
@@ -1,6 +1,6 @@
from pypy.jit.backend.arm.codebuilder import ARMv7Builder
from pypy.jit.backend.arm import registers as r
-from pypy.jit.backend.arm.regalloc import RegAlloc, ARMRegisterManager
+#from pypy.jit.backend.arm.regalloc import RegAlloc, ARMRegisterManager
from pypy.jit.metainterp.resoperation import rop
from pypy.rpython.lltypesystem import lltype
# XXX Move to llsupport
From david at codespeak.net Wed Oct 6 15:07:59 2010
From: david at codespeak.net (david at codespeak.net)
Date: Wed, 6 Oct 2010 15:07:59 +0200 (CEST)
Subject: [pypy-svn] r77647 -
pypy/branch/arm-backend/pypy/jit/backend/arm/test
Message-ID: <20101006130759.CD2F136C22E@codespeak.net>
Author: david
Date: Wed Oct 6 15:07:58 2010
New Revision: 77647
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py
Log:
move platform based skipping to the module level
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py Wed Oct 6 15:07:58 2010
@@ -7,15 +7,15 @@
func = rffi.cast(lltype.Ptr(BOOTSTRAP_TP), addr)
return func()
-def skip_unless_arm(f):
+def skip_unless_arm():
import os
- def skip_it(*args):
- import py
- py.test.skip('only for arm')
+ import py
- func = skip_it
+ skip = True
try:
if os.uname()[4].index('arm') >= 0:
- func=f
+ skip = False
finally:
- return func
+ if skip:
+ py.test.skip('only for arm')
+
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py Wed Oct 6 15:07:58 2010
@@ -2,25 +2,24 @@
from pypy.jit.backend.arm.assembler import AssemblerARM
from pypy.jit.backend.arm.test.support import skip_unless_arm, run_asm
+skip_unless_arm()
+
class TestRunningAssembler():
def setup_method(self, method):
self.a = AssemblerARM(None)
- @skip_unless_arm
def test_load_small_int_to_reg(self):
self.a.gen_preamble()
self.a.gen_load_int(r.r0, 123)
self.a.gen_out()
assert run_asm(self.a) == 123
- @skip_unless_arm
def test_load_medium_int_to_reg(self):
self.a.gen_preamble()
self.a.gen_load_int(r.r0, 0xBBD7)
self.a.gen_out()
assert run_asm(self.a) == 48087
- @skip_unless_arm
def test_load_int_to_reg(self):
self.a.gen_preamble()
self.a.gen_load_int(r.r0, 0xFFFFFF85)
@@ -28,7 +27,6 @@
assert run_asm(self.a) == -123
- @skip_unless_arm
def test_or(self):
self.a.gen_preamble()
self.a.mc.MOV_ri(r.r1, 8)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py Wed Oct 6 15:07:58 2010
@@ -1,5 +1,8 @@
from pypy.jit.backend.arm.runner import ArmCPU
from pypy.jit.backend.test.runner_test import LLtypeBackendTest
+from pypy.jit.backend.arm.test.support import skip_unless_arm
+
+skip_unless_arm()
class FakeStats(object):
pass
From antocuni at codespeak.net Wed Oct 6 15:13:20 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Wed, 6 Oct 2010 15:13:20 +0200 (CEST)
Subject: [pypy-svn] r77648 - pypy/branch/jitffi/pypy/rlib/test
Message-ID: <20101006131320.30C5736C22E@codespeak.net>
Author: antocuni
Date: Wed Oct 6 15:13:18 2010
New Revision: 77648
Modified:
pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
Log:
use an ad-hoc function instead of time: it tests the same behaviour, but at least we don't have to sleep() for 1 second waiting for the time to change
Modified: pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_libffi.py Wed Oct 6 15:13:18 2010
@@ -172,29 +172,43 @@
res = self.call(func, [chr(20), 22], rffi.LONG)
assert res == 42
- def test_call_time(self):
- import time
- libc = self.get_libc()
- # XXX assume time_t is long
- # XXX: on msvcr80 the name of the function is _time32, fix it in that case
- func = (libc, 'time', [types.pointer], types.ulong)
- LONGP = rffi.CArray(rffi.LONG)
- null = lltype.nullptr(LONGP)
- t0 = self.call(func, [null], rffi.LONG)
- time.sleep(1)
- t1 = self.call(func, [null], rffi.LONG)
- assert t1 > t0
+ def test_pointer_as_argument(self):
+ """#include
Make sure you installed the dependencies. See the list here.
+Enter the goal directory:
cd pypy-trunk/pypy/translator/goal
From cfbolz at codespeak.net Wed Oct 13 15:22:15 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Wed, 13 Oct 2010 15:22:15 +0200 (CEST)
Subject: [pypy-svn] r77871 - pypy/extradoc/talk/pepm2011/benchmarks
Message-ID: <20101013132215.55375282BDA@codespeak.net>
Author: cfbolz
Date: Wed Oct 13 15:22:13 2010
New Revision: 77871
Added:
pypy/extradoc/talk/pepm2011/benchmarks/twisted_names.trace.bz2 (contents, props changed)
Removed:
pypy/extradoc/talk/pepm2011/benchmarks/twisted-names.trace.bz2
Modified:
pypy/extradoc/talk/pepm2011/benchmarks/bench.gnumeric
pypy/extradoc/talk/pepm2011/benchmarks/crypto_pyaes.py.trace.bz2
pypy/extradoc/talk/pepm2011/benchmarks/django.trace.bz2
pypy/extradoc/talk/pepm2011/benchmarks/fannkuch.py.trace.bz2
pypy/extradoc/talk/pepm2011/benchmarks/go.py.trace.bz2
pypy/extradoc/talk/pepm2011/benchmarks/html5lib.trace.bz2
pypy/extradoc/talk/pepm2011/benchmarks/meteor-contest.py.trace.bz2
pypy/extradoc/talk/pepm2011/benchmarks/nbody_modified.py.trace.bz2
pypy/extradoc/talk/pepm2011/benchmarks/pyflate-fast.py.trace.bz2
pypy/extradoc/talk/pepm2011/benchmarks/raytrace-simple.py.trace.bz2
pypy/extradoc/talk/pepm2011/benchmarks/richards.trace.bz2
pypy/extradoc/talk/pepm2011/benchmarks/spambayes.trace.bz2
pypy/extradoc/talk/pepm2011/benchmarks/spectral-norm.py.trace.bz2
pypy/extradoc/talk/pepm2011/benchmarks/telco.py.trace.bz2
Log:
redo the traces. the pyaes one will have to be redone.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/bench.gnumeric
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/crypto_pyaes.py.trace.bz2
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/django.trace.bz2
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/fannkuch.py.trace.bz2
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/go.py.trace.bz2
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/html5lib.trace.bz2
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/meteor-contest.py.trace.bz2
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/nbody_modified.py.trace.bz2
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/pyflate-fast.py.trace.bz2
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/raytrace-simple.py.trace.bz2
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/richards.trace.bz2
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/spambayes.trace.bz2
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/spectral-norm.py.trace.bz2
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/telco.py.trace.bz2
==============================================================================
Binary files. No diff available.
Added: pypy/extradoc/talk/pepm2011/benchmarks/twisted_names.trace.bz2
==============================================================================
Binary file. No diff available.
From leuschel at codespeak.net Wed Oct 13 15:31:22 2010
From: leuschel at codespeak.net (leuschel at codespeak.net)
Date: Wed, 13 Oct 2010 15:31:22 +0200 (CEST)
Subject: [pypy-svn] r77872 - pypy/extradoc/talk/pepm2011
Message-ID: <20101013133122.CD52D282BDA@codespeak.net>
Author: leuschel
Date: Wed Oct 13 15:31:20 2010
New Revision: 77872
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
some changes to Sect 1
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Wed Oct 13 15:31:20 2010
@@ -131,7 +131,7 @@
A recently popular approach to implementing just-in-time compilers for dynamic
languages is that of a tracing JIT. A tracing JIT works by observing the running
-program and recording its hot parts into linear execution traces, which are then turned into
+program and recording its hot spots into linear execution traces, which are then turned into
machine code. One reason for the popularity of tracing JITs is their relative
simplicity. They can often be added to an interpreter and a lot of the
infrastructure of an interpreter can be reused. They give some important
@@ -140,16 +140,16 @@
hard in a compiler, such as register allocation.
The usage of a tracing JIT can remove the overhead of bytecode dispatch and that
-of the interpreter data structures. In this paper we want to present an approach
-how an optimization can be added to a tracing JIT that also removes some of the
+of the interpreter data structures. In this paper we want to present a new
+optimization that can be added to a tracing JIT that further removes some of the
overhead more closely associated to dynamic languages, such as boxing overhead
and type dispatching. Our experimental platform is the PyPy project, which is an
environment for implementing dynamic programming languages. PyPy and tracing
JITs are described in more detail in Section~\ref{sec:Background}.
Section~\ref{sec:lifetimes} analyzes the problem to be solved more closely.
-The most important technique we use to achieve to optimize traces is a
-straightforward application of partial evaluation. The partial evaluation
+The core of our trace optimization technique can be
+viewed as partial evaluation: the partial evaluation
performs a form of escape analysis \cite{bruno_blanchet_escape_2003} on the traces and make some
objects that are allocated in the trace \emph{static}\footnote{These objects are
called \emph{virtual} in Psyco \cite{rigo_representation-based_2004}.} which
From arigo at codespeak.net Wed Oct 13 15:35:34 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 13 Oct 2010 15:35:34 +0200 (CEST)
Subject: [pypy-svn] r77873 - in pypy/trunk/pypy/jit/codewriter: . test
Message-ID: <20101013133534.0AA61282BE0@codespeak.net>
Author: arigo
Date: Wed Oct 13 15:35:33 2010
New Revision: 77873
Modified:
pypy/trunk/pypy/jit/codewriter/jtransform.py
pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py
Log:
Test and fix: optimized unicode equality functions that take a single
char as argument were incorrectly typed like the string equality function,
and took a lltype.Char instead of lltype.UniChar.
Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/trunk/pypy/jit/codewriter/jtransform.py Wed Oct 13 15:35:33 2010
@@ -1057,28 +1057,32 @@
[c_func] + [varoftype(T) for T in argtypes],
varoftype(resulttype))
calldescr = self.callcontrol.getcalldescr(op, oopspecindex)
- func = heaptracker.adr2int(
- llmemory.cast_ptr_to_adr(c_func.value))
+ if isinstance(c_func.value, str): # in tests only
+ func = c_func.value
+ else:
+ func = heaptracker.adr2int(
+ llmemory.cast_ptr_to_adr(c_func.value))
_callinfo_for_oopspec[oopspecindex] = calldescr, func
def _handle_stroruni_call(self, op, oopspec_name, args):
- if args[0].concretetype.TO == rstr.STR:
+ SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE)
+ if SoU.TO == rstr.STR:
dict = {"stroruni.concat": EffectInfo.OS_STR_CONCAT,
"stroruni.slice": EffectInfo.OS_STR_SLICE,
"stroruni.equal": EffectInfo.OS_STR_EQUAL,
}
- elif args[0].concretetype.TO == rstr.UNICODE:
+ CHR = lltype.Char
+ elif SoU.TO == rstr.UNICODE:
dict = {"stroruni.concat": EffectInfo.OS_UNI_CONCAT,
"stroruni.slice": EffectInfo.OS_UNI_SLICE,
"stroruni.equal": EffectInfo.OS_UNI_EQUAL,
}
+ CHR = lltype.UniChar
else:
assert 0, "args[0].concretetype must be STR or UNICODE"
#
if oopspec_name == "stroruni.equal":
- SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE)
for otherindex, othername, argtypes, resulttype in [
-
(EffectInfo.OS_STREQ_SLICE_CHECKNULL,
"str.eq_slice_checknull",
[SoU, lltype.Signed, lltype.Signed, SoU],
@@ -1089,7 +1093,7 @@
lltype.Signed),
(EffectInfo.OS_STREQ_SLICE_CHAR,
"str.eq_slice_char",
- [SoU, lltype.Signed, lltype.Signed, lltype.Char],
+ [SoU, lltype.Signed, lltype.Signed, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_NONNULL,
"str.eq_nonnull",
@@ -1097,11 +1101,11 @@
lltype.Signed),
(EffectInfo.OS_STREQ_NONNULL_CHAR,
"str.eq_nonnull_char",
- [SoU, lltype.Char],
+ [SoU, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_CHECKNULL_CHAR,
"str.eq_checknull_char",
- [SoU, lltype.Char],
+ [SoU, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_LENGTHOK,
"str.eq_lengthok",
Modified: pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py (original)
+++ pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py Wed Oct 13 15:35:33 2010
@@ -79,6 +79,30 @@
return 'builtin'
def getcalldescr(self, op, oopspecindex=None):
assert oopspecindex is not None # in this test
+ EI = effectinfo.EffectInfo
+ if oopspecindex != EI.OS_ARRAYCOPY:
+ PSTR = lltype.Ptr(rstr.STR)
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ INT = lltype.Signed
+ UNICHAR = lltype.UniChar
+ argtypes = {
+ EI.OS_STR2UNICODE:([PSTR], PUNICODE),
+ EI.OS_STR_CONCAT: ([PSTR, PSTR], PSTR),
+ EI.OS_STR_SLICE: ([PSTR, INT, INT], PSTR),
+ EI.OS_UNI_CONCAT: ([PUNICODE, PUNICODE], PUNICODE),
+ EI.OS_UNI_SLICE: ([PUNICODE, INT, INT], PUNICODE),
+ EI.OS_UNI_EQUAL: ([PUNICODE, PUNICODE], lltype.Bool),
+ EI.OS_UNIEQ_SLICE_CHECKNULL:([PUNICODE, INT, INT, PUNICODE], INT),
+ EI.OS_UNIEQ_SLICE_NONNULL: ([PUNICODE, INT, INT, PUNICODE], INT),
+ EI.OS_UNIEQ_SLICE_CHAR: ([PUNICODE, INT, INT, UNICHAR], INT),
+ EI.OS_UNIEQ_NONNULL: ([PUNICODE, PUNICODE], INT),
+ EI.OS_UNIEQ_NONNULL_CHAR: ([PUNICODE, UNICHAR], INT),
+ EI.OS_UNIEQ_CHECKNULL_CHAR: ([PUNICODE, UNICHAR], INT),
+ EI.OS_UNIEQ_LENGTHOK: ([PUNICODE, PUNICODE], INT),
+ }
+ argtypes = argtypes[oopspecindex]
+ assert argtypes[0] == [v.concretetype for v in op.args[1:]]
+ assert argtypes[1] == op.result.concretetype
return 'calldescr-%d' % oopspecindex
def calldescr_canraise(self, calldescr):
return False
@@ -785,6 +809,28 @@
assert op1.args[2] == ListOfKind('ref', [v1])
assert op1.result == v2
+def test_unicode_eq_checknull_char():
+ # test that the oopspec is present and correctly transformed
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ FUNC = lltype.FuncType([PUNICODE, PUNICODE], lltype.Bool)
+ func = lltype.functionptr(FUNC, 'll_streq',
+ _callable=rstr.LLHelpers.ll_streq)
+ v1 = varoftype(PUNICODE)
+ v2 = varoftype(PUNICODE)
+ v3 = varoftype(lltype.Bool)
+ op = SpaceOperation('direct_call', [const(func), v1, v2], v3)
+ tr = Transformer(FakeCPU(), FakeBuiltinCallControl())
+ op1 = tr.rewrite_operation(op)
+ assert op1.opname == 'residual_call_r_i'
+ assert op1.args[0].value == func
+ assert op1.args[1] == 'calldescr-%d' % effectinfo.EffectInfo.OS_UNI_EQUAL
+ assert op1.args[2] == ListOfKind('ref', [v1, v2])
+ assert op1.result == v3
+ # test that the OS_UNIEQ_* functions are registered
+ cifo = effectinfo._callinfo_for_oopspec
+ assert effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL in cifo
+ assert effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR in cifo
+
def test_list_ll_arraycopy():
from pypy.rlib.rgc import ll_arraycopy
LIST = lltype.GcArray(lltype.Signed)
From afa at codespeak.net Wed Oct 13 15:53:09 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 13 Oct 2010 15:53:09 +0200 (CEST)
Subject: [pypy-svn] r77874 - in pypy/branch/fast-forward/pypy:
module/_hashlib module/_hashlib/test rlib
Message-ID: <20101013135309.D6C80282BE0@codespeak.net>
Author: afa
Date: Wed Oct 13 15:53:07 2010
New Revision: 77874
Modified:
pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py
pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py
pypy/branch/fast-forward/pypy/rlib/ropenssl.py
Log:
Implement the _hashlib module
Modified: pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py (original)
+++ pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py Wed Oct 13 15:53:07 2010
@@ -1,9 +1,10 @@
from pypy.interpreter.gateway import unwrap_spec, interp2app
-from pypy.interpreter.typedef import TypeDef
+from pypy.interpreter.typedef import TypeDef, GetSetProperty
from pypy.tool.sourcetools import func_renamer
from pypy.interpreter.baseobjspace import Wrappable, W_Root, ObjSpace
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.rlib import ropenssl
+from pypy.rlib.rstring import StringBuilder
algorithms = ('md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512')
@@ -12,37 +13,105 @@
self.name = name
self.ctx = lltype.malloc(ropenssl.EVP_MD_CTX.TO, flavor='raw')
- @unwrap_spec('self', ObjSpace, str, str)
- def descr_init(self, space, name, buffer):
- digest = ropenssl.EVT_get_digestbyname(name)
+ digest = ropenssl.EVP_get_digestbyname(name)
if not digest:
raise OperationError(space.w_ValueError,
space.wrap("unknown hash function"))
ropenssl.EVP_DigestInit(self.ctx, digest)
- if buffer:
- self._hash(buffer)
-
@unwrap_spec('self', ObjSpace)
def descr_repr(self, space):
return space.wrap("<%s HASH object @ 0x%x>" % (self.name, id(self)))
@unwrap_spec('self', ObjSpace, str)
def update(self, space, buffer):
- self._hash(buffer)
-
- def _hash(self, buffer):
buf = rffi.str2charp(buffer)
try:
ropenssl.EVP_DigestUpdate(self.ctx, buf, len(buffer))
finally:
rffi.free_charp(buf)
+ @unwrap_spec('self', ObjSpace)
+ def copy(self, space):
+ "Return a copy of the hash object."
+ w_hash = W_Hash(self.name)
+ ropenssl.EVP_MD_CTX_copy(w_hash.ctx, self.ctx)
+ return w_hash
+
+ @unwrap_spec('self', ObjSpace)
+ def digest(self, space):
+ "Return the digest value as a string of binary data."
+ digest = self._digest(space)
+ return space.wrap(digest)
+
+ @unwrap_spec('self', ObjSpace)
+ def hexdigest(self, space):
+ "Return the digest value as a string of hexadecimal digits."
+ digest = self._digest(space)
+ hexdigits = '0123456789abcdef'
+ result = StringBuilder(self._digest_size() * 2)
+ for c in digest:
+ result.append(hexdigits[(ord(c) >> 4) & 0xf])
+ result.append(hexdigits[ ord(c) & 0xf])
+ return space.wrap(result.build())
+
+ def get_digest_size(space, self):
+ return space.wrap(self._digest_size())
+
+ def get_block_size(space, self):
+ return space.wrap(self._block_size())
+
+ def _digest(self, space):
+ ctx = self.copy(space).ctx
+ digest_size = self._digest_size()
+ digest = lltype.malloc(rffi.CCHARP.TO, digest_size, flavor='raw')
+
+ try:
+ ropenssl.EVP_DigestFinal(ctx, digest, None)
+ return rffi.charp2strn(digest, digest_size)
+ finally:
+ lltype.free(digest, flavor='raw')
+
+
+ def _digest_size(self):
+ # XXX This isn't the nicest way, but the EVP_MD_size OpenSSL
+ # XXX function is defined as a C macro on OS X and would be
+ # XXX significantly harder to implement in another way.
+ # Values are digest sizes in bytes
+ return {
+ 'md5': 16,
+ 'sha1': 20,
+ 'sha224': 28,
+ 'sha256': 32,
+ 'sha384': 48,
+ 'sha512': 64,
+ }.get(self.name, 0)
+
+ def _block_size(self):
+ # XXX This isn't the nicest way, but the EVP_MD_CTX_block_size
+ # XXX OpenSSL function is defined as a C macro on some systems
+ # XXX and would be significantly harder to implement in
+ # XXX another way.
+ return {
+ 'md5': 64,
+ 'sha1': 64,
+ 'sha224': 64,
+ 'sha256': 64,
+ 'sha384': 128,
+ 'sha512': 128,
+ }.get(self.name, 0)
+
W_Hash.typedef = TypeDef(
'HASH',
- __init__=interp2app(W_Hash.descr_init),
__repr__=interp2app(W_Hash.descr_repr),
update=interp2app(W_Hash.update),
+ copy=interp2app(W_Hash.copy),
+ digest=interp2app(W_Hash.digest),
+ hexdigest=interp2app(W_Hash.hexdigest),
+ #
+ digest_size=GetSetProperty(W_Hash.get_digest_size),
+ digestsize=GetSetProperty(W_Hash.get_digest_size),
+ block_size=GetSetProperty(W_Hash.get_block_size),
)
@unwrap_spec(ObjSpace, str, str)
Modified: pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py (original)
+++ pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py Wed Oct 13 15:53:07 2010
@@ -10,7 +10,7 @@
assert isinstance(_hashlib.new('md5'), _hashlib.HASH)
def test_attributes(self):
- import _hashlib
+ import hashlib
for name, expected_size in {'md5': 16,
'sha1': 20,
'sha224': 28,
@@ -29,8 +29,12 @@
hexdigest = h.hexdigest()
h2.update('d')
h2.update('ef')
- assert digest == h.digest()
- assert hexdigest == h.hexdigest()
+ assert digest == h2.digest()
+ assert hexdigest == h2.hexdigest()
+ assert len(digest) == h.digest_size
+ assert len(hexdigest) == h.digest_size * 2
+ c_digest = digest
+ c_hexdigest = hexdigest
# also test the pure Python implementation
h = hashlib.__get_builtin_constructor(name)('')
@@ -44,10 +48,14 @@
hexdigest = h.hexdigest()
h2.update('d')
h2.update('ef')
- assert digest == h.digest()
- assert hexdigest == h.hexdigest()
+ assert digest == h2.digest()
+ assert hexdigest == h2.hexdigest()
+
+ # compare both implementations
+ assert c_digest == digest
+ assert c_hexdigest == hexdigest
def test_unicode(self):
import _hashlib
- assert isinstance(hashlib.new('sha1', u'xxx'), _hashlib.HASH)
+ assert isinstance(_hashlib.new('sha1', u'xxx'), _hashlib.HASH)
Modified: pypy/branch/fast-forward/pypy/rlib/ropenssl.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/ropenssl.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/ropenssl.py Wed Oct 13 15:53:07 2010
@@ -16,10 +16,12 @@
# so that openssl/ssl.h can repair this nonsense.
'wincrypt.h',
'openssl/ssl.h',
- 'openssl/err.h']
+ 'openssl/err.h',
+ 'openssl/evp.h']
else:
libraries = ['ssl', 'crypto']
- includes = ['openssl/ssl.h', 'openssl/err.h']
+ includes = ['openssl/ssl.h', 'openssl/err.h',
+ 'openssl/evp.h']
eci = ExternalCompilationInfo(
libraries = libraries,
@@ -176,7 +178,7 @@
ssl_external('SSL_read', [SSL, rffi.CCHARP, rffi.INT], rffi.INT)
-EVP_MD_CTX = rffi.COpaquePtr('EVP_MD_CTX')
+EVP_MD_CTX = rffi.COpaquePtr('EVP_MD_CTX', compilation_info=eci)
EVP_MD = rffi.COpaquePtr('EVP_MD')
EVP_get_digestbyname = external(
@@ -188,6 +190,13 @@
EVP_DigestUpdate = external(
'EVP_DigestUpdate',
[EVP_MD_CTX, rffi.CCHARP, rffi.SIZE_T], rffi.INT)
+EVP_DigestFinal = external(
+ 'EVP_DigestFinal',
+ [EVP_MD_CTX, rffi.CCHARP, rffi.VOIDP], rffi.INT)
+EVP_MD_CTX_copy = external(
+ 'EVP_MD_CTX_copy', [EVP_MD_CTX, EVP_MD_CTX], rffi.INT)
+EVP_MD_CTX_cleanup = external(
+ 'EVP_MD_CTX_cleanup', [EVP_MD_CTX], rffi.INT)
def _init_ssl():
libssl_SSL_load_error_strings()
From arigo at codespeak.net Wed Oct 13 16:03:30 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 13 Oct 2010 16:03:30 +0200 (CEST)
Subject: [pypy-svn] r77875 -
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem
Message-ID: <20101013140330.33688282BE7@codespeak.net>
Author: arigo
Date: Wed Oct 13 16:03:28 2010
New Revision: 77875
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/ll2ctypes.py
Log:
Fix.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/ll2ctypes.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/ll2ctypes.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/ll2ctypes.py Wed Oct 13 16:03:28 2010
@@ -1158,6 +1158,8 @@
_TYPE = llmemory.Address
def __init__(self, void_p):
+ if isinstance(void_p, _lladdress):
+ void_p = void_p.void_p
if isinstance(void_p, (int, long)):
void_p = ctypes.c_void_p(void_p)
self.void_p = void_p
@@ -1170,6 +1172,8 @@
return '<_lladdress %s>' % (self.void_p,)
def __eq__(self, other):
+ if isinstance(other, _lladdress):
+ other = other.intval
if not isinstance(other, (int, long)):
other = cast_adr_to_int(other)
return intmask(other) == self.intval
From leuschel at codespeak.net Wed Oct 13 16:05:30 2010
From: leuschel at codespeak.net (leuschel at codespeak.net)
Date: Wed, 13 Oct 2010 16:05:30 +0200 (CEST)
Subject: [pypy-svn] r77876 - pypy/extradoc/talk/pepm2011
Message-ID: <20101013140530.7E8BA282BEA@codespeak.net>
Author: leuschel
Date: Wed Oct 13 16:05:28 2010
New Revision: 77876
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
some more changes
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Wed Oct 13 16:05:28 2010
@@ -179,8 +179,8 @@
The work described in this paper was done in the context of the PyPy project
\cite{armin_rigo_pypys_2006}. PyPy is an environment where dynamic languages can
-be implemented in a simple yet efficient way. The approach taken when
-implementing a language with PyPy is to write an \emph{interpreter}
+be implemented in a simple yet efficient way.
+When implementing a language with PyPy one writes an \emph{interpreter}
for the language in
\emph{RPython} \cite{davide_ancona_rpython:_2007}. RPython ("restricted Python")
is a subset of Python chosen in such a way that type inference becomes
@@ -191,7 +191,7 @@
aspects of the final VM are woven into the generated code during the translation
to C.
-A number of languages have been implemented with PyPy. The project was started
+A number of languages have been implemented with PyPy. The project was initiated
to get a better Python implementation, which inspired the name of the project
and is still the main focus of development. In addition a number of other
languages were implemented, among them a Prolog interpreter
@@ -203,7 +203,7 @@
support for automated JIT compiler generation \cite{bolz_tracing_2009}. During
the translation to C, PyPy's tools can generate a just-in-time compiler for the
language that the interpreter is implementing. This process is mostly
-automatic; it only needs to be guided by the language implementer by a small number of
+automatic; it only needs to be guided by the language implementer using a small number of
source-code hints. Mostly-automatically generating a JIT compiler has many advantages
over writing one manually, which is an error-prone and tedious process.
By construction, the generated JIT has the same semantics as the interpreter.
@@ -214,7 +214,7 @@
to the original backend for the Intel \emph{x86} architecture. Examples of
additional JIT backends are the one for Intel \emph{x86-64} and an
experimental one for the CLI .NET Virtual Machine \cite{cuni_python_cli_2010}.
-The JIT that is produced by PyPy's JIT generator is a \emph{tracing JIT
+PyPy's JIT generator generates a \emph{tracing JIT
compiler}, a concept which we now explain in more details.
\subsection{Tracing JIT Compilers}
@@ -245,7 +245,7 @@
tracing JIT automatically performs inlining.
This trace of operations is then the basis of the generated code. The trace is
-optimized in some ways, and then turned into machine code. Both optimizations
+first optimized, and then turned into machine code. Both optimizations
and machine code generation is simple, because the traces are linear. This
linearity makes many optimizations a lot more tractable, and the inlining that
happens gives the optimizations automatically more context to work with.
@@ -277,17 +277,20 @@
\subsection{Running Example}
-For the purpose of this paper, we are going to use a very simple object
+For the purpose of this paper, we are going to use a tiny interpreter for a dynamic language with
+ a very simple object
model, that just supports an integer and a float type. The objects support only
two operations, \texttt{add}, which adds two objects (promoting ints to floats in a
mixed addition) and \texttt{is\_positive}, which returns whether the number is greater
than zero. The implementation of \texttt{add} uses classical Smalltalk-like
-double-dispatching. These classes could be part of the implementation of a very
-simple interpreter written in RPython. The classes can be seen in
-Figure~\ref{fig:objmodel}.
+double-dispatching.
+%These classes could be part of the implementation of a very
+%simple interpreter written in RPython.
+The classes can be seen in
+Figure~\ref{fig:objmodel} (written in RPython).
-\begin{figure}
-\begin{verbatim}
+\newcommand{\ignore}[1]{} % {{\tt \small ignore(#1)}}
+\ignore{
class Base(object):
def add(self, other):
""" add self to other """
@@ -303,6 +306,10 @@
def is_positive(self):
""" returns whether self is positive """
raise NotImplementedError("abstract base")
+ }
+\begin{figure}
+\begin{verbatim}
+class Base(object):
class BoxedInteger(Base):
def __init__(self, intval):
@@ -338,7 +345,8 @@
y = y.add(BoxedInteger(-1))
return res
\end{verbatim}
-\caption{A Simple Object Model and an Example Function Using it}
+\caption{An ``interpreter'' for a tiny Dynamic Language written in RPython}
+%\caption{A Simple Object Model and an Example Function Using it}
\label{fig:objmodel}
\end{figure}
@@ -350,7 +358,8 @@
implement the numeric tower needs two method calls per arithmetic operation,
which is costly due to the method dispatch.
-To understand the problems more directly, let us consider the simple function
+To understand the problems more directly, let us consider the simple
+interpreter function
\texttt{f} that uses the object model (see the bottom of
Figure~\ref{fig:objmodel}).
@@ -434,12 +443,18 @@
If the function is executed using the tracing JIT, with \texttt{y} being a
\texttt{BoxedInteger}, the produced trace looks like
-Figure~\ref{fig:unopt-trace}. The operations in the trace are shown indented to
+Figure~\ref{fig:unopt-trace} (lines starting with the hash ``\#'' are comments).
+
+XXX in which language is the trace written in ? still RPython ?
+
+The operations in the trace are shown indented to
correspond to the stack level of the function that contains the traced
operation. The trace is in single-assignment form, meaning that each variable is
assigned to exactly once. The arguments $p_0$ and $p_1$ of the loop correspond
to the live variables \texttt{y} and \texttt{res} in the original function.
+XXX explain set and get + int_add briefly
+
The trace shows the inefficiencies of \texttt{f} clearly, if one
looks at the number of \texttt{new} (corresponding to object creation),
\texttt{set/get} (corresponding to attribute reads/writes) and
From arigo at codespeak.net Wed Oct 13 16:12:18 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 13 Oct 2010 16:12:18 +0200 (CEST)
Subject: [pypy-svn] r77877 -
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem
Message-ID: <20101013141218.9E3DD282BEC@codespeak.net>
Author: arigo
Date: Wed Oct 13 16:12:17 2010
New Revision: 77877
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/ll2ctypes.py
Log:
More fixes.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/ll2ctypes.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/ll2ctypes.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/ll2ctypes.py Wed Oct 13 16:12:17 2010
@@ -587,6 +587,8 @@
res = intmask(cobj.value)
_int2obj[res] = llobj.adr.ptr._obj
return res
+ if isinstance(llobj, _lladdress):
+ llobj = llobj.intval
if isinstance(llobj, llmemory.fakeaddress):
llobj = llobj.ptr or 0
@@ -1192,6 +1194,8 @@
_name = "_llgcopaque"
def __init__(self, void_p):
+ if isinstance(void_p, _lladdress):
+ void_p = void_p.intval
if isinstance(void_p, (int, long)):
self.intval = intmask(void_p)
else:
From arigo at codespeak.net Wed Oct 13 16:32:44 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 13 Oct 2010 16:32:44 +0200 (CEST)
Subject: [pypy-svn] r77879 -
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem
Message-ID: <20101013143244.24BF8282BF7@codespeak.net>
Author: arigo
Date: Wed Oct 13 16:32:42 2010
New Revision: 77879
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rdict.py
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rtupletype.py
Log:
Fix for jit.metainterp.test.test_dict. Reverts the signature of
ll_kvi() in rdict.py to what it was, and in case of 'dict.items()',
fish for the correct tuple type manually from inside the function.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rdict.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rdict.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rdict.py Wed Oct 13 16:32:42 2010
@@ -275,14 +275,10 @@
def _rtype_method_kvi(self, hop, ll_func):
v_dic, = hop.inputargs(self)
- s_list = hop.s_result
- s_value = s_list.listdef.listitem.s_value
- r_value = hop.rtyper.getrepr(s_value)
- cLISTITEM = hop.inputconst(lltype.Void, r_value.lowleveltype)
r_list = hop.r_result
cLIST = hop.inputconst(lltype.Void, r_list.lowleveltype.TO)
hop.exception_cannot_occur()
- return hop.gendirectcall(ll_func, cLIST, cLISTITEM, v_dic)
+ return hop.gendirectcall(ll_func, cLIST, v_dic)
def rtype_method_keys(self, hop):
return self._rtype_method_kvi(hop, ll_dict_keys)
@@ -782,7 +778,17 @@
return v
def _make_ll_keys_values_items(kind):
- def ll_kvi(LIST, LISTITEM, dic):
+ from pypy.rpython.lltypesystem.rtupletype import TUPLE_TYPE_2
+ #
+ def ll_kvi(LIST, dic):
+ DICT = lltype.typeOf(dic).TO
+ if kind == 'items':
+ LISTITEM = TUPLE_TYPE_2(DICT.KEY, DICT.VALUE)
+ elif kind == 'keys':
+ LISTITEM = DICT.KEY
+ elif kind == 'values':
+ LISTITEM = DICT.VALUE
+ #
res = LIST.ll_newlist(dic.num_items)
entries = dic.entries
dlen = len(entries)
@@ -807,6 +813,7 @@
i += 1
assert p == res.ll_length()
return res
+ #
ll_kvi.oopspec = 'dict.%s(dic)' % kind
return ll_kvi
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rtupletype.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rtupletype.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rtupletype.py Wed Oct 13 16:32:42 2010
@@ -13,3 +13,7 @@
kwds = {'hints': {'immutable': True,
'noidentity': True}}
return Ptr(GcStruct('tuple%d' % len(field_lltypes), *fields, **kwds))
+
+def TUPLE_TYPE_2(type1, type2): # hack for annotation
+ return TUPLE_TYPE([type1, type2])
+TUPLE_TYPE_2._annspecialcase_ = 'specialize:memo'
From arigo at codespeak.net Wed Oct 13 16:34:03 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 13 Oct 2010 16:34:03 +0200 (CEST)
Subject: [pypy-svn] r77880 -
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test
Message-ID: <20101013143403.D93BB282BFA@codespeak.net>
Author: arigo
Date: Wed Oct 13 16:34:02 2010
New Revision: 77880
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_gctypelayout.py
Log:
Fix test.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_gctypelayout.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_gctypelayout.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/memory/test/test_gctypelayout.py Wed Oct 13 16:34:02 2010
@@ -38,9 +38,10 @@
assert len(offsets_to_gc_pointers(T)) == c
def test_hiddenptr32():
- from pypy.rpython.lltypesystem.llmemory import HIDDENPTR32
+ from pypy.rpython.lltypesystem.llmemory import HiddenGcRef32
from pypy.rpython.lltypesystem.llmemory import has_odd_value_marker
- T = lltype.GcStruct('T', ('foo', lltype.Ptr(GC_S3)), ('bar', HIDDENPTR32))
+ T = lltype.GcStruct('T', ('foo', lltype.Ptr(GC_S3)),
+ ('bar', HiddenGcRef32))
ofs = offsets_to_gc_pointers(T)
assert len(ofs) == 2
assert not has_odd_value_marker(ofs[0])
From arigo at codespeak.net Wed Oct 13 16:43:38 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 13 Oct 2010 16:43:38 +0200 (CEST)
Subject: [pypy-svn] r77881 - pypy/branch/32ptr-on-64bit/pypy/doc/config
Message-ID: <20101013144338.8E071282BFB@codespeak.net>
Author: arigo
Date: Wed Oct 13 16:43:37 2010
New Revision: 77881
Added:
pypy/branch/32ptr-on-64bit/pypy/doc/config/translation.compressptr.txt
Log:
Documentation.
Added: pypy/branch/32ptr-on-64bit/pypy/doc/config/translation.compressptr.txt
==============================================================================
--- (empty file)
+++ pypy/branch/32ptr-on-64bit/pypy/doc/config/translation.compressptr.txt Wed Oct 13 16:43:37 2010
@@ -0,0 +1,7 @@
+On 64-bit platforms, "compress" some pointers to 32 bits.
+
+Pointers to GC-managed, fixed-size objects are compressed.
+This saves potentially a lot of memory. The drawback is that
+it will refuse to allocate more than 32GB of RAM for fixed-size
+objects. (The program can use more RAM in total, e.g. with big
+strings, which are not fixed-size objects.)
From antocuni at codespeak.net Wed Oct 13 17:12:22 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Wed, 13 Oct 2010 17:12:22 +0200 (CEST)
Subject: [pypy-svn] r77882 - pypy/extradoc/talk/pepm2011
Message-ID: <20101013151222.65363282BFC@codespeak.net>
Author: antocuni
Date: Wed Oct 13 17:12:20 2010
New Revision: 77882
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
make the code nicer
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Wed Oct 13 17:12:20 2010
@@ -10,6 +10,27 @@
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage[utf8]{inputenc}
+\usepackage{setspace}
+
+\usepackage{listings}
+
+\usepackage[T1]{fontenc}
+\usepackage{beramono}
+
+
+\definecolor{gray}{rgb}{0.3,0.3,0.3}
+
+\lstset{
+ basicstyle=\setstretch{1.1}\ttfamily\footnotesize,
+ language=Python,
+ keywordstyle=\bfseries,
+ stringstyle=\color{blue},
+ commentstyle=\color{gray}\textit,
+ fancyvrb=true,
+ showstringspaces=false,
+ keywords={def,while,if,elif,return,class,get,set,new,guard_class}
+}
+
\newboolean{showcomments}
\setboolean{showcomments}{true}
@@ -255,10 +276,10 @@
To make sure that the trace is maintaining the correct semantics, it contains a
\emph{guard} at all places where the execution could have diverged from the
path. Those guards check the assumptions under which execution can stay on the
-trace. As an example, if a loop contains an \texttt{if} statement, the trace
+trace. As an example, if a loop contains an \lstinline{if} statement, the trace
will contain the execution of one of the paths only, which is the path that was
taken during the production of the trace. The trace will also contain a guard
-that checks that the condition of the \texttt{if} statement is true, because if
+that checks that the condition of the \lstinline{if} statement is true, because if
it isn't, the rest of the trace is not valid.
When generating machine code, every guard is be turned into a quick check to
@@ -280,9 +301,9 @@
For the purpose of this paper, we are going to use a tiny interpreter for a dynamic language with
a very simple object
model, that just supports an integer and a float type. The objects support only
-two operations, \texttt{add}, which adds two objects (promoting ints to floats in a
-mixed addition) and \texttt{is\_positive}, which returns whether the number is greater
-than zero. The implementation of \texttt{add} uses classical Smalltalk-like
+two operations, \lstinline{add}, which adds two objects (promoting ints to floats in a
+mixed addition) and \lstinline{is_positive}, which returns whether the number is greater
+than zero. The implementation of \lstinline{add} uses classical Smalltalk-like
double-dispatching.
%These classes could be part of the implementation of a very
%simple interpreter written in RPython.
@@ -308,43 +329,52 @@
raise NotImplementedError("abstract base")
}
\begin{figure}
-\begin{verbatim}
+\begin{lstlisting}[mathescape]
class Base(object):
+ ...
class BoxedInteger(Base):
- def __init__(self, intval):
- self.intval = intval
- def add(self, other):
- return other.add__int(self.intval)
- def add__int(self, intother):
- return BoxedInteger(intother + self.intval)
- def add__float(self, floatother):
- floatvalue = floatother + float(self.intval)
- return BoxedFloat(floatvalue)
- def is_positive(self):
- return self.intval > 0
+ def __init__(self, intval):
+ self.intval = intval
+
+ def add(self, other):
+ return other.add__int(self.intval)
+
+ def add__int(self, intother):
+ return BoxedInteger(intother + self.intval)
+
+ def add__float(self, floatother):
+ floatvalue = floatother + float(self.intval)
+ return BoxedFloat(floatvalue)
+
+ def is_positive(self):
+ return self.intval > 0
class BoxedFloat(Base):
- def __init__(self, floatval):
- self.floatval = floatval
- def add(self, other):
- return other.add__float(self.floatval)
- def add__int(self, intother):
- floatvalue = float(intother) + self.floatval
- return BoxedFloat(floatvalue)
- def add__float(self, floatother):
- return BoxedFloat(floatother + self.floatval)
- def is_positive(self):
- return self.floatval > 0.0
+ def __init__(self, floatval):
+ self.floatval = floatval
+
+ def add(self, other):
+ return other.add__float(self.floatval)
+
+ def add__int(self, intother):
+ floatvalue = float(intother) + self.floatval
+ return BoxedFloat(floatvalue)
+
+ def add__float(self, floatother):
+ return BoxedFloat(floatother + self.floatval)
+
+ def is_positive(self):
+ return self.floatval > 0.0
def f(y):
- res = BoxedInteger(0)
- while y.is_positive():
- res = res.add(y).add(BoxedInteger(-100))
- y = y.add(BoxedInteger(-1))
- return res
-\end{verbatim}
+ res = BoxedInteger(0)
+ while y.is_positive():
+ res = res.add(y).add(BoxedInteger(-100))
+ y = y.add(BoxedInteger(-1))
+ return res
+\end{lstlisting}
\caption{An ``interpreter'' for a tiny Dynamic Language written in RPython}
%\caption{A Simple Object Model and an Example Function Using it}
\label{fig:objmodel}
@@ -352,7 +382,7 @@
Using these classes to implement arithmetic shows the basic problem that a
dynamic language implementation has. All the numbers are instances of either
-\texttt{BoxedInteger} or \texttt{BoxedFloat}, thus they consume space on the
+\lstinline{BoxedInteger} or \lstinline{BoxedFloat}, thus they consume space on the
heap. Performing many arithmetic operations produces lots of garbage quickly,
thus putting pressure on the garbage collector. Using double dispatching to
implement the numeric tower needs two method calls per arithmetic operation,
@@ -360,7 +390,7 @@
To understand the problems more directly, let us consider the simple
interpreter function
-\texttt{f} that uses the object model (see the bottom of
+\lstinline{f} that uses the object model (see the bottom of
Figure~\ref{fig:objmodel}).
XXX this is not an RPython interpreter; put a reference to the previous
@@ -368,81 +398,80 @@
the interpretation overhead, turning it into basically something
equivalent to the example here, which is the start of the present paper.
-The loop in \texttt{f} iterates \texttt{y} times, and computes something in the process.
+The loop in \lstinline{f} iterates \lstinline{y} times, and computes something in the process.
Simply running this function is slow, because there are lots of virtual method
-calls inside the loop, one for each \texttt{is\_positive} and even two for each
-call to \texttt{add}. These method calls need to check the type of the involved
+calls inside the loop, one for each \lstinline{is_positive} and even two for each
+call to \lstinline{add}. These method calls need to check the type of the involved
objects repeatedly and redundantly. In addition, a lot of objects are created
when executing that loop, many of these objects do not survive for very long.
-The actual computation that is performed by \texttt{f} is simply a sequence of
+The actual computation that is performed by \lstinline{f} is simply a sequence of
float or integer additions.
\begin{figure}
-\texttt{
-\begin{tabular}{l}
-\# arguments to the trace: $p_{0}$, $p_{1}$ \\
-\# inside f: res.add(y) \\
-guard\_class($p_{1}$, BoxedInteger) \\
-~~~~\# inside BoxedInteger.add \\
-~~~~$i_{2}$ = get($p_{1}$, intval) \\
-~~~~guard\_class($p_{0}$, BoxedInteger) \\
-~~~~~~~~\# inside BoxedInteger.add\_\_int \\
-~~~~~~~~$i_{3}$ = get($p_{0}$, intval) \\
-~~~~~~~~$i_{4}$ = int\_add($i_{2}$, $i_{3}$) \\
-~~~~~~~~$p_{5}$ = new(BoxedInteger) \\
-~~~~~~~~~~~~\# inside BoxedInteger.\_\_init\_\_ \\
-~~~~~~~~~~~~set($p_{5}$, intval, $i_{4}$) \\
-\# inside f: BoxedInteger(-100) \\
-$p_{6}$ = new(BoxedInteger) \\
-~~~~\# inside BoxedInteger.\_\_init\_\_ \\
-~~~~set($p_{6}$, intval, -100) \\
-~\\
-\# inside f: .add(BoxedInteger(-100)) \\
-guard\_class($p_{5}$, BoxedInteger) \\
-~~~~\# inside BoxedInteger.add \\
-~~~~$i_{7}$ = get($p_{5}$, intval) \\
-~~~~guard\_class($p_{6}$, BoxedInteger) \\
-~~~~~~~~\# inside BoxedInteger.add\_\_int \\
-~~~~~~~~$i_{8}$ = get($p_{6}$, intval) \\
-~~~~~~~~$i_{9}$ = int\_add($i_{7}$, $i_{8}$) \\
-~~~~~~~~$p_{10}$ = new(BoxedInteger) \\
-~~~~~~~~~~~~\# inside BoxedInteger.\_\_init\_\_ \\
-~~~~~~~~~~~~set($p_{10}$, intval, $i_{9}$) \\
-~\\
-\# inside f: BoxedInteger(-1) \\
-$p_{11}$ = new(BoxedInteger) \\
-~~~~\# inside BoxedInteger.\_\_init\_\_ \\
-~~~~set($p_{11}$, intval, -1) \\
-~\\
-\# inside f: y.add(BoxedInteger(-1)) \\
-guard\_class($p_{0}$, BoxedInteger) \\
-~~~~\# inside BoxedInteger.add \\
-~~~~$i_{12}$ = get($p_{0}$, intval) \\
-~~~~guard\_class($p_{11}$, BoxedInteger) \\
-~~~~~~~~\# inside BoxedInteger.add\_\_int \\
-~~~~~~~~$i_{13}$ = get($p_{11}$, intval) \\
-~~~~~~~~$i_{14}$ = int\_add($i_{12}$, $i_{13}$) \\
-~~~~~~~~$p_{15}$ = new(BoxedInteger) \\
-~~~~~~~~~~~~\# inside BoxedInteger.\_\_init\_\_ \\
-~~~~~~~~~~~~set($p_{15}$, intval, $i_{14}$) \\
-~\\
-\# inside f: y.is\_positive() \\
-guard\_class($p_{15}$, BoxedInteger) \\
-~~~~\# inside BoxedInteger.is\_positive \\
-~~~~$i_{16}$ = get($p_{15}$, intval) \\
-~~~~$i_{17}$ = int\_gt($i_{16}$, 0) \\
-\# inside f \\
-guard\_true($i_{17}$) \\
-jump($p_{15}$, $p_{10}$) \\
-\end{tabular}
-}
+\begin{lstlisting}[mathescape]
+# arguments to the trace: $p_{0}$, $p_{1}$
+# inside f: res.add(y)
+guard_class($p_{1}$, BoxedInteger)
+ # inside BoxedInteger.add
+ $i_{2}$ = get($p_{1}$, intval)
+ guard_class($p_{0}$, BoxedInteger)
+ # inside BoxedInteger.add__int
+ $i_{3}$ = get($p_{0}$, intval)
+ $i_{4}$ = int_add($i_{2}$, $i_{3}$)
+ $p_{5}$ = new(BoxedInteger)
+ # inside BoxedInteger.__init__
+ set($p_{5}$, intval, $i_{4}$)
+
+# inside f: BoxedInteger(-100)
+$p_{6}$ = new(BoxedInteger)
+ # inside BoxedInteger.__init__
+ set($p_{6}$, intval, -100)
+
+# inside f: .add(BoxedInteger(-100))
+guard_class($p_{5}$, BoxedInteger)
+ # inside BoxedInteger.add
+ $i_{7}$ = get($p_{5}$, intval)
+ guard_class($p_{6}$, BoxedInteger)
+ # inside BoxedInteger.add__int
+ $i_{8}$ = get($p_{6}$, intval)
+ $i_{9}$ = int_add($i_{7}$, $i_{8}$)
+ $p_{10}$ = new(BoxedInteger)
+ # inside BoxedInteger.__init__
+ set($p_{10}$, intval, $i_{9}$)
+
+# inside f: BoxedInteger(-1)
+$p_{11}$ = new(BoxedInteger)
+ # inside BoxedInteger.__init__
+ set($p_{11}$, intval, -1)
+
+# inside f: y.add(BoxedInteger(-1))
+guard_class($p_{0}$, BoxedInteger)
+ # inside BoxedInteger.add
+ $i_{12}$ = get($p_{0}$, intval)
+ guard_class($p_{11}$, BoxedInteger)
+ # inside BoxedInteger.add__int
+ $i_{13}$ = get($p_{11}$, intval)
+ $i_{14}$ = int_add($i_{12}$, $i_{13}$)
+ $p_{15}$ = new(BoxedInteger)
+ # inside BoxedInteger.__init__
+ set($p_{15}$, intval, $i_{14}$)
+
+# inside f: y.is_positive()
+guard_class($p_{15}$, BoxedInteger)
+ # inside BoxedInteger.is_positive
+ $i_{16}$ = get($p_{15}$, intval)
+ $i_{17}$ = int_gt($i_{16}$, 0)
+# inside f
+guard_true($i_{17}$)
+jump($p_{15}$, $p_{10}$)
+\end{lstlisting}
\caption{Unoptimized Trace for the Simple Object Model}
\label{fig:unopt-trace}
\end{figure}
-If the function is executed using the tracing JIT, with \texttt{y} being a
-\texttt{BoxedInteger}, the produced trace looks like
+If the function is executed using the tracing JIT, with \lstinline{y} being a
+\lstinline{BoxedInteger}, the produced trace looks like
Figure~\ref{fig:unopt-trace} (lines starting with the hash ``\#'' are comments).
XXX in which language is the trace written in ? still RPython ?
@@ -451,26 +480,26 @@
correspond to the stack level of the function that contains the traced
operation. The trace is in single-assignment form, meaning that each variable is
assigned to exactly once. The arguments $p_0$ and $p_1$ of the loop correspond
-to the live variables \texttt{y} and \texttt{res} in the original function.
+to the live variables \lstinline{y} and \lstinline{res} in the original function.
-XXX explain set and get + int_add briefly
+XXX explain set and get + int\_add briefly
-The trace shows the inefficiencies of \texttt{f} clearly, if one
-looks at the number of \texttt{new} (corresponding to object creation),
-\texttt{set/get} (corresponding to attribute reads/writes) and
-\texttt{guard\_class} operations (corresponding to method calls).
+The trace shows the inefficiencies of \lstinline{f} clearly, if one
+looks at the number of \lstinline{new} (corresponding to object creation),
+\lstinline{set/get} (corresponding to attribute reads/writes) and
+\lstinline{guard_class} operations (corresponding to method calls).
In the rest of the paper we will see how this trace can be optimized using
partial evaluation.
-Note how the functions that are called by \texttt{f} are automatically inlined
-into the trace. The method calls are always preceded by a \texttt{guard\_class}
+Note how the functions that are called by \lstinline{f} are automatically inlined
+into the trace. The method calls are always preceded by a \lstinline{guard_class}
operation, to check that the class of the receiver is the same as the one that
-was observed during tracing.\footnote{\texttt{guard\_class} performs a precise
+was observed during tracing.\footnote{\lstinline{guard_class} performs a precise
class check, not checking for subclasses.} These guards make the trace specific
-to the situation where \texttt{y} is really a \texttt{BoxedInteger}, it can
-already be said to be specialized for \texttt{BoxedIntegers}. When the trace is
-turned into machine code and then executed with \texttt{BoxedFloats}, the
-first \texttt{guard\_class} instruction will fail and execution will continue
+to the situation where \lstinline{y} is really a \lstinline{BoxedInteger}, it can
+already be said to be specialized for \lstinline{BoxedIntegers}. When the trace is
+turned into machine code and then executed with \lstinline{BoxedFloats}, the
+first \lstinline{guard_class} instruction will fail and execution will continue
using the interpreter.
@@ -499,15 +528,15 @@
executed until one of the guards in the trace fails, and the execution is
aborted and interpretation resumes.
-Some of the operations within this trace are \texttt{new} operations, which each
+Some of the operations within this trace are \lstinline{new} operations, which each
create a new instance of some class. These instances are used for a while, e.g.
by calling methods on them (which are inlined into the trace), reading and
writing their fields. Some of these instances \emph{escape}, which means that
they are stored in some globally accessible place or are passed into a
non-inlined function via a residual call.
-Together with the \texttt{new} operations, the figure shows the lifetimes of the
-created objects. The objects that are created within a trace using \texttt{new}
+Together with the \lstinline{new} operations, the figure shows the lifetimes of the
+created objects. The objects that are created within a trace using \lstinline{new}
fall into one of several categories:
\begin{itemize}
@@ -550,62 +579,57 @@
but it is only used to optimized operations within a trace. XXX mention Prolog.
The partial evaluation works by walking the trace from beginning to end.
-Whenever a \texttt{new} operation is seen, the operation is removed and a static
+Whenever a \lstinline{new} operation is seen, the operation is removed and a static
object is constructed and associated with the variable that would have stored
-the result of \texttt{new}. The static object describes the shape of the
+the result of \lstinline{new}. The static object describes the shape of the
original object, \eg where the values that would be stored in the fields of the
allocated object come from, as well as the type of the object. Whenever the
-optimizer sees a \texttt{set} that writes into such an object, that shape
+optimizer sees a \lstinline{set} that writes into such an object, that shape
description is updated and the operation can be removed, which means that the
operation was done at partial evaluation time. When the optimizer encounters a
-\texttt{get} from such an object, the result is read from the shape
+\lstinline{get} from such an object, the result is read from the shape
description, and the operation is also removed. Equivalently, a
-\texttt{guard\_class} on a variable that has a shape description can be removed
+\lstinline{guard_class} on a variable that has a shape description can be removed
as well, because the shape description stores the type and thus the outcome of
the type check the guard does is statically known.
In the example from last section, the following operations would produce two
static objects, and be completely removed from the optimized trace:
-\texttt{
-\begin{tabular}{l}
-$p_{5}$ = new(BoxedInteger) \\
-set($p_{5}$, intval, $i_{4}$) \\
-$p_{6}$ = new(BoxedInteger) \\
-set($p_{6}$, intval, -100) \\
-\end{tabular}
-}
+\begin{lstlisting}[mathescape,xleftmargin=20pt]
+$p_{5}$ = new(BoxedInteger)
+set($p_{5}$, intval, $i_{4}$)
+$p_{6}$ = new(BoxedInteger)
+set($p_{6}$, intval, -100)
+\end{lstlisting}
+
The static object associated with $p_{5}$ would know that it is a
-\texttt{BoxedInteger} whose \texttt{intval} field contains $i_{4}$; the
-one associated with $p_{6}$ would know that it is a \texttt{BoxedInteger}
-whose \texttt{intval} field contains the constant -100.
+\lstinline{BoxedInteger} whose \lstinline{intval} field contains $i_{4}$; the
+one associated with $p_{6}$ would know that it is a \lstinline{BoxedInteger}
+whose \lstinline{intval} field contains the constant -100.
The following operations on $p_{5}$ and $p_{6}$ could then be
optimized using that knowledge:
-\texttt{
-\begin{tabular}{l}
-guard\_class($p_{5}$, BoxedInteger) \\
-$i_{7}$ = get($p_{5}$, intval) \\
-\# inside BoxedInteger.add \\
-guard\_class($p_{6}$, BoxedInteger) \\
-\# inside BoxedInteger.add\_\_int \\
-$i_{8}$ = get($p_{6}$, intval) \\
-$i_{9}$ = int\_add($i_{7}$, $i_{8}$) \\
-\end{tabular}
-}
+\begin{lstlisting}[mathescape,xleftmargin=20pt]
+guard_class($p_{5}$, BoxedInteger)
+$i_{7}$ = get($p_{5}$, intval)
+# inside BoxedInteger.add
+guard_class($p_{6}$, BoxedInteger)
+# inside BoxedInteger.add__int
+$i_{8}$ = get($p_{6}$, intval)
+$i_{9}$ = int_add($i_{7}$, $i_{8}$)
+\end{lstlisting}
-The \texttt{guard\_class} operations can be removed, because the classes of $p_{5}$ and
-$p_{6}$ are known to be \texttt{BoxedInteger}. The \texttt{get} operations can be removed
+The \lstinline{guard_class} operations can be removed, because the classes of $p_{5}$ and
+$p_{6}$ are known to be \lstinline{BoxedInteger}. The \lstinline{get} operations can be removed
and $i_{7}$ and $i_{8}$ are just replaced by $i_{4}$ and -100. Thus the only
remaining operation in the optimized trace would be:
-\texttt{
-\begin{tabular}{l}
-$i_{9}$ = int\_add($i_{4}$, -100) \\
-\end{tabular}
-}
+\begin{lstlisting}[mathescape,xleftmargin=20pt]
+$i_{9}$ = int_add($i_{4}$, -100)
+\end{lstlisting}
The rest of the trace is optimized similarly.
@@ -619,7 +643,7 @@
necessary to put operations into the residual code that actually allocate the
static object at runtime.
-This is what happens at the end of the trace in Figure~\ref{fig:unopt-trace}, when the \texttt{jump} operation
+This is what happens at the end of the trace in Figure~\ref{fig:unopt-trace}, when the \lstinline{jump} operation
is hit. The arguments of the jump are at this point static objects. Before the
jump is emitted, they are \emph{lifted}. This means that the optimizer produces code
that allocates a new object of the right type and sets its fields to the field
@@ -627,22 +651,20 @@
objects, those need to be lifted as well, recursively) This means that instead of the jump,
the following operations are emitted:
-\texttt{
-\begin{tabular}{l}
-$p_{15}$ = new(BoxedInteger) \\
-set($p_{15}$, intval, $i_{14}$) \\
-$p_{10}$ = new(BoxedInteger) \\
-set($p_{10}$, intval, $i_{9}$) \\
-jump($p_{15}$, $p_{10}$) \\
-\end{tabular}
-}
+\begin{lstlisting}[mathescape,xleftmargin=20pt]
+$p_{15}$ = new(BoxedInteger)
+set($p_{15}$, intval, $i_{14}$)
+$p_{10}$ = new(BoxedInteger)
+set($p_{10}$, intval, $i_{9}$)
+jump($p_{15}$, $p_{10}$)
+\end{lstlisting}
Note how the operations for creating these two instances have been moved down the
trace. It looks like for these operations we actually didn't win much, because
the objects are still allocated at the end. However, the optimization was still
worthwhile even in this case, because some operations that have been performed
-on the lifted static objects have been removed (some \texttt{get} operations
-and \texttt{guard\_class} operations).
+on the lifted static objects have been removed (some \lstinline{get} operations
+and \lstinline{guard_class} operations).
\begin{figure}
\includegraphics{figures/step1.pdf}
@@ -652,7 +674,7 @@
The final optimized trace of the example can be seen in Figure~\ref{fig:step1}.
The optimized trace contains only two allocations, instead of the original five,
-and only three \texttt{guard\_class} operations, from the original seven.
+and only three \lstinline{guard_class} operations, from the original seven.
\section{Formal Description of the Algorithm}
\label{sec:formal}
@@ -699,9 +721,9 @@
as those are the only ones that are actually optimized. Without loss of
generality we also consider only objects with two fields in this section.
-Traces are lists of operations. The operations considered here are \texttt{new} (to make
-a new object), \texttt{get} (to read a field out of an object), \texttt{set} (to write a field
-into an object) and \texttt{guard\_class} (to check the type of an object). The values of all
+Traces are lists of operations. The operations considered here are \lstinline{new} (to make
+a new object), \lstinline{get} (to read a field out of an object), \lstinline{set} (to write a field
+into an object) and \lstinline{guard_class} (to check the type of an object). The values of all
variables are locations (i.e.~pointers). Locations are mapped to objects, which
are represented by triples of a type $T$, and two locations that represent the
fields of the object. When a new object is created, the fields are initialized
@@ -729,19 +751,19 @@
$E[v\mapsto l]$ denotes the environment which is just like $E$, but maps $v$ to
$l$.
-The \texttt{new} operation creates a new object $(T,\mathrm{null},\mathrm{null})$ on the
+The \lstinline{new} operation creates a new object $(T,\mathrm{null},\mathrm{null})$ on the
heap under a fresh location $l$ and adds the result variable to the environment,
mapping it to the new location $l$.
-The \texttt{get} operation reads a field $F$ out of an object, and adds the result
+The \lstinline{get} operation reads a field $F$ out of an object, and adds the result
variable to the environment, mapping it to the read location. The heap is
unchanged.
-The \texttt{set} operation changes field $F$ of an object stored at the location that
+The \lstinline{set} operation changes field $F$ of an object stored at the location that
variable $v$ maps to. The new value of the field is the location in variable
$u$. The environment is unchanged.
-The \texttt{guard\_class} operation is used to check whether the object stored at the location
+The \lstinline{guard_class} operation is used to check whether the object stored at the location
that variable $v$ maps to is of type $T$. If that is the case, then execution
continues without changing heap and environment. Otherwise, execution is
stopped.
@@ -817,7 +839,7 @@
fields of objects in the static heap are also elements of $V^*$ (or null, for
short periods of time).
-When the optimizer sees a \texttt{new} operation, it optimistically removes it and
+When the optimizer sees a \lstinline{new} operation, it optimistically removes it and
assumes that the resulting object can stay static. The optimization for all
further operations is split into two cases. One case is for when the
involved variables are in the static heap, which means that the operation can be
@@ -825,21 +847,21 @@
the execution semantics closely. The other case is for when not enough is known about
the variables, and the operation has to be residualized.
-If the argument $v$ of a \texttt{get} operation is mapped to something in the static
-heap, the \texttt{get} can be performed at optimization time. Otherwise, the \texttt{get}
+If the argument $v$ of a \lstinline{get} operation is mapped to something in the static
+heap, the \lstinline{get} can be performed at optimization time. Otherwise, the \lstinline{get}
operation needs to be residualized.
-If the first argument $v$ to a \texttt{set} operation is mapped to something in the
-static heap, then the \texttt{set} can performed at optimization time and the static heap
-updated. Otherwise the \texttt{set} operation needs to be residualized. This needs to be
+If the first argument $v$ to a \lstinline{set} operation is mapped to something in the
+static heap, then the \lstinline{set} can performed at optimization time and the static heap
+updated. Otherwise the \lstinline{set} operation needs to be residualized. This needs to be
done carefully, because the new value for the field, from the variable $u$,
could itself be static, in which case it needs to be lifted first.
-If a \texttt{guard\_class} is performed on a variable that is in the static heap, the type check
+If a \lstinline{guard_class} is performed on a variable that is in the static heap, the type check
can be performed at optimization time, which means the operation can be removed
if the types match. If the type check fails statically or if the object is not
-in the static heap, the \texttt{guard\_class} is residualized. This also needs to
-lift the variable on which the \texttt{guard\_class} is performed.
+in the static heap, the \lstinline{guard_class} is residualized. This also needs to
+lift the variable on which the \lstinline{guard_class} is performed.
Lifting takes a variable that is potentially in the static heap and makes sure
that it is turned into a dynamic variable. This means that operations are
@@ -849,7 +871,7 @@
Lifting a static object needs to recursively lift its fields. Some care needs to
be taken when lifting a static object, because the structures described by the
static heap can be cyclic. To make sure that the same static object is not lifted
-twice, the \texttt{liftfield} operation removes it from the static heap \emph{before}
+twice, the \lstinline{liftfield} operation removes it from the static heap \emph{before}
recursively lifting its fields.
From afa at codespeak.net Wed Oct 13 17:19:08 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 13 Oct 2010 17:19:08 +0200 (CEST)
Subject: [pypy-svn] r77883 - in pypy/branch/fast-forward: lib_pypy
lib_pypy/ctypes_config_cache lib_pypy/pypy_test
pypy/module/_ssl pypy/rlib
Message-ID: <20101013151908.B535E282BFD@codespeak.net>
Author: afa
Date: Wed Oct 13 17:19:07 2010
New Revision: 77883
Removed:
pypy/branch/fast-forward/lib_pypy/_hashlib.py
pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/hashlib.ctc.py
pypy/branch/fast-forward/lib_pypy/pypy_test/test_hashlib.py
Modified:
pypy/branch/fast-forward/pypy/module/_ssl/__init__.py
pypy/branch/fast-forward/pypy/module/_ssl/interp_ssl.py
pypy/branch/fast-forward/pypy/rlib/ropenssl.py
Log:
Remove the ctypes-based _hashlib,
and have _ssl use the new rlib.ropenssl functions
Modified: pypy/branch/fast-forward/pypy/module/_ssl/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_ssl/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/_ssl/__init__.py Wed Oct 13 17:19:07 2010
@@ -26,5 +26,5 @@
super(Module, cls).buildloaders()
def startup(self, space):
- from pypy.module._ssl.interp_ssl import _init_ssl
- _init_ssl()
+ from pypy.rlib.ropenssl import init_ssl
+ init_ssl()
Modified: pypy/branch/fast-forward/pypy/module/_ssl/interp_ssl.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_ssl/interp_ssl.py (original)
+++ pypy/branch/fast-forward/pypy/module/_ssl/interp_ssl.py Wed Oct 13 17:19:07 2010
@@ -3,42 +3,12 @@
from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable
from pypy.interpreter.typedef import TypeDef
from pypy.interpreter.gateway import interp2app
-from pypy.rpython.tool import rffi_platform
-from pypy.translator.platform import platform
-from pypy.translator.tool.cbuild import ExternalCompilationInfo
from pypy.rlib import rpoll
+from pypy.rlib.ropenssl import *
import sys
-if sys.platform == 'win32' and platform.name != 'mingw32':
- libraries = ['libeay32', 'ssleay32', 'user32', 'advapi32', 'gdi32']
- includes = [
- # ssl.h includes winsock.h, which will conflict with our own
- # need of winsock2. Remove this when separate compilation is
- # available...
- 'winsock2.h',
- # wincrypt.h defines X509_NAME, include it here
- # so that openssl/ssl.h can repair this nonsense.
- 'wincrypt.h',
- 'openssl/ssl.h',
- 'openssl/err.h']
-else:
- libraries = ['ssl', 'crypto']
- includes = ['openssl/ssl.h', 'openssl/err.h']
-
-eci = ExternalCompilationInfo(
- libraries = libraries,
- includes = includes,
- export_symbols = ['SSL_load_error_strings'],
- )
-
-eci = rffi_platform.configure_external_library(
- 'openssl', eci,
- [dict(prefix='openssl-',
- include_dir='inc32', library_dir='out32'),
- ])
-
## user defined constants
X509_NAME_MAXLEN = 256
## # these mirror ssl.h
@@ -60,53 +30,8 @@
SOCKET_HAS_TIMED_OUT, SOCKET_HAS_BEEN_CLOSED = 2, 3
SOCKET_TOO_LARGE_FOR_SELECT, SOCKET_OPERATION_OK = 4, 5
-# WinSock does not use a bitmask in select, and uses
-# socket handles greater than FD_SETSIZE
-if sys.platform == 'win32':
- MAX_FD_SIZE = None
-else:
- from pypy.rlib._rsocket_rffi import FD_SETSIZE as MAX_FD_SIZE
-
HAVE_RPOLL = True # Even win32 has rpoll.poll
-class CConfig:
- _compilation_info_ = eci
-
- OPENSSL_VERSION_NUMBER = rffi_platform.ConstantInteger(
- "OPENSSL_VERSION_NUMBER")
- SSLEAY_VERSION = rffi_platform.DefinedConstantString(
- "SSLEAY_VERSION", "SSLeay_version(SSLEAY_VERSION)")
- SSL_FILETYPE_PEM = rffi_platform.ConstantInteger("SSL_FILETYPE_PEM")
- SSL_OP_ALL = rffi_platform.ConstantInteger("SSL_OP_ALL")
- SSL_VERIFY_NONE = rffi_platform.ConstantInteger("SSL_VERIFY_NONE")
- SSL_ERROR_WANT_READ = rffi_platform.ConstantInteger(
- "SSL_ERROR_WANT_READ")
- SSL_ERROR_WANT_WRITE = rffi_platform.ConstantInteger(
- "SSL_ERROR_WANT_WRITE")
- SSL_ERROR_ZERO_RETURN = rffi_platform.ConstantInteger(
- "SSL_ERROR_ZERO_RETURN")
- SSL_ERROR_WANT_X509_LOOKUP = rffi_platform.ConstantInteger(
- "SSL_ERROR_WANT_X509_LOOKUP")
- SSL_ERROR_WANT_CONNECT = rffi_platform.ConstantInteger(
- "SSL_ERROR_WANT_CONNECT")
- SSL_ERROR_SYSCALL = rffi_platform.ConstantInteger("SSL_ERROR_SYSCALL")
- SSL_ERROR_SSL = rffi_platform.ConstantInteger("SSL_ERROR_SSL")
- SSL_CTRL_OPTIONS = rffi_platform.ConstantInteger("SSL_CTRL_OPTIONS")
- BIO_C_SET_NBIO = rffi_platform.ConstantInteger("BIO_C_SET_NBIO")
-
-for k, v in rffi_platform.configure(CConfig).items():
- globals()[k] = v
-
-# opaque structures
-SSL_METHOD = rffi.COpaquePtr('SSL_METHOD')
-SSL_CTX = rffi.COpaquePtr('SSL_CTX')
-SSL = rffi.COpaquePtr('SSL')
-BIO = rffi.COpaquePtr('BIO')
-X509 = rffi.COpaquePtr('X509')
-X509_NAME = rffi.COpaquePtr('X509_NAME')
-
-HAVE_OPENSSL_RAND = OPENSSL_VERSION_NUMBER >= 0x0090500f
-
constants = {}
constants["SSL_ERROR_ZERO_RETURN"] = PY_SSL_ERROR_ZERO_RETURN
constants["SSL_ERROR_WANT_READ"] = PY_SSL_ERROR_WANT_READ
@@ -137,54 +62,11 @@
constants["OPENSSL_VERSION_INFO"] = (major, minor, fix, patch, status)
constants["OPENSSL_VERSION"] = SSLEAY_VERSION
-def ssl_external(name, argtypes, restype, **kw):
- kw['compilation_info'] = eci
- globals()['libssl_' + name] = rffi.llexternal(
- name, argtypes, restype, **kw)
-
-ssl_external('SSL_load_error_strings', [], lltype.Void)
-ssl_external('SSL_library_init', [], rffi.INT)
-if HAVE_OPENSSL_RAND:
- ssl_external('RAND_add', [rffi.CCHARP, rffi.INT, rffi.DOUBLE], lltype.Void)
- ssl_external('RAND_status', [], rffi.INT)
- ssl_external('RAND_egd', [rffi.CCHARP], rffi.INT)
-ssl_external('SSL_CTX_new', [SSL_METHOD], SSL_CTX)
-ssl_external('SSLv23_method', [], SSL_METHOD)
-ssl_external('SSL_CTX_use_PrivateKey_file', [SSL_CTX, rffi.CCHARP, rffi.INT], rffi.INT)
-ssl_external('SSL_CTX_use_certificate_chain_file', [SSL_CTX, rffi.CCHARP], rffi.INT)
-ssl_external('SSL_CTX_ctrl', [SSL_CTX, rffi.INT, rffi.INT, rffi.VOIDP], rffi.INT)
-ssl_external('SSL_CTX_set_verify', [SSL_CTX, rffi.INT, rffi.VOIDP], lltype.Void)
-ssl_external('SSL_new', [SSL_CTX], SSL)
-ssl_external('SSL_set_fd', [SSL, rffi.INT], rffi.INT)
-ssl_external('BIO_ctrl', [BIO, rffi.INT, rffi.INT, rffi.VOIDP], rffi.INT)
-ssl_external('SSL_get_rbio', [SSL], BIO)
-ssl_external('SSL_get_wbio', [SSL], BIO)
-ssl_external('SSL_set_connect_state', [SSL], lltype.Void)
-ssl_external('SSL_connect', [SSL], rffi.INT)
-ssl_external('SSL_get_error', [SSL, rffi.INT], rffi.INT)
-
-ssl_external('ERR_get_error', [], rffi.INT)
-ssl_external('ERR_error_string', [rffi.ULONG, rffi.CCHARP], rffi.CCHARP)
-ssl_external('SSL_get_peer_certificate', [SSL], X509)
-ssl_external('X509_get_subject_name', [X509], X509_NAME)
-ssl_external('X509_get_issuer_name', [X509], X509_NAME)
-ssl_external('X509_NAME_oneline', [X509_NAME, rffi.CCHARP, rffi.INT], rffi.CCHARP)
-ssl_external('X509_free', [X509], lltype.Void)
-ssl_external('SSL_free', [SSL], lltype.Void)
-ssl_external('SSL_CTX_free', [SSL_CTX], lltype.Void)
-ssl_external('SSL_write', [SSL, rffi.CCHARP, rffi.INT], rffi.INT)
-ssl_external('SSL_pending', [SSL], rffi.INT)
-ssl_external('SSL_read', [SSL, rffi.CCHARP, rffi.INT], rffi.INT)
-
def ssl_error(space, msg):
w_module = space.getbuiltinmodule('_ssl')
w_exception = space.getattr(w_module, space.wrap('SSLError'))
return OperationError(w_exception, space.wrap(msg))
-def _init_ssl():
- libssl_SSL_load_error_strings()
- libssl_SSL_library_init()
-
if HAVE_OPENSSL_RAND:
# helper routines for seeding the SSL PRNG
def RAND_add(space, string, entropy):
Modified: pypy/branch/fast-forward/pypy/rlib/ropenssl.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/ropenssl.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/ropenssl.py Wed Oct 13 17:19:07 2010
@@ -35,27 +35,6 @@
include_dir='inc32', library_dir='out32'),
])
-## user defined constants
-X509_NAME_MAXLEN = 256
-## # these mirror ssl.h
-PY_SSL_ERROR_NONE, PY_SSL_ERROR_SSL = 0, 1
-PY_SSL_ERROR_WANT_READ, PY_SSL_ERROR_WANT_WRITE = 2, 3
-PY_SSL_ERROR_WANT_X509_LOOKUP = 4
-PY_SSL_ERROR_SYSCALL = 5 # look at error stack/return value/errno
-PY_SSL_ERROR_ZERO_RETURN, PY_SSL_ERROR_WANT_CONNECT = 6, 7
-# start of non ssl.h errorcodes
-PY_SSL_ERROR_EOF = 8 # special case of SSL_ERROR_SYSCALL
-PY_SSL_ERROR_INVALID_ERROR_CODE = 9
-
-PY_SSL_CERT_NONE, PY_SSL_CERT_OPTIONAL, PY_SSL_CERT_REQUIRED = 0, 1, 2
-
-(PY_SSL_VERSION_SSL2, PY_SSL_VERSION_SSL3,
- PY_SSL_VERSION_SSL23, PY_SSL_VERSION_TLS1) = range(4)
-
-SOCKET_IS_NONBLOCKING, SOCKET_IS_BLOCKING = 0, 1
-SOCKET_HAS_TIMED_OUT, SOCKET_HAS_BEEN_CLOSED = 2, 3
-SOCKET_TOO_LARGE_FOR_SELECT, SOCKET_OPERATION_OK = 4, 5
-
# WinSock does not use a bitmask in select, and uses
# socket handles greater than FD_SETSIZE
if sys.platform == 'win32':
@@ -63,8 +42,6 @@
else:
from pypy.rlib._rsocket_rffi import FD_SETSIZE as MAX_FD_SIZE
-HAVE_RPOLL = True # Even win32 has rpoll.poll
-
class CConfig:
_compilation_info_ = eci
@@ -103,36 +80,6 @@
HAVE_OPENSSL_RAND = OPENSSL_VERSION_NUMBER >= 0x0090500f
-constants = {}
-constants["SSL_ERROR_ZERO_RETURN"] = PY_SSL_ERROR_ZERO_RETURN
-constants["SSL_ERROR_WANT_READ"] = PY_SSL_ERROR_WANT_READ
-constants["SSL_ERROR_WANT_WRITE"] = PY_SSL_ERROR_WANT_WRITE
-constants["SSL_ERROR_WANT_X509_LOOKUP"] = PY_SSL_ERROR_WANT_X509_LOOKUP
-constants["SSL_ERROR_SYSCALL"] = PY_SSL_ERROR_SYSCALL
-constants["SSL_ERROR_SSL"] = PY_SSL_ERROR_SSL
-constants["SSL_ERROR_WANT_CONNECT"] = PY_SSL_ERROR_WANT_CONNECT
-constants["SSL_ERROR_EOF"] = PY_SSL_ERROR_EOF
-constants["SSL_ERROR_INVALID_ERROR_CODE"] = PY_SSL_ERROR_INVALID_ERROR_CODE
-
-constants["CERT_NONE"] = PY_SSL_CERT_NONE
-constants["CERT_OPTIONAL"] = PY_SSL_CERT_OPTIONAL
-constants["CERT_REQUIRED"] = PY_SSL_CERT_REQUIRED
-
-constants["PROTOCOL_SSLv2"] = PY_SSL_VERSION_SSL2
-constants["PROTOCOL_SSLv3"] = PY_SSL_VERSION_SSL3
-constants["PROTOCOL_SSLv23"] = PY_SSL_VERSION_SSL23
-constants["PROTOCOL_TLSv1"] = PY_SSL_VERSION_TLS1
-
-constants["OPENSSL_VERSION_NUMBER"] = OPENSSL_VERSION_NUMBER
-ver = OPENSSL_VERSION_NUMBER
-ver, status = divmod(ver, 16)
-ver, patch = divmod(ver, 256)
-ver, fix = divmod(ver, 256)
-ver, minor = divmod(ver, 256)
-ver, major = divmod(ver, 256)
-constants["OPENSSL_VERSION_INFO"] = (major, minor, fix, patch, status)
-constants["OPENSSL_VERSION"] = SSLEAY_VERSION
-
def external(name, argtypes, restype, **kw):
kw['compilation_info'] = eci
return rffi.llexternal(
@@ -198,7 +145,7 @@
EVP_MD_CTX_cleanup = external(
'EVP_MD_CTX_cleanup', [EVP_MD_CTX], rffi.INT)
-def _init_ssl():
+def init_ssl():
libssl_SSL_load_error_strings()
libssl_SSL_library_init()
From antocuni at codespeak.net Wed Oct 13 17:19:30 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Wed, 13 Oct 2010 17:19:30 +0200 (CEST)
Subject: [pypy-svn] r77884 - pypy/extradoc/talk/pepm2011
Message-ID: <20101013151930.8947D282BFE@codespeak.net>
Author: antocuni
Date: Wed Oct 13 17:19:28 2010
New Revision: 77884
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
don't highlight get/set/new/guard_class in bold
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Wed Oct 13 17:19:28 2010
@@ -28,7 +28,7 @@
commentstyle=\color{gray}\textit,
fancyvrb=true,
showstringspaces=false,
- keywords={def,while,if,elif,return,class,get,set,new,guard_class}
+ %keywords={def,while,if,elif,return,class,get,set,new,guard_class}
}
From arigo at codespeak.net Wed Oct 13 17:22:16 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 13 Oct 2010 17:22:16 +0200 (CEST)
Subject: [pypy-svn] r77885 - pypy/trunk/pypy/module/gc
Message-ID: <20101013152216.603A9282BFF@codespeak.net>
Author: arigo
Date: Wed Oct 13 17:22:14 2010
New Revision: 77885
Modified:
pypy/trunk/pypy/module/gc/referents.py
Log:
Yet another attempt.
Modified: pypy/trunk/pypy/module/gc/referents.py
==============================================================================
--- pypy/trunk/pypy/module/gc/referents.py (original)
+++ pypy/trunk/pypy/module/gc/referents.py Wed Oct 13 17:22:14 2010
@@ -15,12 +15,16 @@
def try_cast_gcref_to_w_root(gcref):
w_obj = rgc.try_cast_gcref_to_instance(W_Root, gcref)
- # Ignore instances of exactly the class W_Root. Seems that
- # we get one on 64-bit translations, for some reason.
- if w_obj is not None and type(w_obj) is W_Root:
- w_obj = None
- if not we_are_translated() and not hasattr(w_obj, 'typedef'):
- w_obj = None
+ # Ignore the instances of W_Root that are not really valid as Python
+ # objects. There is e.g. WeakrefLifeline in module/_weakref that
+ # inherits from W_Root for internal reasons. Such instances don't
+ # have a typedef at all (or have a null typedef after translation).
+ if not we_are_translated():
+ if not hasattr(w_obj, 'typedef'):
+ return None
+ else:
+ if w_obj is None or not w_obj.typedef:
+ return None
return w_obj
def wrap(space, gcref):
From cfbolz at codespeak.net Wed Oct 13 17:29:56 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Wed, 13 Oct 2010 17:29:56 +0200 (CEST)
Subject: [pypy-svn] r77886 - in pypy/extradoc/talk/pepm2011: . benchmarks
Message-ID: <20101013152956.0A082282C00@codespeak.net>
Author: cfbolz
Date: Wed Oct 13 17:29:53 2010
New Revision: 77886
Modified:
pypy/extradoc/talk/pepm2011/benchmarks/bench.gnumeric
pypy/extradoc/talk/pepm2011/paper.bib
pypy/extradoc/talk/pepm2011/paper.tex
Log:
Try to solve the XXXs of Michael. Add a new figure with the operation numbers.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/bench.gnumeric
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.bib
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.bib (original)
+++ pypy/extradoc/talk/pepm2011/paper.bib Wed Oct 13 17:29:53 2010
@@ -122,6 +122,14 @@
pages = {217--242}
},
+ at phdthesis{cuni_high_2010,
+ title = {High performance implementation of Python for {CLI/.NET} with {JIT} compiler generation for dynamic languages.},
+ school = {Dipartimento di Informatica e Scienze {dell'Informazione,} University of Genova},
+ author = {Antonio Cuni},
+ year = {2010},
+ note = {Technical Report {DISI-TH-2010-05}}
+},
+
@inproceedings{chang_tracing_2009,
address = {Washington, {DC,} {USA}},
title = {Tracing for Web 3.0: Trace Compilation for the Next Generation Web Applications},
@@ -240,7 +248,6 @@
pages = {18--25}
},
-
@techreport{mason_chang_efficient_2007,
title = {Efficient {Just-In-Time} Execution of Dynamically Typed Languages
Via Code Specialization Using Precise Runtime Type Inference},
@@ -317,13 +324,3 @@
year = {2008},
pages = {123--139}
}
-
-
- at phdthesis{cuni_python_cli_2010,
- author = {Antonio Cuni},
- title = {High performance implementation of {Python} for {CLI/.NET} with
- {JIT} compiler generation for dynamic languages.},
- school = {Dipartimento di Informatica e Scienze dell'Informazione, University of Genova},
- note = {Technical Report {DISI-TH-2010-05}},
- year = {2010},
-},
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Wed Oct 13 17:29:53 2010
@@ -54,6 +54,7 @@
\newcommand\revb[1]{\nb{Reviewer 2}{#1}}
\newcommand\revc[1]{\nb{Reviewer 3}{#1}}
\newcommand{\commentout}[1]{}
+\newcommand{\ignore}[1]{} % {{\tt \small ignore(#1)}}
\newcommand\ie{i.e.,\xspace}
\newcommand\eg{e.g.,\xspace}
@@ -234,7 +235,7 @@
to add new \emph{backends} for producing the actual machine code, in addition
to the original backend for the Intel \emph{x86} architecture. Examples of
additional JIT backends are the one for Intel \emph{x86-64} and an
-experimental one for the CLI .NET Virtual Machine \cite{cuni_python_cli_2010}.
+experimental one for the CLI .NET Virtual Machine \cite{cuni_high_2010}.
PyPy's JIT generator generates a \emph{tracing JIT
compiler}, a concept which we now explain in more details.
@@ -266,7 +267,7 @@
tracing JIT automatically performs inlining.
This trace of operations is then the basis of the generated code. The trace is
-first optimized, and then turned into machine code. Both optimizations
+first optimized, and then turned into machine code. Both optimization
and machine code generation is simple, because the traces are linear. This
linearity makes many optimizations a lot more tractable, and the inlining that
happens gives the optimizations automatically more context to work with.
@@ -310,28 +311,10 @@
The classes can be seen in
Figure~\ref{fig:objmodel} (written in RPython).
-\newcommand{\ignore}[1]{} % {{\tt \small ignore(#1)}}
-\ignore{
-class Base(object):
- def add(self, other):
- """ add self to other """
- raise NotImplementedError("abstract base")
- def add__int(self, intother):
- """ add intother to self,
- where intother is an integer """
- raise NotImplementedError("abstract base")
- def add__float(self, floatother):
- """ add floatother to self,
- where floatother is a float """
- raise NotImplementedError("abstract base")
- def is_positive(self):
- """ returns whether self is positive """
- raise NotImplementedError("abstract base")
- }
\begin{figure}
\begin{lstlisting}[mathescape]
class Base(object):
- ...
+ pass
class BoxedInteger(Base):
def __init__(self, intval):
@@ -375,8 +358,7 @@
y = y.add(BoxedInteger(-1))
return res
\end{lstlisting}
-\caption{An ``interpreter'' for a tiny Dynamic Language written in RPython}
-%\caption{A Simple Object Model and an Example Function Using it}
+\caption{An ``Interpreter'' for a Tiny Dynamic Language Written in RPython}
\label{fig:objmodel}
\end{figure}
@@ -389,15 +371,9 @@
which is costly due to the method dispatch.
To understand the problems more directly, let us consider the simple
-interpreter function
-\lstinline{f} that uses the object model (see the bottom of
+interpreter function \lstinline{f} that uses the object model (see the bottom of
Figure~\ref{fig:objmodel}).
-XXX this is not an RPython interpreter; put a reference to the previous
-paper to show how we deal with an interpreted piece of code and remove
-the interpretation overhead, turning it into basically something
-equivalent to the example here, which is the start of the present paper.
-
The loop in \lstinline{f} iterates \lstinline{y} times, and computes something in the process.
Simply running this function is slow, because there are lots of virtual method
calls inside the loop, one for each \lstinline{is_positive} and even two for each
@@ -474,22 +450,24 @@
\lstinline{BoxedInteger}, the produced trace looks like
Figure~\ref{fig:unopt-trace} (lines starting with the hash ``\#'' are comments).
-XXX in which language is the trace written in ? still RPython ?
-
The operations in the trace are shown indented to
correspond to the stack level of the function that contains the traced
operation. The trace is in single-assignment form, meaning that each variable is
assigned to exactly once. The arguments $p_0$ and $p_1$ of the loop correspond
to the live variables \lstinline{y} and \lstinline{res} in the original function.
-XXX explain set and get + int\_add briefly
+The operations in the trace correspond to the operations in the RPython program
+in Figure~\ref{fig:objmodel}:
-The trace shows the inefficiencies of \lstinline{f} clearly, if one
-looks at the number of \lstinline{new} (corresponding to object creation),
-\lstinline{set/get} (corresponding to attribute reads/writes) and
-\lstinline{guard_class} operations (corresponding to method calls).
-In the rest of the paper we will see how this trace can be optimized using
-partial evaluation.
+\begin{itemize}
+ \item \lstinline{new} corresponds to object creation.
+ \item \lstinline{get} correspond to attribute reads.
+ \item \lstinline{set} correspond to attribute writes.
+ \item \lstinline{guard\_class} correspond to method calls and are followed by
+ the trace of the called method.
+ \item \lstinline{int\_add} and \lstinline{int\_get} are integer addition and
+ comparison (``greater than''), respectively.
+\end{itemize}
Note how the functions that are called by \lstinline{f} are automatically inlined
into the trace. The method calls are always preceded by a \lstinline{guard_class}
@@ -502,6 +480,10 @@
first \lstinline{guard_class} instruction will fail and execution will continue
using the interpreter.
+The trace shows the inefficiencies of \lstinline{f} clearly, if one looks at the
+number of \lstinline{new}, \lstinline{set/get} and \lstinline{guard\_class} operations.
+In the rest of the paper we will see how this trace can be optimized using
+partial evaluation.
\section{Object Lifetimes in a Tracing JIT}
\label{sec:lifetimes}
@@ -576,7 +558,7 @@
This is a process that is usually called \emph{escape analysis}. In this paper we will
perform escape analysis by using partial evaluation. The partial evalution is a
bit peculiar in that there are not actually any constant arguments to the trace,
-but it is only used to optimized operations within a trace. XXX mention Prolog.
+but it is only used to optimized operations within a trace.
The partial evaluation works by walking the trace from beginning to end.
Whenever a \lstinline{new} operation is seen, the operation is removed and a static
@@ -989,25 +971,29 @@
allocation operations.
\begin{figure*}
-\begin{tabular}{lrrrrrrrrrrrrrrrrrrrrrrr}
- &\#loops &new & & &get & & &set & & &guard & & &rest & & & & & &total & &\\
-crypto\_pyaes &75 &1663 &446 &27\% &18842 &2127 &11\% &14552 &2020 &14\% &8952 &235 &3\% &53339 &7794 &15\% &44009 &4828 &11\% &97348 &12622 &13\%\\
-django &53 &453 &155 &34\% &11970 &1457 &12\% &8738 &991 &11\% &4093 &275 &7\% &33988 &4076 &12\% &25254 &2878 &11\% &59242 &6954 &12\%\\
-fannkuch &40 &161 &76 &47\% &574 &306 &53\% &278 &139 &50\% &1121 &202 &18\% &2581 &1685 &65\% &2134 &723 &34\% &4715 &2408 &51\%\\
-go &510 &5604 &1651 &29\% &112403 &12612 &11\% &88439 &11915 &13\% &51621 &3194 &6\% &300831 &30834 &10\% &258067 &29372 &11\% &558898 &60206 &11\%\\
-html5lib &416 &5939 &1793 &30\% &241402 &15863 &7\% &178723 &13960 &8\% &58486 &3037 &5\% &682654 &40341 &6\% &484550 &34653 &7\% &1167204 &74994 &6\%\\
-meteor-contest &56 &221 &141 &64\% &2326 &606 &26\% &2047 &645 &32\% &1040 &169 &16\% &7018 &2250 &32\% &5634 &1561 &28\% &12652 &3811 &30\%\\
-nbody &10 &84 &48 &57\% &287 &159 &55\% &121 &74 &61\% &411 &90 &22\% &986 &737 &75\% &903 &371 &41\% &1889 &1108 &59\%\\
-pyflate-fast &161 &1878 &723 &38\% &22815 &2592 &11\% &18415 &3690 &20\% &8291 &628 &8\% &64649 &11327 &18\% &51399 &7633 &15\% &116048 &18960 &16\%\\
-raytrace-simple &117 &1906 &359 &19\% &52995 &2202 &4\% &39518 &1823 &5\% &13451 &447 &3\% &140723 &6533 &5\% &107870 &4831 &4\% &248593 &11364 &5\%\\
-richards &79 &375 &191 &51\% &27507 &2095 &8\% &21217 &2004 &9\% &3972 &332 &8\% &76751 &6171 &8\% &53071 &4622 &9\% &129822 &10793 &8\%\\
-spambayes &267 &2413 &505 &21\% &53327 &3033 &6\% &40938 &2428 &6\% &19343 &718 &4\% &143575 &10322 &7\% &116021 &6684 &6\% &259596 &17006 &7\%\\
-spectral-norm &35 &266 &88 &33\% &3004 &398 &13\% &2544 &414 &16\% &1084 &78 &7\% &4803 &1184 &25\% &6898 &978 &14\% &11701 &2162 &18\%\\
-telco &42 &660 &70 &11\% &20716 &424 &2\% &14444 &608 &4\% &6574 &56 &1\% &53674 &1224 &2\% &42394 &1158 &3\% &96068 &2382 &2\%\\
-twisted-names &201 &2706 &317 &12\% &64438 &2795 &4\% &47960 &1968 &4\% &24091 &555 &2\% &171582 &6828 &4\% &139195 &5635 &4\% &310777 &12463 &4\%\\
-total &2062 &24329 &6563 &27\% &632606 &46669 &7\% &477934 &42679 &9\% &202530 &10016 &5\% &1737154 &131306 &8\% &1337399 &105927 &8\% &3074553 &237233 &8\%\\
+\begin{tabular}{|l||r|rr|rr|rr|rr|}
+\hline
+ &num loops &new &removed &get/set &removed &guard &removed &all ops &removed\\
+\hline
+crypto\_pyaes &78 &3088 &50\% &57148 &25\% &9055 &95\% &137189 &80\%\\
+django &51 &673 &54\% &19318 &18\% &3876 &93\% &55682 &85\%\\
+fannkuch &43 &171 &49\% &886 &63\% &1159 &81\% &4935 &45\%\\
+go &517 &12234 &76\% &200842 &21\% &53138 &90\% &568542 &84\%\\
+html5lib &498 &14432 &68\% &503390 &11\% &71592 &94\% &1405780 &91\%\\
+meteor-contest &59 &277 &36\% &4402 &31\% &1078 &83\% &12862 &68\%\\
+nbody &13 &96 &38\% &443 &69\% &449 &78\% &2107 &38\%\\
+pyflate-fast &162 &2278 &55\% &39126 &20\% &8194 &92\% &112857 &80\%\\
+raytrace-simple &120 &3118 &59\% &91982 &15\% &13572 &95\% &247436 &89\%\\
+richards &87 &844 &4\% &49875 &22\% &4130 &91\% &133898 &83\%\\
+spambayes &314 &5608 &79\% &117002 &11\% &25313 &94\% &324125 &90\%\\
+spectral-norm &38 &360 &64\% &5553 &20\% &1122 &92\% &11878 &77\%\\
+telco &46 &1257 &90\% &37470 &3\% &6644 &99\% &98590 &97\%\\
+twisted-names &214 &5273 &84\% &100010 &10\% &23247 &96\% &279667 &92\%\\
+\hline
+total &2240 &49709 &70\% &1227447 &14\% &222569 &93\% &3395548 &89\%\\
+\hline
\end{tabular}
-\caption{Number of Operations Before and After Optimization}
+\caption{Number of Operations and Percentage Removed By Optimization}
\label{fig:numops}
\end{figure*}
From cfbolz at codespeak.net Wed Oct 13 17:46:00 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Wed, 13 Oct 2010 17:46:00 +0200 (CEST)
Subject: [pypy-svn] r77887 - pypy/extradoc/talk/pepm2011
Message-ID: <20101013154600.4F9BF282BDA@codespeak.net>
Author: cfbolz
Date: Wed Oct 13 17:45:58 2010
New Revision: 77887
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
improve benchmark section
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Wed Oct 13 17:45:58 2010
@@ -963,12 +963,30 @@
know how many allocations could be optimized away. On the other hand, we want
to know how much the run times of the benchmarks is improved.
-For the former we counted the occurring operations in all generated traces
-before and after the optimization phase for all benchmarks. The results can be
-seen in Figure~\ref{fig:numops}. The optimization removes as many as XXX and as
-little as XXX percent of allocation operations in the traces of the benchmarks.
-All benchmarks taken together, the optimization removes XXX percent of
-allocation operations.
+The benchmarks were run on an otherwise idle Intel Core2 Duo P8400 processor
+with 2.26 GHz and 3072 KB of cache on a machine with 3GB RAM running Linux
+2.6.35. We compared the performance of various Python implementations on the
+benchmarks. As a baseline, we used the standard Python implementation in C,
+CPython 2.6.6\footnote{\texttt{http://python.org}}, which uses a bytecode-based
+interpreter. Furthermore we compared against Psyco 1.6
+\cite{rigo_representation-based_2004}, an extension to CPython which is a
+just-in-time compiler that produces machine code at run-time. It is not based
+on traces. Finally, we used two versions of PyPy's Python interpreter (revision
+77823 of SVN trunk\footnote{\texttt{https://codespeak.net/svn/pypy/trunk}}): one
+including the JIT but not optimizing the traces, and one using the allocation
+removal optimizations (as well as some minor other optimizations, such as
+constant folding).
+
+As the first step, we counted the occurring operations in all generated traces
+before and after the optimization phase for all benchmarks. The resulting
+numbers can be
+seen in Figure~\ref{fig:numops}. The optimization removes as many as 90\% and as
+little as 4\% percent of allocation operations in the traces of the benchmarks.
+All benchmarks taken together, the optimization removes 70\% percent of
+allocation operations. The numbers look similar for reading and writing of
+attributes. There are even more \lstinline{guard} operations that are removed,
+however there is an additional optimization that removes guards, so not all the
+removed guards are an effect of the optimization described here.
\begin{figure*}
\begin{tabular}{|l||r|rr|rr|rr|rr|}
@@ -997,24 +1015,14 @@
\label{fig:numops}
\end{figure*}
-In addition to the count of operations we also performed time measurements. The
-machine the benchmarks were run on is XXX. We compared the performance of
-various Python implementations on the benchmarks. As a baseline, we used the
-standard Python implementation in C, called
-CPython\footnote{\texttt{http://python.org}}, which uses a bytecode-based
-interpreter. Furthermore we compared against Psyco \cite{rigo_representation-based_2004}, an extension to
-CPython which is a just-in-time compiler that produces machine code at run-time.
-It is not based on traces. Finally, we used three versions of PyPy's Python interpreter:
-one without a JIT, one including the JIT but not using the allocation removal
-optimization, and one using the allocation removal optimizations.
-
+In addition to the count of operations we also performed time measurements.
All benchmarks were run 50 times in the same process, to give the JIT time to produce machine
code. The arithmetic mean of the times of the last 30 runs were used as the
result. The errors were computed using a confidence interval with a 95\%
confidence level \cite{georges_statistically_2007}. The results are reported in Figure~\ref{fig:times}.
With the optimization turned on, PyPy's Python interpreter outperforms CPython
in all benchmarks except spambayes (which heavily relies on regular expression
-performance). All benchmarks are improved by the allocation removal
+performance XXX). All benchmarks are improved by the allocation removal
optimization, some as much as XXX. XXX Psyco
XXX runtimes of the algorithm somehow?
From afa at codespeak.net Wed Oct 13 19:08:08 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 13 Oct 2010 19:08:08 +0200 (CEST)
Subject: [pypy-svn] r77888 - in pypy/branch/fast-forward/pypy: config
module/_hashlib
Message-ID: <20101013170808.6385C282BDC@codespeak.net>
Author: afa
Date: Wed Oct 13 19:08:05 2010
New Revision: 77888
Modified:
pypy/branch/fast-forward/pypy/config/pypyoption.py
pypy/branch/fast-forward/pypy/module/_hashlib/__init__.py
pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py
Log:
Various translation fixes + enable _hashlib by default
Modified: pypy/branch/fast-forward/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/fast-forward/pypy/config/pypyoption.py (original)
+++ pypy/branch/fast-forward/pypy/config/pypyoption.py Wed Oct 13 19:08:05 2010
@@ -28,8 +28,8 @@
working_modules.update(dict.fromkeys(
["_socket", "unicodedata", "mmap", "fcntl",
"rctime" , "select", "zipimport", "_lsprof",
- "crypt", "signal", "_rawffi", "termios", "zlib",
- "struct", "_md5", "_sha", "bz2", "_minimal_curses", "cStringIO",
+ "crypt", "signal", "_rawffi", "termios", "zlib", "bz2",
+ "struct", "_hashlib", "_md5", "_sha", "_minimal_curses", "cStringIO",
"thread", "itertools", "pyexpat", "_ssl", "cpyext", "array"]
))
Modified: pypy/branch/fast-forward/pypy/module/_hashlib/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_hashlib/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/_hashlib/__init__.py Wed Oct 13 19:08:05 2010
@@ -1,5 +1,5 @@
from pypy.interpreter.mixedmodule import MixedModule
-from pypy.module._hashlib import interp_hashlib
+from pypy.module._hashlib.interp_hashlib import algorithms
class Module(MixedModule):
@@ -11,5 +11,5 @@
appleveldefs = {
}
- for name in interp_hashlib.algorithms:
- interpleveldefs[name] = getattr(interp_hashlib, 'new_' + name)
+ for name in algorithms:
+ interpleveldefs[name] = 'interp_hashlib.new_%s' % (name,)
Modified: pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py (original)
+++ pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py Wed Oct 13 19:08:05 2010
@@ -1,5 +1,6 @@
from pypy.interpreter.gateway import unwrap_spec, interp2app
from pypy.interpreter.typedef import TypeDef, GetSetProperty
+from pypy.interpreter.error import OperationError
from pypy.tool.sourcetools import func_renamer
from pypy.interpreter.baseobjspace import Wrappable, W_Root, ObjSpace
from pypy.rpython.lltypesystem import lltype, rffi
@@ -9,7 +10,7 @@
algorithms = ('md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512')
class W_Hash(Wrappable):
- def __init__(self, name):
+ def __init__(self, space, name):
self.name = name
self.ctx = lltype.malloc(ropenssl.EVP_MD_CTX.TO, flavor='raw')
@@ -21,7 +22,9 @@
@unwrap_spec('self', ObjSpace)
def descr_repr(self, space):
- return space.wrap("<%s HASH object @ 0x%x>" % (self.name, id(self)))
+ addrstring = self.getaddrstring(space)
+ return space.wrap("<%s HASH object at 0x%s>" % (
+ self.name, addrstring))
@unwrap_spec('self', ObjSpace, str)
def update(self, space, buffer):
@@ -34,7 +37,7 @@
@unwrap_spec('self', ObjSpace)
def copy(self, space):
"Return a copy of the hash object."
- w_hash = W_Hash(self.name)
+ w_hash = W_Hash(space, self.name)
ropenssl.EVP_MD_CTX_copy(w_hash.ctx, self.ctx)
return w_hash
@@ -115,8 +118,8 @@
)
@unwrap_spec(ObjSpace, str, str)
-def new(space, method, string=''):
- w_hash = W_Hash(method)
+def new(space, name, string=''):
+ w_hash = W_Hash(space, name)
w_hash.update(space, string)
return space.wrap(w_hash)
@@ -124,6 +127,7 @@
for name in algorithms:
newname = 'new_%s' % (name,)
@func_renamer(newname)
- def new_hash(w_string=''):
- return _new(name, w_string)
+ @unwrap_spec(ObjSpace, str)
+ def new_hash(space, string=''):
+ return new(space, name, string)
globals()[newname] = new_hash
From fijal at codespeak.net Wed Oct 13 20:20:09 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Wed, 13 Oct 2010 20:20:09 +0200 (CEST)
Subject: [pypy-svn] r77891 - pypy/trunk/pypy/jit/tool
Message-ID: <20101013182009.0954C282BDA@codespeak.net>
Author: fijal
Date: Wed Oct 13 20:20:07 2010
New Revision: 77891
Modified:
pypy/trunk/pypy/jit/tool/traceviewer.py
Log:
Disable counts for now (format has changed)
Modified: pypy/trunk/pypy/jit/tool/traceviewer.py
==============================================================================
--- pypy/trunk/pypy/jit/tool/traceviewer.py (original)
+++ pypy/trunk/pypy/jit/tool/traceviewer.py Wed Oct 13 20:20:07 2010
@@ -253,10 +253,10 @@
def main(loopfile, use_threshold, view=True):
countname = py.path.local(loopfile + '.count')
if countname.check():
- counts = [re.split('( 20 and use_threshold:
counts.threshold = l[-20]
From cfbolz at codespeak.net Wed Oct 13 22:14:31 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Wed, 13 Oct 2010 22:14:31 +0200 (CEST)
Subject: [pypy-svn] r77892 - pypy/extradoc/talk/pepm2011/benchmarks
Message-ID: <20101013201431.EA3D9282BDA@codespeak.net>
Author: cfbolz
Date: Wed Oct 13 22:14:28 2010
New Revision: 77892
Modified:
pypy/extradoc/talk/pepm2011/benchmarks/bench.gnumeric
pypy/extradoc/talk/pepm2011/benchmarks/jit-noops.json
pypy/extradoc/talk/pepm2011/benchmarks/jit-nopspec.json
Log:
hopefully final benchmark numbers
Modified: pypy/extradoc/talk/pepm2011/benchmarks/bench.gnumeric
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/benchmarks/jit-noops.json
==============================================================================
--- pypy/extradoc/talk/pepm2011/benchmarks/jit-noops.json (original)
+++ pypy/extradoc/talk/pepm2011/benchmarks/jit-noops.json Wed Oct 13 22:14:28 2010
@@ -1 +1 @@
-{"options": "", "results": [["crypto_pyaes", "RawResult", {"changed_times": [83.608081817599995, 81.512792825700004, 81.586477994899994, 81.442406892799994, 81.350951910000006, 81.291965007800002, 81.274972915600003, 81.285789966600007, 81.302440881699994, 81.471642970999994, 81.280892133699993, 81.281094074199999, 81.301096916199995, 81.287539005300005, 81.236584901800001, 81.283654928199994, 81.291748046899997, 81.5425839424, 81.236918926200005, 80.983955144899994, 80.997071981399998, 80.963902950299996, 80.967394113500006, 80.967602014500002, 81.2447729111, 81.042499065399994, 81.091964006400005, 81.300304889700001, 81.322211027099996, 81.287165164900003, 81.302358150499998, 81.531341075900002, 81.253934860200005, 81.308516979199993, 81.402919053999995, 81.258775949500006, 81.280184984200005, 81.299252033200005, 81.306777000400004, 81.550753116600006, 81.246633052799993, 81.279681921000005, 81.2769200802, 81.316480874999996, 81.294853925699996, 81.257145166399994, 81.467491149899999, 81.312170982400005, 81.252816200300003, 81.258814811700006], "base_times": [2.80382990837, 2.7956340313000001, 2.8010652065300001, 2.7976541519200002, 2.7950928211199999, 2.8041520118699998, 2.8043060302699998, 2.7999861240400001, 2.8013310432399998, 2.8088479042099999, 2.8007519245100001, 2.8009748458899999, 2.8043870925899999, 2.80448412895, 2.80184221268, 2.80495095253, 2.8023998737300002, 2.8056631088300001, 2.8033299446100002, 2.7996089458500002, 2.8002779483800002, 2.8084959983800002, 2.8017699718500002, 2.8082520961799999, 2.8050520420099998, 2.8028419017799999, 2.7981400489800001, 2.7999629974400002, 2.7998390197799998, 2.8000919818900001, 2.80037903786, 2.79657483101, 2.7981040477799999, 2.8026959896100001, 2.7995419502300001, 2.7954881191299998, 2.7970201969100001, 2.79839706421, 2.7952239513400001, 2.7976598739599998, 2.7945868968999998, 2.7983119487799999, 2.7953550815599999, 2.79377818108, 2.7956900596600001, 2.8036141395600001, 2.79533600807, 2.7930212020899998, 2.7950999736800002, 2.7983438968700001]}], ["django", "RawResult", {"changed_times": [0.94225811958299999, 0.68628406524700003, 0.70013499259900003, 0.68669486045799999, 0.70185899734500001, 0.68997812271100001, 0.70118880271899997, 0.68901705741899999, 0.70103502273600005, 0.68790817260699999, 0.69985699653599998, 0.68816995620700006, 0.73480010032700005, 0.68603014945999996, 0.70323491096500002, 0.68623781204199996, 0.70352911949200003, 0.68471789360000002, 0.70435285568200001, 0.68524813652000005, 0.70314097404499998, 0.68677711486799997, 0.70417714118999997, 0.68562698364300001, 0.70090913772600005, 0.68850708007799999, 0.70100784301800001, 0.68775796890300001, 0.70195102691699995, 0.68907117843599996, 0.70120120048500001, 0.68785810470599995, 0.70091485977199997, 0.68843507766699996, 0.70012712478600003, 0.68629503250099999, 0.70415806770300005, 0.68661308288599998, 0.70444512367199996, 0.68531179428099998, 0.70345211029099997, 0.68501901626600004, 0.704196929932, 0.68532204628000004, 0.70484805107100001, 0.68551492691000004, 0.70121288299600004, 0.68866515159599995, 0.70078611373900002, 0.68853020667999998], "base_times": [0.98770689964299996, 0.98978590965299995, 0.98754811286900002, 0.98878502845799998, 0.98909711837799996, 0.989701032639, 1.0125169754000001, 0.99142193794300004, 0.98840188980099997, 0.99162793159499996, 0.99071907997099995, 0.99076414108300004, 0.98944306373599999, 0.98936295509299998, 0.99107599258400003, 0.99007892608600001, 0.98786211013799996, 0.992697954178, 0.99010705947900002, 0.98694705963100005, 0.98762106895400004, 0.98965787887599999, 0.98832798004199995, 0.98824381828300001, 0.98842883110000002, 0.98763680458100001, 0.99011301994300005, 0.98688387870799998, 0.98969793319699995, 0.98943901061999995, 0.98809719085699999, 0.98793482780499997, 0.98821902275100004, 0.98756289482100001, 0.99030303955099996, 0.98743104934699999, 0.98614192009000001, 0.98893618583700005, 0.98977398872400002, 0.98548102378799995, 0.987926959991, 0.98703289031999997, 0.98901891708400003, 0.98792290687600004, 0.98637413978599997, 0.98895907402000005, 0.98991489410400002, 0.98569011688200003, 0.98801302909900002, 0.99004602432300004]}], ["fannkuch", "RawResult", {"changed_times": [0.65027499198899996, 0.57184004783599995, 0.56566786766099997, 0.56521892547599994, 0.56701707840000004, 0.57264208793600002, 0.56822896003699996, 0.56470012664799996, 0.56810212135299998, 0.56781196594200001, 0.564800024033, 0.56483197212199998, 0.56917119026200003, 0.56767296791099997, 0.56695199012800002, 0.56578302383400003, 0.56515192985499996, 0.57563495635999995, 0.56515789031999997, 0.56409192085299997, 0.56773018837, 0.56481719016999998, 0.56611680984500001, 0.56627202034000002, 0.56611609458900003, 0.56967306137100004, 0.567966938019, 0.56780695915199997, 0.57448220253000004, 0.57842707634000001, 0.56447005271899997, 0.56811785697899997, 0.56486487388600004, 0.56649589538599998, 0.56501197814899995, 0.56486201286299997, 0.56838393211399996, 0.56405186653100003, 0.56555008888199998, 0.56801414489699997, 0.56562018394500002, 0.56669306755100002, 0.56592893600500005, 0.56481790542599997, 0.56828188896200005, 0.56566095352199997, 0.56510496139499999, 0.56719899177599997, 0.56526398658800003, 0.56593322753899999], "base_times": [1.84706997871, 1.8756129741700001, 1.85868191719, 1.8472619056699999, 1.87546992302, 1.8508591651899999, 1.8696298599200001, 1.86460900307, 1.85237002373, 1.87226009369, 1.8722259998299999, 1.8783550262499999, 1.8736140727999999, 1.87361216545, 1.87663817406, 1.8724210262300001, 1.8727889061, 1.8782889842999999, 1.8723690509799999, 1.8713910579699999, 1.8771908283200001, 1.87365293503, 1.87231016159, 1.87713694572, 1.87258410454, 1.87410783768, 1.8863110542299999, 1.87264585495, 1.87151694298, 1.8771209716799999, 1.8736791610700001, 1.8717048168199999, 1.8770380020099999, 1.8727111816399999, 1.87127304077, 1.87773990631, 1.89574599266, 1.87074995041, 1.92922902107, 1.9290540218400001, 1.94486498833, 1.8772900104500001, 1.87182497978, 1.87466216087, 1.8774259090400001, 1.87289023399, 1.87402200699, 1.8763341903699999, 1.8749959468799999, 1.8724749088299999]}], ["go", "RawResult", {"changed_times": [2.3995778560600001, 2.3866791725200001, 2.3967480659499998, 2.5598640441899998, 2.2185318469999999, 2.3486278057100001, 2.3838458061200001, 2.4448201656299999, 2.19373893738, 2.1973280906700001, 2.1919651031499998, 2.5130779743199998, 2.2208979129799999, 2.2443778514899999, 2.3977251052900002, 2.1551439762100002, 2.1595408916499998, 2.23243093491, 2.17227602005, 2.1977610588099998, 2.3166739940599999, 2.2348148822799998, 2.1878528595, 2.1694250106799999, 2.1508538722999999, 2.3760650157900001, 2.1592259406999998, 2.1955871581999999, 2.1416928768200001, 2.2533910274500002, 2.1912820339199999, 2.2604429721799999, 2.2298648357399999, 2.1796870231600001, 2.20620703697, 2.16007018089, 2.1497859954799998, 2.2840979099299998, 2.2118680477099999, 2.1375119686100001, 2.15009498596, 2.1446928978000002, 2.1316239833799999, 2.13385415077, 2.1266629695899999, 2.3893008232100001, 2.1834890842400001, 2.1347019672399998, 2.21711921692, 2.1232719421400001], "base_times": [0.93096089363099999, 0.95089387893699995, 0.93110585212700003, 0.92558383941699995, 0.93034386634800004, 0.92623090744000003, 0.93461894989000005, 0.93881106376599999, 0.92857980728099998, 0.93526601791399999, 0.92670106887799997, 0.93337702751200002, 0.93183684349100004, 0.93682098388699997, 0.92921209335300003, 0.93008303642300005, 0.92924404144299999, 0.93299007415799995, 0.93573307990999999, 0.930922985077, 0.93020296096800004, 0.93374800682100001, 0.93536710739100004, 0.93496298789999999, 0.93623590469399998, 0.93143200874300003, 0.92915606498699999, 0.92980098724399995, 0.94962000846899997, 0.93416810035700004, 0.93491911888099999, 0.92967605590799995, 0.927985906601, 0.934444904327, 0.93304419517500004, 0.93216395378100003, 0.93552398681600002, 0.94005179405200001, 0.93092107772800003, 0.93850493431100002, 0.932507038116, 0.93334603309599995, 0.94100594520600001, 0.93229484558099995, 0.93091201782199995, 0.93197703361499995, 0.938956975937, 0.93856000900299996, 0.93897986412000001, 0.92620086669900004]}], ["html5lib", "RawResult", {"changed_times": [38.3545889854, 29.4800949097, 28.267448186900001, 27.7905261517, 27.673813104600001, 27.295396089600001, 27.640147924400001, 27.573884964000001, 27.469006061599998, 27.410823106799999, 27.163343191100001, 27.421272039400002, 27.498446941400001, 27.377060890199999, 27.862658977500001, 27.3771958351, 27.126015901599999, 27.002305030799999, 27.370562076599999, 27.4734969139, 27.1270251274, 27.124243021000002, 27.327117919900001, 27.165127992599999, 27.1296830177, 27.4667949677, 27.082341194200001, 27.294297933599999, 27.245062112799999, 27.360240936299999, 27.298542022700001, 27.139825105700002, 27.091902017599999, 27.068879127500001, 27.3232738972, 27.168828010599999, 27.2176198959, 27.3889100552, 27.146837949799998, 27.0338580608, 27.022614002200001, 27.345255851699999, 27.146274805099999, 27.0234370232, 27.357608795200001, 27.049774169900001, 27.077157974199999, 27.123055934900002, 27.078799009299999, 27.409001111999999], "base_times": [13.864197969399999, 14.0840089321, 14.0057208538, 14.0325820446, 14.0150129795, 14.1008889675, 14.097244978000001, 14.067807912799999, 14.0274028778, 14.012732029, 13.961874008200001, 14.1108551025, 14.0863909721, 14.0686590672, 14.004349947, 14.185241937600001, 13.9961509705, 14.1126899719, 14.1245219707, 14.086353063600001, 14.048186063799999, 14.0126638412, 13.965003013600001, 14.136936903, 14.125771045700001, 14.0789718628, 14.0491027832, 13.986720800400001, 13.978865861899999, 14.133366823199999, 14.0888400078, 14.118774890899999, 14.0262331963, 14.035698890700001, 13.984946966200001, 14.1640629768, 14.148418188100001, 14.091625928899999, 14.075366020200001, 14.0536768436, 13.9736099243, 14.1238529682, 14.118108987799999, 14.103012085, 14.082417964899999, 14.0419850349, 14.015744209299999, 13.983751058599999, 14.112045049700001, 14.1095650196]}], ["meteor-contest", "RawResult", {"changed_times": [0.54956293106099996, 0.44127202034000002, 0.43488597869899998, 0.43912196159400002, 0.43368196487400001, 0.43549299240099998, 0.46642994880700001, 0.43648004531899998, 0.43979382514999998, 0.43357920646699999, 0.43448400497400003, 0.43302893638599999, 0.44884395599400001, 0.43243002891499999, 0.43497920036299997, 0.43333315849300003, 0.43120193481399999, 0.44183921813999999, 0.43201804161099999, 0.43070816993700001, 0.43093800544700001, 0.43396782875099998, 0.431432962418, 0.43151283264200002, 0.432732105255, 0.43281292915300001, 0.43130493164099998, 0.43159103393600001, 0.442057132721, 0.43084406852700002, 0.44665694236800002, 0.43070578575099999, 0.43876791000400001, 0.42976713180499998, 0.43031406402599998, 0.43287897110000001, 0.43102192878700002, 0.43873286247299997, 0.42987489700300002, 0.43340301513700003, 0.42970800399800002, 0.43067002296399998, 0.43263816833500002, 0.43107914924599999, 0.42996597289999999, 0.429845809937, 0.433667898178, 0.43897414207500002, 0.43040704727200002, 0.43301296234100001], "base_times": [0.34740400314300002, 0.34649682045000002, 0.34654498100300002, 0.34828615188599998, 0.34688496589700002, 0.34619617462199997, 0.348422050476, 0.35294890403700002, 0.346195220947, 0.34620308876, 0.34614801406899998, 0.34765100479099997, 0.34694099426300001, 0.34586310386699998, 0.34622812271100001, 0.34607410430899999, 0.348037004471, 0.34679603576700002, 0.34619593620299999, 0.34622883796699999, 0.34815096855200001, 0.34621715545699999, 0.347053050995, 0.34751987457299999, 0.34610795974699998, 0.34874510765099997, 0.34702110290499999, 0.35155105590800001, 0.34627103805499998, 0.34825491905200001, 0.34611105918899998, 0.34621214866599997, 0.34627509117100003, 0.34635615348799997, 0.34847092628499998, 0.34634494781500003, 0.34610104560900001, 0.34599280357399997, 0.34852099418600002, 0.34607791900599999, 0.34652614593499997, 0.34637212753300001, 0.348030090332, 0.346480131149, 0.34624314308199999, 0.346033096313, 0.346461057663, 0.348530054092, 0.34677815437300002, 0.34670209884600001]}], ["nbody_modified", "RawResult", {"changed_times": [0.178552150726, 0.14297604560900001, 0.135512828827, 0.13514518737799999, 0.13505196571399999, 0.13496088981599999, 0.135854959488, 0.13503122329700001, 0.13507986068700001, 0.13504290580700001, 0.13506603241000001, 0.13503408432, 0.13780403137200001, 0.13503003120400001, 0.13493585586500001, 0.13506293296800001, 0.135177135468, 0.13551998138400001, 0.134822845459, 0.13526797294599999, 0.134998083115, 0.13494396209699999, 0.13502788543700001, 0.13709807395900001, 0.135847091675, 0.135095119476, 0.13512802124000001, 0.13511490821800001, 0.13506793975799999, 0.135057926178, 0.135121107101, 0.135377883911, 0.13559198379500001, 0.13520693779000001, 0.13507509231600001, 0.138010978699, 0.13512301445, 0.13525795936599999, 0.13526701927199999, 0.135272979736, 0.13510107994100001, 0.13522982597399999, 0.13502717018099999, 0.13525414466899999, 0.135074853897, 0.13518810272199999, 0.13801503181499999, 0.13605713844299999, 0.134965896606, 0.13791918754599999], "base_times": [0.60831594467200001, 0.60184097290000005, 0.60403108596800004, 0.604898929596, 0.59930682182299999, 0.59702301025399995, 0.60113716125500005, 0.59830880165099998, 0.60461401939399995, 0.59961390495300004, 0.60151195526099999, 0.60288000106799999, 0.60938596725500005, 0.60743498802200002, 0.600023984909, 0.60061693191499999, 0.60342597961400002, 0.59932994842499998, 0.60085797309900002, 0.60232901573200004, 0.59961009025599998, 0.65427803993199996, 0.60101699829099997, 0.60087990760800003, 0.60391497612, 0.60592889785799997, 0.60827589034999996, 0.60652494430500004, 0.60713696479799995, 0.61633515358000002, 0.60946607589699997, 0.61303901672399996, 0.61002016067499998, 0.61310410499599999, 0.61289215087899995, 0.61635208129899999, 0.62035799026500005, 0.61284279823300003, 0.61309885978699996, 0.61907792091400005, 0.61475586891199996, 0.61362600326500005, 0.61063408851599998, 0.61360001564, 0.61051511764499999, 0.61368680000300002, 0.60794496536300002, 0.61147809028599998, 0.61695909500099999, 0.61682605743399999]}], ["pyflate-fast", "RawResult", {"changed_times": [3.3121449947400001, 3.32391285896, 3.3113570213300001, 3.3125739097600002, 3.3533909320799999, 3.27320981026, 3.3294079303699999, 3.3954648971600001, 3.36747694016, 3.3318870067600002, 3.3314881324800001, 3.29482102394, 3.2987878322599999, 3.3235919475600002, 3.2715950012200001, 3.2692580223099998, 3.3049309253699999, 3.2853808403000002, 3.2795870304100001, 3.2983829975100001, 3.25447106361, 3.3078680038499999, 3.2952280044600002, 3.27434897423, 3.2544469833399998, 3.3341419696800001, 3.2960839271500002, 3.2732009887700002, 3.2589120864900001, 3.3169569969200001, 3.2971358299300002, 3.2613229751600001, 3.3093330860100001, 3.25996899605, 3.3377301692999999, 3.2606930732700001, 3.2845060825300001, 3.2952270507799999, 3.2635328769699998, 3.2907948494000001, 3.2753908634200002, 3.29530191422, 3.2842710018200001, 3.2528989314999999, 3.2978489399000002, 3.3096721172299999, 3.2890150547000001, 3.2512168884300001, 3.3030450344100002, 3.2920761108400001], "base_times": [3.18242001534, 3.1863389015200001, 3.1912939548499999, 3.1865100860600002, 3.1824879646299999, 3.19247794151, 3.1979899406399999, 3.1937181949600002, 3.1884422302200002, 3.20042586327, 3.18729782104, 3.1916460990900002, 3.1865410804700001, 3.1863729953800002, 3.1931619644200002, 3.1834938526199998, 3.2051031589500001, 3.1961328983300001, 3.19024610519, 3.1944649219499999, 3.19651508331, 3.2056679725600001, 3.1969439983400001, 3.1953899860399999, 3.1878740787500002, 3.1908388137800001, 3.1916890144300001, 3.1885077953300001, 3.1937019824999999, 3.1973328590399999, 3.1935818195299999, 3.1918790340399998, 3.18938112259, 3.1975688934300002, 3.1952290534999999, 3.1982200145699999, 3.18593597412, 3.1900849342300002, 3.20125794411, 3.1916630268100001, 3.2036459445999999, 3.20056295395, 3.1956481933599998, 3.1894419193300001, 3.2185740470900002, 3.1986999511700001, 3.1960220336899998, 3.1938240528100001, 3.1854040622699999, 3.1919748783099999]}], ["raytrace-simple", "RawResult", {"changed_times": [2.7798359394099998, 2.7791428566, 2.7640888690900001, 2.76987791061, 2.8119649887099998, 2.7948009967799998, 2.8166298866299999, 2.7788488864900001, 2.8109869957, 2.7679738998399999, 2.8182439804100001, 2.8162288665799999, 2.8095500469200001, 2.7651660442399999, 2.76630306244, 2.81266593933, 2.80415391922, 2.76372790337, 2.7615659236900001, 2.7600309848800002, 2.9178488254500001, 2.76185297966, 2.7615509033199999, 2.7574489116700001, 2.7583611011500002, 2.7616539001499998, 2.8071138858800002, 2.7628870010400002, 2.7632458209999999, 2.9184291362799999, 2.7783088684099999, 2.7595660686499999, 2.7599799633000002, 2.7592060565900001, 2.7621810436200001, 2.8037729263300002, 2.7557120323199999, 2.75506186485, 2.7569091320000001, 2.7592680454299998, 2.7995710372899998, 2.7594900131200002, 2.7773869037600001, 2.7546079158799999, 2.7564780712100001, 2.8319108486200002, 2.75832509995, 2.7571489810899998, 2.7593541145299998, 2.7733960151699999], "base_times": [2.66753315926, 2.6737060546899998, 2.67894601822, 2.6753239631699999, 2.6803069114700002, 2.6736009120899999, 2.6738917827600002, 2.67639303207, 2.6885011196100002, 2.6775891780899999, 2.67288613319, 2.6740000248000002, 2.6716890335099999, 2.6766500472999999, 2.6777601242100002, 2.6722629070299999, 2.67888998985, 2.6715672016099998, 2.6804389953599999, 2.6803979873700001, 2.6752359867100002, 2.67298698425, 2.67721605301, 2.6766111850700001, 2.67558693886, 2.6749620437599999, 2.67335510254, 2.67534399033, 2.6777729988100001, 2.67326784134, 2.67556595802, 2.6889100074800001, 2.6720960140200001, 2.6758561134300001, 2.6755039691900002, 2.6716270446800001, 2.6724560260799999, 2.6727318763699999, 2.6774148941, 2.6764769554100001, 2.6750671863600002, 2.6714379787400002, 2.7435669899000001, 2.7485840320600001, 2.70751786232, 2.7092530727400002, 2.67544198036, 2.6766531467400001, 2.6786830425299999, 2.67903494835]}], ["richards", "RawResult", {"changed_times": [0.42485690116899999, 0.391672134399, 0.38697290420500002, 0.38978099823000001, 0.38653588295000002, 0.38539409637499999, 0.38544106483500001, 0.38723397254899999, 0.38484597206100002, 0.38460087776200003, 0.38481712341300001, 0.387125015259, 0.38511180877700002, 0.38335013389599998, 0.38253307342499998, 0.38541388511699998, 0.40138697624199998, 0.41167879104600003, 0.385834932327, 0.4046189785, 0.38262510299699998, 0.39745593071000002, 0.38132977485699998, 0.38368082046500002, 0.380796909332, 0.37990593910199999, 0.38024783134500001, 0.38323211669899998, 0.379715919495, 0.38007807731600002, 0.37966990470899997, 0.40578198432899998, 0.38037204742399999, 0.38033199310299998, 0.38059902191200001, 0.382125139236, 0.38005089759799998, 0.38041114807100002, 0.37975502014200002, 0.41739606857299999, 0.37962198257399998, 0.37924098968499997, 0.40164279937699998, 0.38241791725199997, 0.37951016426099998, 0.37923693656899998, 0.37933707237199998, 0.38282299041700002, 0.37891888618500003, 0.37944197654700001], "base_times": [0.34984207153300001, 0.35352921485900002, 0.35030293464700002, 0.35039401054399999, 0.35356593132000003, 0.35617804527300001, 0.35045003890999998, 0.35419392585800002, 0.34917712211599999, 0.35496902465800001, 0.34898185729999998, 0.35016894340499999, 0.34672594070399998, 0.35246014595000003, 0.34967398643499997, 0.35236191749599999, 0.34780907630899999, 0.35169386863699997, 0.35305500030499998, 0.35085582733199999, 0.35157895088199997, 0.348843097687, 0.355578899384, 0.35684394836400002, 0.35460305213900001, 0.34866809845000002, 0.34962701797500001, 0.35786795616099998, 0.35232281684900002, 0.34995484352099998, 0.357528924942, 0.35109806060799997, 0.353901147842, 0.34876418113699997, 0.34823799133299999, 0.34872198104899998, 0.35193896293600002, 0.34771203994799998, 0.34916996955899998, 0.34796500205999997, 0.34642791748000001, 0.35339903831500002, 0.34859085083000002, 0.35140800476099998, 0.34648203849800002, 0.355483055115, 0.353637933731, 0.34880208969100002, 0.37337613105799999, 0.35136389732399997]}], ["spambayes", "RawResult", {"changed_times": [0.59816980361899996, 0.78452110290499999, 0.83023285865800001, 0.93411684036300002, 0.69184017181400004, 1.10234284401, 0.74256396293600002, 0.71328186988800002, 0.57437109947200005, 0.86799788474999995, 0.67963004112199998, 0.58452200889600003, 0.58005404472400002, 0.52808690071099995, 1.1698250770600001, 0.69769787788399995, 0.64825582504299994, 0.694402217865, 0.63659095764200002, 0.58709597587600004, 0.52931094169600001, 0.65094089508099995, 0.54797601699800003, 0.52007412910499995, 0.61390900611900001, 0.545217990875, 0.66334605216999998, 0.54549193382299999, 0.67743301391599997, 0.51848697662400001, 0.71530890464800001, 0.76569509506199995, 0.52243494987500005, 0.51943516731299999, 0.61838197708099996, 0.60029411315900005, 0.64244699478099998, 0.53688502311700004, 0.51747083663899995, 0.51864504814099999, 0.64398002624499995, 0.516402006149, 0.54355907440200002, 0.60826396942100003, 0.51326608657799999, 0.509041070938, 0.61415004730199996, 0.592803001404, 0.50892901420600001, 0.60751390457200005], "base_times": [0.29755282402, 0.29553508758500002, 0.29539799690200003, 0.29539704322799998, 0.29553294181799999, 0.29756402969399998, 0.295476913452, 0.29511308670000003, 0.29523301124599999, 0.294858932495, 0.29885101318399998, 0.29543805122400002, 0.29549407959000001, 0.29555702209500001, 0.295747041702, 0.29760813713099998, 0.29595708847000002, 0.29637312889099998, 0.29531884193399999, 0.295172929764, 0.29555201530500003, 0.29735279083299998, 0.29529094696000002, 0.29522705078099998, 0.29555082321199999, 0.29535007476800001, 0.29763197898900001, 0.29533004760699999, 0.29603099823000001, 0.29527783393899998, 0.29541301727300001, 0.29754710197399997, 0.29553294181799999, 0.295480012894, 0.295413970947, 0.29505491256700001, 0.29747200012199998, 0.29603290557900003, 0.29523515701300002, 0.295083999634, 0.29551792144799999, 0.29732418060299998, 0.29565906524699997, 0.29551196098299998, 0.29589390754700001, 0.295162916183, 0.29713106155399999, 0.29647493362400001, 0.295417070389, 0.29561495780899999]}], ["spectral-norm", "RawResult", {"changed_times": [0.46351408958399998, 0.36286616325400001, 0.36123991012599999, 0.35890722274800002, 0.35978198051499999, 0.35527896881100002, 0.35585403442399999, 0.35739207267799999, 0.35975408554100002, 0.35413813590999998, 0.35580110549900001, 0.35268497466999998, 0.364890098572, 0.35489296913099999, 0.35201191902200002, 0.354170084, 0.35294985771199999, 0.35513281822199999, 0.35184288024900001, 0.35136795043899999, 0.35138392448400002, 0.35504508018499997, 0.35184502601599998, 0.35189914703399999, 0.35134291648900001, 0.35165905952499998, 0.35476517677300001, 0.352794885635, 0.35161089897199999, 0.35130691528300001, 0.355042934418, 0.35176992416399999, 0.35218715667700001, 0.353415966034, 0.35497093200699997, 0.35284900665300001, 0.351359128952, 0.35298585891700002, 0.35289096832299999, 0.35544204711900002, 0.35241413116499998, 0.35347700118999997, 0.35199999809299998, 0.35367488861099999, 0.35371303558299999, 0.35233402252200002, 0.35166788101200003, 0.35387802124000001, 0.37293601036099999, 0.35278296470600001], "base_times": [0.50352501869199995, 0.47929906845100001, 0.48261594772299998, 0.47830605506899998, 0.48235297203100003, 0.48249197006200001, 0.48067307472199999, 0.48097801208500002, 0.47920393943799999, 0.47869086265600003, 0.48005580902099998, 0.47897410392799999, 0.48137307167100002, 0.483394145966, 0.47975897788999999, 0.47675085067700002, 0.48147702217100002, 0.47802019119299999, 0.482943058014, 0.47830295562699998, 0.48056387901300002, 0.47623491287199998, 0.47782897949199998, 0.47907996177700002, 0.48068904876700003, 0.47843909263599999, 0.48215293884299998, 0.478461027145, 0.48303794860799998, 0.47899913787800003, 0.478845119476, 0.480139970779, 0.48173093795799998, 0.47723102569600001, 0.47754788398699999, 0.47700595855700001, 0.48208713531500003, 0.47760915756200001, 0.480431079865, 0.48341894149800002, 0.47852706909199999, 0.47913908958399998, 0.48555207252499999, 0.490929841995, 0.47841882705700001, 0.48059010505700001, 0.47776508331299999, 0.47688412666300001, 0.48040413856500003, 0.47939014434799998]}], ["telco", "RawResult", {"changed_times": [1.452091, 1.324082, 1.3400840000000001, 1.4160889999999999, 1.3320829999999999, 1.3800859999999999, 1.280081, 1.304081, 1.304082, 1.300081, 1.308082, 1.2840800000000001, 1.308082, 1.296081, 1.3240829999999999, 1.308081, 1.284081, 1.324082, 1.288081, 1.300081, 1.284081, 1.328082, 1.3200829999999999, 1.288081, 1.300081, 1.2880799999999999, 1.288081, 1.308081, 1.288081, 1.308082, 1.284081, 1.304081, 1.2800800000000001, 1.304081, 1.288081, 1.304081, 1.288081, 1.300081, 1.288081, 1.2920799999999999, 1.312082, 1.284081, 1.308082, 1.2800800000000001, 1.304081, 1.288081, 1.2960799999999999, 1.288081, 1.304081, 1.284081], "base_times": [1.21, 1.21, 1.2, 1.21, 1.21, 1.21, 1.21, 1.2, 1.21, 1.21, 1.21, 1.21, 1.2, 1.21, 1.21, 1.21, 1.2, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.2, 1.21, 1.21, 1.21, 1.2, 1.22, 1.2, 1.2, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.2, 1.21, 1.21, 1.21, 1.2, 1.2, 1.21, 1.21, 1.21]}], ["twisted_names", "RawResult", {"changed_times": [0.022883295194508008, 0.020242914979757085, 0.024330900243309004, 0.02004008016032064, 0.020080321285140562, 0.018832391713747645, 0.017793594306049824, 0.018484288354898338, 0.017953321364452424, 0.017825311942959002, 0.017793594306049824, 0.019801980198019802, 0.017825311942959002, 0.017667844522968199, 0.017761989342806393, 0.017761989342806393, 0.018691588785046728, 0.018214936247723135, 0.017636684303350969, 0.021231422505307854, 0.017699115044247787, 0.018518518518518517, 0.017793594306049824, 0.017699115044247787, 0.018050541516245487, 0.017730496453900711, 0.01937984496124031, 0.01824817518248175, 0.017761989342806393, 0.018181818181818181, 0.017574692442882251, 0.017953321364452424, 0.017574692442882251, 0.017482517482517484, 0.017574692442882251, 0.017574692442882251, 0.017513134851138354, 0.020161290322580645, 0.017543859649122806, 0.017482517482517484, 0.018796992481203006, 0.017482517482517484, 0.017421602787456445, 0.017513134851138354, 0.017574692442882251, 0.017513134851138354, 0.017513134851138354, 0.020161290322580645, 0.017574692442882251, 0.018761726078799251], "base_times": [0.0095693779904306216, 0.0095147478591817315, 0.0095877277085330784, 0.0095602294455066923, 0.0095877277085330784, 0.0096339113680154135, 0.0095693779904306216, 0.0095602294455066923, 0.0095602294455066923, 0.0095785440613026813, 0.0095510983763132766, 0.0095785440613026813, 0.0095969289827255271, 0.0095969289827255271, 0.0096246390760346481, 0.0095602294455066923, 0.0095969289827255271, 0.0095785440613026813, 0.0095785440613026813, 0.0095602294455066923, 0.0096153846153846159, 0.0095602294455066923, 0.0096618357487922701, 0.0095693779904306216, 0.009727626459143969, 0.0095693779904306216, 0.0095602294455066923, 0.0095419847328244278, 0.0095693779904306216, 0.0095785440613026813, 0.0095693779904306216, 0.0095510983763132766, 0.0095969289827255271, 0.0095602294455066923, 0.0095785440613026813, 0.0095877277085330784, 0.0095693779904306216, 0.0095419847328244278, 0.0095785440613026813, 0.0095238095238095247, 0.0095693779904306216, 0.0095693779904306216, 0.0095877277085330784, 0.0095693779904306216, 0.0095877277085330784, 0.0095969289827255271, 0.0095785440613026813, 0.0095693779904306216, 0.0095602294455066923, 0.0095785440613026813]}]], "branch": "trunk", "revision": 0}
\ No newline at end of file
+{"options": "", "results": [["crypto_pyaes", "RawResult", {"changed_times": [2.6510879993400001, 1.9344520568800001, 1.73843193054, 1.7308630943300001, 1.7928988933600001, 1.68379402161, 1.6968870163000001, 1.75850701332, 1.7397270202599999, 1.69991493225, 1.6527018547100001, 1.6478400230400001, 1.6623890399900001, 1.6820631027199999, 1.6487820148500001, 1.65380096436, 1.65068292618, 1.64833211899, 1.66016221046, 1.6446061134300001, 1.64752292633, 1.68245911598, 1.6478810310400001, 1.6496109962500001, 1.6509699821499999, 1.6476500034299999, 1.6489810943600001, 1.6475958824200001, 1.64714002609, 1.6492788791699999, 1.6498639583600001, 1.68475699425, 1.6480910778, 1.64762187004, 1.6511659622199999, 1.64651894569, 1.64819908142, 1.64752602577, 1.64961481094, 1.64846897125, 1.64679908752, 1.68699193001, 1.6472530365, 1.6470811367, 1.6470909118699999, 1.6486299037900001, 1.6494009494799999, 1.64831399918, 1.64873719215, 1.6486461162599999], "base_times": [2.80382990837, 2.7956340313000001, 2.8010652065300001, 2.7976541519200002, 2.7950928211199999, 2.8041520118699998, 2.8043060302699998, 2.7999861240400001, 2.8013310432399998, 2.8088479042099999, 2.8007519245100001, 2.8009748458899999, 2.8043870925899999, 2.80448412895, 2.80184221268, 2.80495095253, 2.8023998737300002, 2.8056631088300001, 2.8033299446100002, 2.7996089458500002, 2.8002779483800002, 2.8084959983800002, 2.8017699718500002, 2.8082520961799999, 2.8050520420099998, 2.8028419017799999, 2.7981400489800001, 2.7999629974400002, 2.7998390197799998, 2.8000919818900001, 2.80037903786, 2.79657483101, 2.7981040477799999, 2.8026959896100001, 2.7995419502300001, 2.7954881191299998, 2.7970201969100001, 2.79839706421, 2.7952239513400001, 2.7976598739599998, 2.7945868968999998, 2.7983119487799999, 2.7953550815599999, 2.79377818108, 2.7956900596600001, 2.8036141395600001, 2.79533600807, 2.7930212020899998, 2.7950999736800002, 2.7983438968700001]}], ["django", "RawResult", {"changed_times": [0.94225811958299999, 0.68628406524700003, 0.70013499259900003, 0.68669486045799999, 0.70185899734500001, 0.68997812271100001, 0.70118880271899997, 0.68901705741899999, 0.70103502273600005, 0.68790817260699999, 0.69985699653599998, 0.68816995620700006, 0.73480010032700005, 0.68603014945999996, 0.70323491096500002, 0.68623781204199996, 0.70352911949200003, 0.68471789360000002, 0.70435285568200001, 0.68524813652000005, 0.70314097404499998, 0.68677711486799997, 0.70417714118999997, 0.68562698364300001, 0.70090913772600005, 0.68850708007799999, 0.70100784301800001, 0.68775796890300001, 0.70195102691699995, 0.68907117843599996, 0.70120120048500001, 0.68785810470599995, 0.70091485977199997, 0.68843507766699996, 0.70012712478600003, 0.68629503250099999, 0.70415806770300005, 0.68661308288599998, 0.70444512367199996, 0.68531179428099998, 0.70345211029099997, 0.68501901626600004, 0.704196929932, 0.68532204628000004, 0.70484805107100001, 0.68551492691000004, 0.70121288299600004, 0.68866515159599995, 0.70078611373900002, 0.68853020667999998], "base_times": [0.98770689964299996, 0.98978590965299995, 0.98754811286900002, 0.98878502845799998, 0.98909711837799996, 0.989701032639, 1.0125169754000001, 0.99142193794300004, 0.98840188980099997, 0.99162793159499996, 0.99071907997099995, 0.99076414108300004, 0.98944306373599999, 0.98936295509299998, 0.99107599258400003, 0.99007892608600001, 0.98786211013799996, 0.992697954178, 0.99010705947900002, 0.98694705963100005, 0.98762106895400004, 0.98965787887599999, 0.98832798004199995, 0.98824381828300001, 0.98842883110000002, 0.98763680458100001, 0.99011301994300005, 0.98688387870799998, 0.98969793319699995, 0.98943901061999995, 0.98809719085699999, 0.98793482780499997, 0.98821902275100004, 0.98756289482100001, 0.99030303955099996, 0.98743104934699999, 0.98614192009000001, 0.98893618583700005, 0.98977398872400002, 0.98548102378799995, 0.987926959991, 0.98703289031999997, 0.98901891708400003, 0.98792290687600004, 0.98637413978599997, 0.98895907402000005, 0.98991489410400002, 0.98569011688200003, 0.98801302909900002, 0.99004602432300004]}], ["fannkuch", "RawResult", {"changed_times": [0.65027499198899996, 0.57184004783599995, 0.56566786766099997, 0.56521892547599994, 0.56701707840000004, 0.57264208793600002, 0.56822896003699996, 0.56470012664799996, 0.56810212135299998, 0.56781196594200001, 0.564800024033, 0.56483197212199998, 0.56917119026200003, 0.56767296791099997, 0.56695199012800002, 0.56578302383400003, 0.56515192985499996, 0.57563495635999995, 0.56515789031999997, 0.56409192085299997, 0.56773018837, 0.56481719016999998, 0.56611680984500001, 0.56627202034000002, 0.56611609458900003, 0.56967306137100004, 0.567966938019, 0.56780695915199997, 0.57448220253000004, 0.57842707634000001, 0.56447005271899997, 0.56811785697899997, 0.56486487388600004, 0.56649589538599998, 0.56501197814899995, 0.56486201286299997, 0.56838393211399996, 0.56405186653100003, 0.56555008888199998, 0.56801414489699997, 0.56562018394500002, 0.56669306755100002, 0.56592893600500005, 0.56481790542599997, 0.56828188896200005, 0.56566095352199997, 0.56510496139499999, 0.56719899177599997, 0.56526398658800003, 0.56593322753899999], "base_times": [1.84706997871, 1.8756129741700001, 1.85868191719, 1.8472619056699999, 1.87546992302, 1.8508591651899999, 1.8696298599200001, 1.86460900307, 1.85237002373, 1.87226009369, 1.8722259998299999, 1.8783550262499999, 1.8736140727999999, 1.87361216545, 1.87663817406, 1.8724210262300001, 1.8727889061, 1.8782889842999999, 1.8723690509799999, 1.8713910579699999, 1.8771908283200001, 1.87365293503, 1.87231016159, 1.87713694572, 1.87258410454, 1.87410783768, 1.8863110542299999, 1.87264585495, 1.87151694298, 1.8771209716799999, 1.8736791610700001, 1.8717048168199999, 1.8770380020099999, 1.8727111816399999, 1.87127304077, 1.87773990631, 1.89574599266, 1.87074995041, 1.92922902107, 1.9290540218400001, 1.94486498833, 1.8772900104500001, 1.87182497978, 1.87466216087, 1.8774259090400001, 1.87289023399, 1.87402200699, 1.8763341903699999, 1.8749959468799999, 1.8724749088299999]}], ["go", "RawResult", {"changed_times": [2.3995778560600001, 2.3866791725200001, 2.3967480659499998, 2.5598640441899998, 2.2185318469999999, 2.3486278057100001, 2.3838458061200001, 2.4448201656299999, 2.19373893738, 2.1973280906700001, 2.1919651031499998, 2.5130779743199998, 2.2208979129799999, 2.2443778514899999, 2.3977251052900002, 2.1551439762100002, 2.1595408916499998, 2.23243093491, 2.17227602005, 2.1977610588099998, 2.3166739940599999, 2.2348148822799998, 2.1878528595, 2.1694250106799999, 2.1508538722999999, 2.3760650157900001, 2.1592259406999998, 2.1955871581999999, 2.1416928768200001, 2.2533910274500002, 2.1912820339199999, 2.2604429721799999, 2.2298648357399999, 2.1796870231600001, 2.20620703697, 2.16007018089, 2.1497859954799998, 2.2840979099299998, 2.2118680477099999, 2.1375119686100001, 2.15009498596, 2.1446928978000002, 2.1316239833799999, 2.13385415077, 2.1266629695899999, 2.3893008232100001, 2.1834890842400001, 2.1347019672399998, 2.21711921692, 2.1232719421400001], "base_times": [0.93096089363099999, 0.95089387893699995, 0.93110585212700003, 0.92558383941699995, 0.93034386634800004, 0.92623090744000003, 0.93461894989000005, 0.93881106376599999, 0.92857980728099998, 0.93526601791399999, 0.92670106887799997, 0.93337702751200002, 0.93183684349100004, 0.93682098388699997, 0.92921209335300003, 0.93008303642300005, 0.92924404144299999, 0.93299007415799995, 0.93573307990999999, 0.930922985077, 0.93020296096800004, 0.93374800682100001, 0.93536710739100004, 0.93496298789999999, 0.93623590469399998, 0.93143200874300003, 0.92915606498699999, 0.92980098724399995, 0.94962000846899997, 0.93416810035700004, 0.93491911888099999, 0.92967605590799995, 0.927985906601, 0.934444904327, 0.93304419517500004, 0.93216395378100003, 0.93552398681600002, 0.94005179405200001, 0.93092107772800003, 0.93850493431100002, 0.932507038116, 0.93334603309599995, 0.94100594520600001, 0.93229484558099995, 0.93091201782199995, 0.93197703361499995, 0.938956975937, 0.93856000900299996, 0.93897986412000001, 0.92620086669900004]}], ["html5lib", "RawResult", {"changed_times": [38.3545889854, 29.4800949097, 28.267448186900001, 27.7905261517, 27.673813104600001, 27.295396089600001, 27.640147924400001, 27.573884964000001, 27.469006061599998, 27.410823106799999, 27.163343191100001, 27.421272039400002, 27.498446941400001, 27.377060890199999, 27.862658977500001, 27.3771958351, 27.126015901599999, 27.002305030799999, 27.370562076599999, 27.4734969139, 27.1270251274, 27.124243021000002, 27.327117919900001, 27.165127992599999, 27.1296830177, 27.4667949677, 27.082341194200001, 27.294297933599999, 27.245062112799999, 27.360240936299999, 27.298542022700001, 27.139825105700002, 27.091902017599999, 27.068879127500001, 27.3232738972, 27.168828010599999, 27.2176198959, 27.3889100552, 27.146837949799998, 27.0338580608, 27.022614002200001, 27.345255851699999, 27.146274805099999, 27.0234370232, 27.357608795200001, 27.049774169900001, 27.077157974199999, 27.123055934900002, 27.078799009299999, 27.409001111999999], "base_times": [13.864197969399999, 14.0840089321, 14.0057208538, 14.0325820446, 14.0150129795, 14.1008889675, 14.097244978000001, 14.067807912799999, 14.0274028778, 14.012732029, 13.961874008200001, 14.1108551025, 14.0863909721, 14.0686590672, 14.004349947, 14.185241937600001, 13.9961509705, 14.1126899719, 14.1245219707, 14.086353063600001, 14.048186063799999, 14.0126638412, 13.965003013600001, 14.136936903, 14.125771045700001, 14.0789718628, 14.0491027832, 13.986720800400001, 13.978865861899999, 14.133366823199999, 14.0888400078, 14.118774890899999, 14.0262331963, 14.035698890700001, 13.984946966200001, 14.1640629768, 14.148418188100001, 14.091625928899999, 14.075366020200001, 14.0536768436, 13.9736099243, 14.1238529682, 14.118108987799999, 14.103012085, 14.082417964899999, 14.0419850349, 14.015744209299999, 13.983751058599999, 14.112045049700001, 14.1095650196]}], ["meteor-contest", "RawResult", {"changed_times": [0.54956293106099996, 0.44127202034000002, 0.43488597869899998, 0.43912196159400002, 0.43368196487400001, 0.43549299240099998, 0.46642994880700001, 0.43648004531899998, 0.43979382514999998, 0.43357920646699999, 0.43448400497400003, 0.43302893638599999, 0.44884395599400001, 0.43243002891499999, 0.43497920036299997, 0.43333315849300003, 0.43120193481399999, 0.44183921813999999, 0.43201804161099999, 0.43070816993700001, 0.43093800544700001, 0.43396782875099998, 0.431432962418, 0.43151283264200002, 0.432732105255, 0.43281292915300001, 0.43130493164099998, 0.43159103393600001, 0.442057132721, 0.43084406852700002, 0.44665694236800002, 0.43070578575099999, 0.43876791000400001, 0.42976713180499998, 0.43031406402599998, 0.43287897110000001, 0.43102192878700002, 0.43873286247299997, 0.42987489700300002, 0.43340301513700003, 0.42970800399800002, 0.43067002296399998, 0.43263816833500002, 0.43107914924599999, 0.42996597289999999, 0.429845809937, 0.433667898178, 0.43897414207500002, 0.43040704727200002, 0.43301296234100001], "base_times": [0.34740400314300002, 0.34649682045000002, 0.34654498100300002, 0.34828615188599998, 0.34688496589700002, 0.34619617462199997, 0.348422050476, 0.35294890403700002, 0.346195220947, 0.34620308876, 0.34614801406899998, 0.34765100479099997, 0.34694099426300001, 0.34586310386699998, 0.34622812271100001, 0.34607410430899999, 0.348037004471, 0.34679603576700002, 0.34619593620299999, 0.34622883796699999, 0.34815096855200001, 0.34621715545699999, 0.347053050995, 0.34751987457299999, 0.34610795974699998, 0.34874510765099997, 0.34702110290499999, 0.35155105590800001, 0.34627103805499998, 0.34825491905200001, 0.34611105918899998, 0.34621214866599997, 0.34627509117100003, 0.34635615348799997, 0.34847092628499998, 0.34634494781500003, 0.34610104560900001, 0.34599280357399997, 0.34852099418600002, 0.34607791900599999, 0.34652614593499997, 0.34637212753300001, 0.348030090332, 0.346480131149, 0.34624314308199999, 0.346033096313, 0.346461057663, 0.348530054092, 0.34677815437300002, 0.34670209884600001]}], ["nbody_modified", "RawResult", {"changed_times": [0.178552150726, 0.14297604560900001, 0.135512828827, 0.13514518737799999, 0.13505196571399999, 0.13496088981599999, 0.135854959488, 0.13503122329700001, 0.13507986068700001, 0.13504290580700001, 0.13506603241000001, 0.13503408432, 0.13780403137200001, 0.13503003120400001, 0.13493585586500001, 0.13506293296800001, 0.135177135468, 0.13551998138400001, 0.134822845459, 0.13526797294599999, 0.134998083115, 0.13494396209699999, 0.13502788543700001, 0.13709807395900001, 0.135847091675, 0.135095119476, 0.13512802124000001, 0.13511490821800001, 0.13506793975799999, 0.135057926178, 0.135121107101, 0.135377883911, 0.13559198379500001, 0.13520693779000001, 0.13507509231600001, 0.138010978699, 0.13512301445, 0.13525795936599999, 0.13526701927199999, 0.135272979736, 0.13510107994100001, 0.13522982597399999, 0.13502717018099999, 0.13525414466899999, 0.135074853897, 0.13518810272199999, 0.13801503181499999, 0.13605713844299999, 0.134965896606, 0.13791918754599999], "base_times": [0.60831594467200001, 0.60184097290000005, 0.60403108596800004, 0.604898929596, 0.59930682182299999, 0.59702301025399995, 0.60113716125500005, 0.59830880165099998, 0.60461401939399995, 0.59961390495300004, 0.60151195526099999, 0.60288000106799999, 0.60938596725500005, 0.60743498802200002, 0.600023984909, 0.60061693191499999, 0.60342597961400002, 0.59932994842499998, 0.60085797309900002, 0.60232901573200004, 0.59961009025599998, 0.65427803993199996, 0.60101699829099997, 0.60087990760800003, 0.60391497612, 0.60592889785799997, 0.60827589034999996, 0.60652494430500004, 0.60713696479799995, 0.61633515358000002, 0.60946607589699997, 0.61303901672399996, 0.61002016067499998, 0.61310410499599999, 0.61289215087899995, 0.61635208129899999, 0.62035799026500005, 0.61284279823300003, 0.61309885978699996, 0.61907792091400005, 0.61475586891199996, 0.61362600326500005, 0.61063408851599998, 0.61360001564, 0.61051511764499999, 0.61368680000300002, 0.60794496536300002, 0.61147809028599998, 0.61695909500099999, 0.61682605743399999]}], ["pyflate-fast", "RawResult", {"changed_times": [3.3121449947400001, 3.32391285896, 3.3113570213300001, 3.3125739097600002, 3.3533909320799999, 3.27320981026, 3.3294079303699999, 3.3954648971600001, 3.36747694016, 3.3318870067600002, 3.3314881324800001, 3.29482102394, 3.2987878322599999, 3.3235919475600002, 3.2715950012200001, 3.2692580223099998, 3.3049309253699999, 3.2853808403000002, 3.2795870304100001, 3.2983829975100001, 3.25447106361, 3.3078680038499999, 3.2952280044600002, 3.27434897423, 3.2544469833399998, 3.3341419696800001, 3.2960839271500002, 3.2732009887700002, 3.2589120864900001, 3.3169569969200001, 3.2971358299300002, 3.2613229751600001, 3.3093330860100001, 3.25996899605, 3.3377301692999999, 3.2606930732700001, 3.2845060825300001, 3.2952270507799999, 3.2635328769699998, 3.2907948494000001, 3.2753908634200002, 3.29530191422, 3.2842710018200001, 3.2528989314999999, 3.2978489399000002, 3.3096721172299999, 3.2890150547000001, 3.2512168884300001, 3.3030450344100002, 3.2920761108400001], "base_times": [3.18242001534, 3.1863389015200001, 3.1912939548499999, 3.1865100860600002, 3.1824879646299999, 3.19247794151, 3.1979899406399999, 3.1937181949600002, 3.1884422302200002, 3.20042586327, 3.18729782104, 3.1916460990900002, 3.1865410804700001, 3.1863729953800002, 3.1931619644200002, 3.1834938526199998, 3.2051031589500001, 3.1961328983300001, 3.19024610519, 3.1944649219499999, 3.19651508331, 3.2056679725600001, 3.1969439983400001, 3.1953899860399999, 3.1878740787500002, 3.1908388137800001, 3.1916890144300001, 3.1885077953300001, 3.1937019824999999, 3.1973328590399999, 3.1935818195299999, 3.1918790340399998, 3.18938112259, 3.1975688934300002, 3.1952290534999999, 3.1982200145699999, 3.18593597412, 3.1900849342300002, 3.20125794411, 3.1916630268100001, 3.2036459445999999, 3.20056295395, 3.1956481933599998, 3.1894419193300001, 3.2185740470900002, 3.1986999511700001, 3.1960220336899998, 3.1938240528100001, 3.1854040622699999, 3.1919748783099999]}], ["raytrace-simple", "RawResult", {"changed_times": [2.7798359394099998, 2.7791428566, 2.7640888690900001, 2.76987791061, 2.8119649887099998, 2.7948009967799998, 2.8166298866299999, 2.7788488864900001, 2.8109869957, 2.7679738998399999, 2.8182439804100001, 2.8162288665799999, 2.8095500469200001, 2.7651660442399999, 2.76630306244, 2.81266593933, 2.80415391922, 2.76372790337, 2.7615659236900001, 2.7600309848800002, 2.9178488254500001, 2.76185297966, 2.7615509033199999, 2.7574489116700001, 2.7583611011500002, 2.7616539001499998, 2.8071138858800002, 2.7628870010400002, 2.7632458209999999, 2.9184291362799999, 2.7783088684099999, 2.7595660686499999, 2.7599799633000002, 2.7592060565900001, 2.7621810436200001, 2.8037729263300002, 2.7557120323199999, 2.75506186485, 2.7569091320000001, 2.7592680454299998, 2.7995710372899998, 2.7594900131200002, 2.7773869037600001, 2.7546079158799999, 2.7564780712100001, 2.8319108486200002, 2.75832509995, 2.7571489810899998, 2.7593541145299998, 2.7733960151699999], "base_times": [2.66753315926, 2.6737060546899998, 2.67894601822, 2.6753239631699999, 2.6803069114700002, 2.6736009120899999, 2.6738917827600002, 2.67639303207, 2.6885011196100002, 2.6775891780899999, 2.67288613319, 2.6740000248000002, 2.6716890335099999, 2.6766500472999999, 2.6777601242100002, 2.6722629070299999, 2.67888998985, 2.6715672016099998, 2.6804389953599999, 2.6803979873700001, 2.6752359867100002, 2.67298698425, 2.67721605301, 2.6766111850700001, 2.67558693886, 2.6749620437599999, 2.67335510254, 2.67534399033, 2.6777729988100001, 2.67326784134, 2.67556595802, 2.6889100074800001, 2.6720960140200001, 2.6758561134300001, 2.6755039691900002, 2.6716270446800001, 2.6724560260799999, 2.6727318763699999, 2.6774148941, 2.6764769554100001, 2.6750671863600002, 2.6714379787400002, 2.7435669899000001, 2.7485840320600001, 2.70751786232, 2.7092530727400002, 2.67544198036, 2.6766531467400001, 2.6786830425299999, 2.67903494835]}], ["richards", "RawResult", {"changed_times": [0.42485690116899999, 0.391672134399, 0.38697290420500002, 0.38978099823000001, 0.38653588295000002, 0.38539409637499999, 0.38544106483500001, 0.38723397254899999, 0.38484597206100002, 0.38460087776200003, 0.38481712341300001, 0.387125015259, 0.38511180877700002, 0.38335013389599998, 0.38253307342499998, 0.38541388511699998, 0.40138697624199998, 0.41167879104600003, 0.385834932327, 0.4046189785, 0.38262510299699998, 0.39745593071000002, 0.38132977485699998, 0.38368082046500002, 0.380796909332, 0.37990593910199999, 0.38024783134500001, 0.38323211669899998, 0.379715919495, 0.38007807731600002, 0.37966990470899997, 0.40578198432899998, 0.38037204742399999, 0.38033199310299998, 0.38059902191200001, 0.382125139236, 0.38005089759799998, 0.38041114807100002, 0.37975502014200002, 0.41739606857299999, 0.37962198257399998, 0.37924098968499997, 0.40164279937699998, 0.38241791725199997, 0.37951016426099998, 0.37923693656899998, 0.37933707237199998, 0.38282299041700002, 0.37891888618500003, 0.37944197654700001], "base_times": [0.34984207153300001, 0.35352921485900002, 0.35030293464700002, 0.35039401054399999, 0.35356593132000003, 0.35617804527300001, 0.35045003890999998, 0.35419392585800002, 0.34917712211599999, 0.35496902465800001, 0.34898185729999998, 0.35016894340499999, 0.34672594070399998, 0.35246014595000003, 0.34967398643499997, 0.35236191749599999, 0.34780907630899999, 0.35169386863699997, 0.35305500030499998, 0.35085582733199999, 0.35157895088199997, 0.348843097687, 0.355578899384, 0.35684394836400002, 0.35460305213900001, 0.34866809845000002, 0.34962701797500001, 0.35786795616099998, 0.35232281684900002, 0.34995484352099998, 0.357528924942, 0.35109806060799997, 0.353901147842, 0.34876418113699997, 0.34823799133299999, 0.34872198104899998, 0.35193896293600002, 0.34771203994799998, 0.34916996955899998, 0.34796500205999997, 0.34642791748000001, 0.35339903831500002, 0.34859085083000002, 0.35140800476099998, 0.34648203849800002, 0.355483055115, 0.353637933731, 0.34880208969100002, 0.37337613105799999, 0.35136389732399997]}], ["spambayes", "RawResult", {"changed_times": [0.59816980361899996, 0.78452110290499999, 0.83023285865800001, 0.93411684036300002, 0.69184017181400004, 1.10234284401, 0.74256396293600002, 0.71328186988800002, 0.57437109947200005, 0.86799788474999995, 0.67963004112199998, 0.58452200889600003, 0.58005404472400002, 0.52808690071099995, 1.1698250770600001, 0.69769787788399995, 0.64825582504299994, 0.694402217865, 0.63659095764200002, 0.58709597587600004, 0.52931094169600001, 0.65094089508099995, 0.54797601699800003, 0.52007412910499995, 0.61390900611900001, 0.545217990875, 0.66334605216999998, 0.54549193382299999, 0.67743301391599997, 0.51848697662400001, 0.71530890464800001, 0.76569509506199995, 0.52243494987500005, 0.51943516731299999, 0.61838197708099996, 0.60029411315900005, 0.64244699478099998, 0.53688502311700004, 0.51747083663899995, 0.51864504814099999, 0.64398002624499995, 0.516402006149, 0.54355907440200002, 0.60826396942100003, 0.51326608657799999, 0.509041070938, 0.61415004730199996, 0.592803001404, 0.50892901420600001, 0.60751390457200005], "base_times": [0.29755282402, 0.29553508758500002, 0.29539799690200003, 0.29539704322799998, 0.29553294181799999, 0.29756402969399998, 0.295476913452, 0.29511308670000003, 0.29523301124599999, 0.294858932495, 0.29885101318399998, 0.29543805122400002, 0.29549407959000001, 0.29555702209500001, 0.295747041702, 0.29760813713099998, 0.29595708847000002, 0.29637312889099998, 0.29531884193399999, 0.295172929764, 0.29555201530500003, 0.29735279083299998, 0.29529094696000002, 0.29522705078099998, 0.29555082321199999, 0.29535007476800001, 0.29763197898900001, 0.29533004760699999, 0.29603099823000001, 0.29527783393899998, 0.29541301727300001, 0.29754710197399997, 0.29553294181799999, 0.295480012894, 0.295413970947, 0.29505491256700001, 0.29747200012199998, 0.29603290557900003, 0.29523515701300002, 0.295083999634, 0.29551792144799999, 0.29732418060299998, 0.29565906524699997, 0.29551196098299998, 0.29589390754700001, 0.295162916183, 0.29713106155399999, 0.29647493362400001, 0.295417070389, 0.29561495780899999]}], ["spectral-norm", "RawResult", {"changed_times": [0.46351408958399998, 0.36286616325400001, 0.36123991012599999, 0.35890722274800002, 0.35978198051499999, 0.35527896881100002, 0.35585403442399999, 0.35739207267799999, 0.35975408554100002, 0.35413813590999998, 0.35580110549900001, 0.35268497466999998, 0.364890098572, 0.35489296913099999, 0.35201191902200002, 0.354170084, 0.35294985771199999, 0.35513281822199999, 0.35184288024900001, 0.35136795043899999, 0.35138392448400002, 0.35504508018499997, 0.35184502601599998, 0.35189914703399999, 0.35134291648900001, 0.35165905952499998, 0.35476517677300001, 0.352794885635, 0.35161089897199999, 0.35130691528300001, 0.355042934418, 0.35176992416399999, 0.35218715667700001, 0.353415966034, 0.35497093200699997, 0.35284900665300001, 0.351359128952, 0.35298585891700002, 0.35289096832299999, 0.35544204711900002, 0.35241413116499998, 0.35347700118999997, 0.35199999809299998, 0.35367488861099999, 0.35371303558299999, 0.35233402252200002, 0.35166788101200003, 0.35387802124000001, 0.37293601036099999, 0.35278296470600001], "base_times": [0.50352501869199995, 0.47929906845100001, 0.48261594772299998, 0.47830605506899998, 0.48235297203100003, 0.48249197006200001, 0.48067307472199999, 0.48097801208500002, 0.47920393943799999, 0.47869086265600003, 0.48005580902099998, 0.47897410392799999, 0.48137307167100002, 0.483394145966, 0.47975897788999999, 0.47675085067700002, 0.48147702217100002, 0.47802019119299999, 0.482943058014, 0.47830295562699998, 0.48056387901300002, 0.47623491287199998, 0.47782897949199998, 0.47907996177700002, 0.48068904876700003, 0.47843909263599999, 0.48215293884299998, 0.478461027145, 0.48303794860799998, 0.47899913787800003, 0.478845119476, 0.480139970779, 0.48173093795799998, 0.47723102569600001, 0.47754788398699999, 0.47700595855700001, 0.48208713531500003, 0.47760915756200001, 0.480431079865, 0.48341894149800002, 0.47852706909199999, 0.47913908958399998, 0.48555207252499999, 0.490929841995, 0.47841882705700001, 0.48059010505700001, 0.47776508331299999, 0.47688412666300001, 0.48040413856500003, 0.47939014434799998]}], ["telco", "RawResult", {"changed_times": [1.452091, 1.324082, 1.3400840000000001, 1.4160889999999999, 1.3320829999999999, 1.3800859999999999, 1.280081, 1.304081, 1.304082, 1.300081, 1.308082, 1.2840800000000001, 1.308082, 1.296081, 1.3240829999999999, 1.308081, 1.284081, 1.324082, 1.288081, 1.300081, 1.284081, 1.328082, 1.3200829999999999, 1.288081, 1.300081, 1.2880799999999999, 1.288081, 1.308081, 1.288081, 1.308082, 1.284081, 1.304081, 1.2800800000000001, 1.304081, 1.288081, 1.304081, 1.288081, 1.300081, 1.288081, 1.2920799999999999, 1.312082, 1.284081, 1.308082, 1.2800800000000001, 1.304081, 1.288081, 1.2960799999999999, 1.288081, 1.304081, 1.284081], "base_times": [1.21, 1.21, 1.2, 1.21, 1.21, 1.21, 1.21, 1.2, 1.21, 1.21, 1.21, 1.21, 1.2, 1.21, 1.21, 1.21, 1.2, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.2, 1.21, 1.21, 1.21, 1.2, 1.22, 1.2, 1.2, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.2, 1.21, 1.21, 1.21, 1.2, 1.2, 1.21, 1.21, 1.21]}], ["twisted_names", "RawResult", {"changed_times": [0.022883295194508008, 0.020242914979757085, 0.024330900243309004, 0.02004008016032064, 0.020080321285140562, 0.018832391713747645, 0.017793594306049824, 0.018484288354898338, 0.017953321364452424, 0.017825311942959002, 0.017793594306049824, 0.019801980198019802, 0.017825311942959002, 0.017667844522968199, 0.017761989342806393, 0.017761989342806393, 0.018691588785046728, 0.018214936247723135, 0.017636684303350969, 0.021231422505307854, 0.017699115044247787, 0.018518518518518517, 0.017793594306049824, 0.017699115044247787, 0.018050541516245487, 0.017730496453900711, 0.01937984496124031, 0.01824817518248175, 0.017761989342806393, 0.018181818181818181, 0.017574692442882251, 0.017953321364452424, 0.017574692442882251, 0.017482517482517484, 0.017574692442882251, 0.017574692442882251, 0.017513134851138354, 0.020161290322580645, 0.017543859649122806, 0.017482517482517484, 0.018796992481203006, 0.017482517482517484, 0.017421602787456445, 0.017513134851138354, 0.017574692442882251, 0.017513134851138354, 0.017513134851138354, 0.020161290322580645, 0.017574692442882251, 0.018761726078799251], "base_times": [0.0095693779904306216, 0.0095147478591817315, 0.0095877277085330784, 0.0095602294455066923, 0.0095877277085330784, 0.0096339113680154135, 0.0095693779904306216, 0.0095602294455066923, 0.0095602294455066923, 0.0095785440613026813, 0.0095510983763132766, 0.0095785440613026813, 0.0095969289827255271, 0.0095969289827255271, 0.0096246390760346481, 0.0095602294455066923, 0.0095969289827255271, 0.0095785440613026813, 0.0095785440613026813, 0.0095602294455066923, 0.0096153846153846159, 0.0095602294455066923, 0.0096618357487922701, 0.0095693779904306216, 0.009727626459143969, 0.0095693779904306216, 0.0095602294455066923, 0.0095419847328244278, 0.0095693779904306216, 0.0095785440613026813, 0.0095693779904306216, 0.0095510983763132766, 0.0095969289827255271, 0.0095602294455066923, 0.0095785440613026813, 0.0095877277085330784, 0.0095693779904306216, 0.0095419847328244278, 0.0095785440613026813, 0.0095238095238095247, 0.0095693779904306216, 0.0095693779904306216, 0.0095877277085330784, 0.0095693779904306216, 0.0095877277085330784, 0.0095969289827255271, 0.0095785440613026813, 0.0095693779904306216, 0.0095602294455066923, 0.0095785440613026813]}]], "branch": "trunk", "revision": 0}
Modified: pypy/extradoc/talk/pepm2011/benchmarks/jit-nopspec.json
==============================================================================
--- pypy/extradoc/talk/pepm2011/benchmarks/jit-nopspec.json (original)
+++ pypy/extradoc/talk/pepm2011/benchmarks/jit-nopspec.json Wed Oct 13 22:14:28 2010
@@ -1 +1 @@
-{"options": "", "results": [["crypto_pyaes", "RawResult", {"changed_times": [19.594973802599998, 17.836122035999999, 17.7611591816, 17.7673540115, 17.797179937399999, 17.701630115499999, 17.701794147499999, 17.726331234, 17.7549710274, 17.7046818733, 17.715419054000002, 17.7220032215, 17.708485126500001, 17.698208808899999, 17.943202018699999, 17.7065188885, 17.7018699646, 17.723783969900001, 17.703621864300001, 17.702537059800001, 17.698129892299999, 17.717091083500002, 17.693710088700001, 17.698023796099999, 17.790216922799999, 17.7049281597, 17.7118620872, 17.7670118809, 17.7513260841, 17.712387800199998, 17.703951120399999, 17.7405331135, 17.704728841800002, 17.7038481236, 17.719988107700001, 17.7051708698, 17.701725959800001, 17.709410905799999, 17.716867923700001, 17.694391012200001, 17.700695037799999, 17.712723970399999, 17.692533969900001, 17.705747842800001, 17.712710142100001, 17.758448123899999, 17.701386928600002, 17.700757980300001, 18.095870017999999, 17.7208271027], "base_times": [2.76247382164, 2.75780797005, 2.7526109218600001, 2.7581448554999999, 2.7579169273400002, 2.7591660022700002, 2.7544469833399998, 2.75570702553, 2.7566180229200001, 2.7799978256200002, 2.75436115265, 2.7584359645799998, 2.7715408801999999, 2.7583799362199999, 2.75467610359, 2.7834770679499998, 2.7558917999300001, 2.7597181797000001, 2.75486803055, 2.7581379413599998, 2.7532849311800001, 2.7636089324999999, 2.75440192223, 2.7553980350499998, 2.7562680244400002, 2.7588288784000001, 2.75431799889, 2.7562849521600001, 2.7566390037500001, 2.75684499741, 2.7588860988600001, 2.7639617919899999, 2.758425951, 2.7611510753599999, 2.75679397583, 2.7544021606400002, 2.7587559223200002, 2.7592377662700001, 2.7555589675899999, 2.7590520381900001, 2.75563788414, 2.76086902618, 2.7588081359899999, 2.7565140724199999, 2.75627803802, 2.76230502129, 2.7579100132000001, 2.75721096992, 2.7558379173300001, 2.7606220245399999]}], ["django", "RawResult", {"changed_times": [0.30002903938300002, 0.25580286979700001, 0.261374950409, 0.26095104217499998, 0.25220704078700001, 0.26073884963999999, 0.26132106780999997, 0.254949092865, 0.26201701164199998, 0.26217198371900002, 0.25196504592899999, 0.26127791404700001, 0.26208996772799997, 0.254974842072, 0.261093854904, 0.261404037476, 0.25236296653700002, 0.26157402992200002, 0.26323699951200003, 0.25301718711900001, 0.26132202148400002, 0.26150584220899997, 0.25210714340200002, 0.26118707656899998, 0.26585507392899999, 0.25383305549599999, 0.26143312454200002, 0.26261687278700002, 0.267751932144, 0.26117897033699999, 0.264419078827, 0.252134799957, 0.26171708107000002, 0.26141405105600002, 0.25201201438900001, 0.261445045471, 0.26490807533299998, 0.25207686424300002, 0.26115393638599999, 0.26161694526700002, 0.25278902053800001, 0.261168003082, 0.26426982879599997, 0.25209307670600001, 0.261046171188, 0.26155900955200001, 0.25206208229100002, 0.261226177216, 0.26494908332799999, 0.252925872803], "base_times": [0.995049953461, 0.99419713020300005, 0.99113988876299997, 0.99470710754400005, 0.99212503433200006, 0.99265909194900004, 0.99358582496600001, 0.99400091171299998, 0.99305796623200004, 0.99362897872900002, 0.99429702758799998, 0.99161601066600003, 0.99425101280200001, 0.99299907684300004, 0.99139285087600004, 0.99515914917000003, 0.99177312850999999, 0.99377703666700001, 0.99326705932600001, 0.99304509162900001, 0.99348807334900002, 0.99273705482499996, 0.99258708953899999, 0.99477910995499996, 0.99403500556900004, 0.99051785469099995, 0.994111061096, 0.99377894401599998, 0.99141192436199999, 0.99447512626599999, 0.99370288848900001, 0.99225115776100004, 0.99282193183900003, 0.99233508110000002, 0.99293303489700002, 0.99670696258500002, 0.99345684051500005, 0.99477696418799999, 0.99382996559100001, 0.99122619628899999, 0.99395608901999999, 0.99324393272400002, 0.99102210998499995, 0.99499702453600003, 0.99375295639000005, 0.99156498909000002, 0.99277806282000003, 0.99424910545300005, 0.99084305763199998, 0.993386983871]}], ["fannkuch", "RawResult", {"changed_times": [0.52755904197699999, 0.47701597213699998, 0.46741414070100001, 0.47383809089700002, 0.46101403236400001, 0.46737408638, 0.47243213653600002, 0.47586488723800002, 0.46479701995799999, 0.46248412132299999, 0.46604609489400001, 0.46004605293299999, 0.46376204490700002, 0.46708703041100003, 0.47270798683199999, 0.472360134125, 0.47210407257100001, 0.47691607475300002, 0.47826886177099998, 0.47204399108900003, 0.47201299667399998, 0.46304917335500001, 0.46217513084400003, 0.46784305572500001, 0.460357189178, 0.46357607841499998, 0.46536898613, 0.46894693374599999, 0.46510815620399998, 0.46110606193499998, 0.46555995941200001, 0.46238088607799999, 0.46605896949800002, 0.46549701690700002, 0.45882105827300002, 0.45827507972699999, 0.473033905029, 0.47556495666499998, 0.460822105408, 0.47130393981899998, 0.476540803909, 0.46522593498199999, 0.46266603469799999, 0.47138500213599999, 0.46661305427600003, 0.46587181091300001, 0.47512102127099998, 0.47284221649199998, 0.47128200530999997, 0.47180795669600001], "base_times": [1.9883148670199999, 1.9792110919999999, 1.9909551143599999, 1.98928499222, 1.97898578644, 1.98664617538, 1.9864010810899999, 1.98417901993, 1.9872028827699999, 1.98194694519, 1.9878189563799999, 1.98948693275, 1.9811220169099999, 1.9890899658200001, 1.9894771575900001, 1.9801352024100001, 1.9894909858700001, 1.9891271591199999, 1.98107600212, 1.99568986893, 1.9895131587999999, 1.9803609847999999, 2.0069239139600001, 1.9916350841499999, 1.98022890091, 1.98729896545, 1.99073004723, 1.98082304001, 1.98834180832, 1.9909420013400001, 1.9793639183, 1.98839306831, 1.9902248382600001, 1.9822490215299999, 1.98810195923, 1.99047279358, 1.98221516609, 1.98903012276, 1.9900078773500001, 1.9820079803499999, 1.9892179966000001, 1.9895989894899999, 1.9800350665999999, 1.9895751476300001, 1.9891831874799999, 1.9804949760399999, 1.9896249771100001, 1.9898300170900001, 1.9807391166699999, 1.9895370006599999]}], ["go", "RawResult", {"changed_times": [0.46677207946799998, 0.51466798782300005, 0.388464927673, 0.50362396240200002, 0.36921596527099998, 0.41929197311400002, 0.37606406211900001, 0.428333997726, 0.36696696281399999, 0.33521819114700002, 0.35271787643399999, 0.47167301177999998, 0.36826205253599997, 0.33157110214199997, 0.39287805557299998, 0.30100107192999997, 0.31953597068799999, 0.34765505790700002, 0.31619405746500001, 0.32286810874900002, 0.37090992927600003, 0.328006982803, 0.31113696098299998, 0.355150938034, 0.30218696594200001, 0.31915521621699999, 0.31008601188700002, 0.34765815734900002, 0.30721187591600002, 0.32940602302600003, 0.30662703514099998, 0.33826303482100001, 0.29518413543700001, 0.32855200767499998, 0.37890100479099997, 0.295953989029, 0.28935694694500003, 0.32658910751300002, 0.31018304824800003, 0.32643103599500001, 0.28621220588700003, 0.29142880439800001, 0.28649687766999998, 0.301717042923, 0.288222074509, 0.353245973587, 0.33032798767100002, 0.285399913788, 0.29600501060500001, 0.28842616081200001], "base_times": [0.94673204422000001, 0.94645810127300001, 0.94507908821099995, 0.944802045822, 0.941561937332, 0.944491147995, 0.94291400909400003, 0.95224094390900005, 0.94410514831500003, 0.943855047226, 0.94802284240699997, 0.94546699523900002, 0.94773101806600002, 0.94944882392899999, 0.94337701797499995, 0.94636011123700003, 0.94657802581799999, 0.94187402725199998, 0.94828701019299999, 0.94231510162400001, 0.94628596305799995, 0.94881415367099997, 0.94279098510699999, 0.94889903068500003, 0.95280098915099998, 0.94382286071800003, 0.94861888885500001, 0.94221186637900001, 0.94600486755400004, 0.94160294532800004, 0.94753003120400003, 0.94430899620099995, 0.94843101501500005, 0.96007490158099995, 0.94707822799700003, 0.94710588455199995, 0.95057606697100006, 0.94280695915199997, 0.94800686836199999, 0.94509601593000003, 0.96038508415199997, 0.94446587562600004, 0.94845819473299997, 0.94734406471300003, 0.94510412216200002, 0.94387602806100002, 0.94524908065799995, 0.94992303848299997, 0.94564318656900004, 0.94311594963099998]}], ["html5lib", "RawResult", {"changed_times": [15.457784891099999, 11.3277301788, 10.600764036199999, 10.4216821194, 10.2325599194, 10.2733259201, 10.3756709099, 10.156955957399999, 10.101744890200001, 10.133589983, 10.206784963600001, 10.1694169044, 10.0392251015, 10.155544042600001, 10.1885080338, 10.058653831499999, 10.2810080051, 10.2360329628, 10.039163112600001, 10.1998898983, 10.119593143499999, 10.1641438007, 10.0413920879, 10.107867002500001, 10.135586977000001, 10.1675360203, 10.1493079662, 10.060842990899999, 10.178154945399999, 10.1074330807, 10.170939922300001, 10.047384023699999, 10.0127818584, 10.050449132900001, 10.161669015899999, 10.037492990500001, 10.0900728703, 10.190579891200001, 9.9983761310600006, 10.1199560165, 10.003987073899999, 10.119729042099999, 10.1512379646, 10.054825067499999, 9.9258110523200003, 10.0976631641, 10.0940468311, 10.122640132900001, 9.9850409030900007, 10.099215984300001], "base_times": [13.800020933200001, 13.988759994500001, 13.945523977300001, 13.9515860081, 13.8968560696, 14.0439591408, 14.0257151127, 13.9940800667, 13.977200985, 13.965268135100001, 13.8857018948, 14.0525372028, 14.022065877899999, 14.0086979866, 13.961042880999999, 13.921565771099999, 13.924687862400001, 14.0379269123, 14.256644964199999, 13.9928328991, 14.0013000965, 13.9456880093, 13.914921999000001, 14.061727047, 14.0501580238, 14.0101730824, 13.9787597656, 13.921334981899999, 13.889928102500001, 14.0468928814, 14.0285110474, 14.0453250408, 13.9559979439, 13.9637241364, 13.9317610264, 14.0579519272, 14.0324690342, 14.0096049309, 13.9675040245, 13.964776992799999, 13.924336910199999, 14.0519120693, 14.0543570518, 14.0083789825, 13.9841370583, 13.954766988799999, 13.9033069611, 13.8949439526, 14.038803100599999, 14.0201179981]}], ["meteor-contest", "RawResult", {"changed_times": [0.47674798965499998, 0.403456926346, 0.39809894561800002, 0.40778803825400001, 0.39824509620699999, 0.39766597747799998, 0.40741491317700002, 0.39641904830899999, 0.39668798446699999, 0.39379596710199999, 0.39647078514099998, 0.393444061279, 0.40611696243299999, 0.39961814880399998, 0.39285993576, 0.39224314689599998, 0.39388489723199999, 0.39147710800199997, 0.39165306091300001, 0.39033508300800002, 0.39437389373800003, 0.39175200462300003, 0.391242027283, 0.39103412628200002, 0.39511799812300002, 0.398463010788, 0.39204406738300002, 0.39300107955899999, 0.39334487914999999, 0.391426801682, 0.39687800407399998, 0.39431095123299997, 0.39070391654999997, 0.39106798172000001, 0.39058089256299999, 0.39391088485699999, 0.39834094047500002, 0.39057397842399999, 0.39198994636500001, 0.39368200302099998, 0.39073801040599998, 0.39122891426099998, 0.39064598083500002, 0.39390277862500001, 0.390840053558, 0.39128398895299998, 0.39098405838, 0.39335107803300001, 0.39786505699199998, 0.39080500602700002], "base_times": [0.348359107971, 0.34692883491499998, 0.346405982971, 0.34667205810500001, 0.34701514244100001, 0.34753990173299998, 0.346569061279, 0.34635281562800002, 0.346190929413, 0.34796619415300001, 0.34590101242100002, 0.34643101692200001, 0.34670686721799998, 0.34775090217600002, 0.34620094299300003, 0.34622883796699999, 0.34623694419899997, 0.34779596328700002, 0.348699092865, 0.34642219543500002, 0.347030878067, 0.34581995010400002, 0.347792863846, 0.34766483306899998, 0.34653806686400002, 0.34683513641399999, 0.34936118125900001, 0.34839010238599999, 0.346399068832, 0.34670805931100002, 0.34576678276099998, 0.348170042038, 0.34661507606500003, 0.34650683402999999, 0.34621381759600001, 0.34878396987900001, 0.34638595581100001, 0.34623980522199999, 0.34593510627700003, 0.34620404243500003, 0.34790611267100002, 0.346742153168, 0.34607505798299998, 0.34633517265300001, 0.34812307357799999, 0.34604907035799998, 0.34677386283900002, 0.34667706489599998, 0.34659886360199998, 0.34881806373599999]}], ["nbody_modified", "RawResult", {"changed_times": [0.12817502021800001, 0.114922046661, 0.105006933212, 0.10339498519900001, 0.104254007339, 0.103782892227, 0.103751897812, 0.105894088745, 0.104180812836, 0.103867053986, 0.103599071503, 0.103564023972, 0.103793859482, 0.103470087051, 0.103558063507, 0.103418111801, 0.10340905189500001, 0.10347700119, 0.103801012039, 0.10350394249, 0.103738069534, 0.10368895530699999, 0.106331110001, 0.103569984436, 0.10441899299600001, 0.103644132614, 0.103709936142, 0.10339307785, 0.103735923767, 0.10362410545300001, 0.103760004044, 0.103440999985, 0.10381603240999999, 0.103622913361, 0.10393595695500001, 0.103415966034, 0.103775978088, 0.106086969376, 0.103862047195, 0.103460073471, 0.103792905807, 0.10356497764600001, 0.103854894638, 0.103960990906, 0.103703975677, 0.103446006775, 0.10406708717300001, 0.103525876999, 0.103811979294, 0.10510087013199999], "base_times": [0.64283800125099999, 0.64096808433499997, 0.64577603340099998, 0.64423298835800002, 0.64046311378499998, 0.63754677772500001, 0.63747310638400001, 0.63725614547700005, 0.63658809661899995, 0.64228487014799995, 0.64188194274900001, 0.64234304428099998, 0.63921594619800004, 0.63971495628399999, 0.64251685142500004, 0.64014887809800003, 0.64212799072299998, 0.64027500152600003, 0.63313889503499998, 0.63153290748599999, 0.63076686859099995, 0.63370203971899997, 0.63236403465299995, 0.63347291946399997, 0.63246107101399995, 0.63299584388700003, 0.63348007202099998, 0.63026380538899995, 0.632724046707, 0.63243818283099995, 0.63531398773199999, 0.63819217681899998, 0.63529610633900002, 0.63803815841699996, 0.63671016693100002, 0.64081311225899995, 0.63594484329199996, 0.64017796516400005, 0.64193677902199997, 0.64040708541900004, 0.64205718040500004, 0.64145493507399998, 0.640824079514, 0.65274095535300003, 0.64189887046799998, 0.64417505264300001, 0.64191007614100004, 0.640285015106, 0.63908886909499996, 0.64495897293100002]}], ["pyflate-fast", "RawResult", {"changed_times": [1.83701682091, 1.8073658943199999, 1.8406269550300001, 1.82018709183, 1.8337500095399999, 1.8074929714200001, 1.84810090065, 1.8087329864499999, 1.8338890075700001, 1.81683397293, 1.78763699532, 1.79593896866, 1.81039500237, 1.7986569404599999, 1.8007159233100001, 1.8014991283399999, 1.7972960472099999, 1.80028700829, 1.8007249832200001, 1.85811805725, 1.7931230068199999, 1.82364487648, 1.7935469150500001, 1.82844901085, 1.7920200824700001, 1.82318806648, 1.7929878234900001, 1.82861304283, 1.79402518272, 1.82314896584, 1.79180502892, 1.8225588798500001, 1.80868387222, 1.80998492241, 1.8257219791399999, 1.8084599971799999, 1.81027412415, 1.80907607079, 1.80926918983, 1.82390213013, 1.8229339122799999, 1.9030389785799999, 1.9522318839999999, 1.8275589942899999, 1.81771707535, 1.8227140903500001, 1.8171408176399999, 1.8353171348599999, 1.8287210464500001, 1.83105301857], "base_times": [3.1601369381, 3.17563796043, 3.1705720424699999, 3.17048382759, 3.1654319763199998, 3.1680698394800002, 3.1681051254299999, 3.1718590259599999, 3.1830987930300001, 3.1775770187400001, 3.1700208187099999, 3.1612432002999999, 3.15727186203, 3.1732730865500001, 3.1751198768600002, 3.1624131202700001, 3.1771841049199998, 3.1661839485200001, 3.16439390182, 3.1774570941900002, 3.1796469688400002, 3.16518807411, 3.1729710102099999, 3.1790311336500001, 3.17372894287, 3.1641249656700001, 3.1653139591200001, 3.16444206238, 3.1757740974400002, 3.16623592377, 3.1762218475299999, 3.1651339530899998, 3.1656432151799998, 3.1724681854200001, 3.1735718250299998, 3.1667320728299999, 3.1666581630700001, 3.1662278175399998, 3.1711161136600001, 3.1634638309500001, 3.1635270118699998, 3.1630849838300001, 3.1776990890499999, 3.17032909393, 3.1747660636899999, 3.1702299117999999, 3.17133522034, 3.1701278686499998, 3.1603660583500002, 3.1653149128]}], ["raytrace-simple", "RawResult", {"changed_times": [0.659919023514, 0.66626596450800002, 0.65816497802700002, 0.65398693084699999, 0.63991713523899996, 0.66545104980500003, 0.65175795555100002, 0.63982892036399996, 0.66220998764000005, 0.65392279624899996, 0.63932204246500002, 0.65032505989099998, 0.65231609344499997, 0.63614416122399997, 0.65278100967399999, 0.63543891906700001, 0.65301513671900002, 0.647172212601, 0.63230013847400002, 0.64931488037100005, 0.689571857452, 0.63732600212099999, 0.67306900024399996, 0.63961911201499999, 0.650694131851, 0.63388586044299999, 0.65063095092800005, 0.64781498909000002, 0.63850116729700002, 0.65920114517200001, 0.68630504608200005, 0.63646793365499998, 0.64802885055500004, 0.64920401573200004, 0.63225889205899999, 0.65038490295399998, 0.63108992576599998, 0.64513182640099997, 0.63317394256600001, 0.64602303504900005, 0.64737510681199995, 0.63001990318300005, 0.64657092094400004, 0.63210487365699997, 0.645840883255, 0.65991592407199995, 0.62948513030999997, 0.65090513229400004, 0.63588213920600001, 0.66070103645300005], "base_times": [2.6739389896399999, 2.6792252063799999, 2.67714691162, 2.6755440235100001, 2.6839830875400001, 2.67896699905, 2.6759178638500001, 2.6771140098599999, 2.6831061840100001, 2.6781611442600002, 2.6843791008000002, 2.67555809021, 2.67742395401, 2.68145084381, 2.6856698989900001, 2.6754090786, 2.6820809841200002, 2.7008218765300001, 2.6860649585699998, 2.68976688385, 2.7091708183300001, 2.6831979751600001, 2.6825587749499999, 2.7229101657900001, 2.7227280139899999, 2.7074718475299999, 2.7032160758999999, 2.69513916969, 2.7023360729200001, 2.7511570453599998, 2.6899399757400002, 2.6838908195500002, 2.7002210617100002, 2.8071360588099998, 2.6863989830000001, 2.6864531040199999, 2.69359779358, 2.7453310489699998, 2.7076761722599998, 3.06150817871, 2.7332241535200001, 2.7483820915199999, 2.7639210224199999, 3.4124541282699998, 2.7048859596299999, 2.6892578601800001, 2.6912610530899999, 2.6821250915500001, 2.68154597282, 2.6890358924900002]}], ["richards", "RawResult", {"changed_times": [0.12052297592199999, 0.093158006668099994, 0.091630935668900002, 0.091804027557399998, 0.091670036315899997, 0.091012954711900004, 0.091180086135899993, 0.102787017822, 0.090846061706500003, 0.090148925781199998, 0.093178987503099997, 0.090260028839100004, 0.090500116348300005, 0.098414897918700006, 0.090541839599599994, 0.090975999832200005, 0.090515136718800002, 0.088860034942600002, 0.089382886886599996, 0.087617158889799998, 0.087574005127000004, 0.088225841522200005, 0.087996006012, 0.087512969970700002, 0.0879878997803, 0.096507072448699993, 0.087088108062699995, 0.090749025344800005, 0.087268829345700002, 0.087903976440400003, 0.087549924850500005, 0.087819099426300007, 0.087373971939100001, 0.087810039520300004, 0.087095975875899997, 0.0871419906616, 0.088514089584400005, 0.087375879287700006, 0.086944818496699994, 0.087454080581699997, 0.087461948394799996, 0.097897052764900003, 0.087609052658099998, 0.087203979492200004, 0.089391946792599999, 0.088086128234900002, 0.086940050125100005, 0.087315082550000006, 0.086781978607199994, 0.086982011794999994], "base_times": [0.35359501838700003, 0.35667300224300003, 0.347419977188, 0.35246205329899999, 0.35532999038700003, 0.35530591011000001, 0.35107088088999999, 0.35549902915999998, 0.35110902786300002, 0.35400104522699999, 0.35261011123699998, 0.35541200637800002, 0.35351705551099999, 0.35617089271500002, 0.34958410263099998, 0.35414409637499999, 0.35661983489999999, 0.35693001747100001, 0.35174703598000001, 0.34876704215999998, 0.35491299629200002, 0.35479497909500002, 0.35416102409400002, 0.355372905731, 0.35764288902300001, 0.35487508773799997, 0.35283422470100001, 0.35428977012599999, 0.35132908821100001, 0.35751414299000001, 0.362582206726, 0.35600781440700002, 0.35644507408100001, 0.35086202621500001, 0.35400605201699997, 0.34989190101599998, 0.35055398940999999, 0.35048794746400003, 0.35601019859299998, 0.35249996185299998, 0.35182499885599999, 0.35495114326499999, 0.35285496711699998, 0.355787038803, 0.35039997100800002, 0.35504388809199999, 0.35427594184900002, 0.35611510276800001, 0.35387182235699999, 0.34967398643499997]}], ["spambayes", "RawResult", {"changed_times": [0.48457980155899999, 0.56999397277800001, 0.53518199920700005, 0.54824614524799997, 0.49303603172299998, 0.58462905883799998, 0.51067399978600003, 0.49086999893200001, 0.48435497283899998, 0.42737793922400003, 0.494954109192, 0.43986797332799998, 0.40522003173799997, 0.40721011161800003, 0.57795190811200003, 0.48255205154399999, 0.462779998779, 0.44051218032799999, 0.39301896095299999, 0.39432501792899999, 0.39888882637, 0.43233299255399998, 0.42077803611800002, 0.39932012558000002, 0.40041995048500001, 0.409831047058, 0.44888401031500003, 0.40377283096299998, 0.39395999908399998, 0.40184497833299998, 0.41734480857799999, 0.49758601188700002, 0.39052391052199997, 0.392160177231, 0.39913892746000001, 0.42745113372799998, 0.36878705024699998, 0.38831090927099998, 0.37554693222000002, 0.40881800651599998, 0.37994885444600002, 0.37922286987300002, 0.39221501350400001, 0.39279913902300001, 0.34234404563900001, 0.37785792350800002, 0.38006091117899998, 0.37980985641499998, 0.375820875168, 0.345314979553], "base_times": [0.29905700683600001, 0.298835992813, 0.29825806617700001, 0.29831504821799998, 0.30105614662199998, 0.298439979553, 0.29832100868200001, 0.29835200309799997, 0.29867696762099999, 0.29994201660199998, 0.29847502708399998, 0.29867196083100001, 0.29891991615300001, 0.29807305336000001, 0.30059790611300002, 0.298420906067, 0.29828691482500003, 0.29897713661199998, 0.29811811447100001, 0.300410985947, 0.29847979545600001, 0.2982878685, 0.29861903190599998, 0.29877710342399999, 0.30105996131899998, 0.29816794395399998, 0.29835987091100002, 0.29860401153600002, 0.2985060215, 0.30007410049400002, 0.29955601692200001, 0.299430847168, 0.29854393005399998, 0.29915118217499997, 0.29966092109699999, 0.29980802536000001, 0.29794311523400002, 0.29930615425099999, 0.298702001572, 0.29842782020600001, 0.30247402191200001, 0.29896807670600001, 0.298580169678, 0.29869008064300001, 0.299312114716, 0.30094408988999999, 0.29845499992399999, 0.29868721962, 0.29998612403899999, 0.29916310310400002]}], ["spectral-norm", "RawResult", {"changed_times": [0.18530607223500001, 0.118788957596, 0.11839389801, 0.117673158646, 0.115185976028, 0.115034103394, 0.114280939102, 0.115108966827, 0.115477085114, 0.114944934845, 0.11555695533800001, 0.111644983292, 0.111333847046, 0.111480951309, 0.11155700683600001, 0.11224794387799999, 0.112046003342, 0.111447095871, 0.11138176918000001, 0.11152100563, 0.111184120178, 0.11187982559200001, 0.111309051514, 0.115062952042, 0.111434936523, 0.111573934555, 0.111452102661, 0.128764867783, 0.11133313179, 0.11140704155, 0.11155796051, 0.111167907715, 0.111421108246, 0.11097002029399999, 0.111446857452, 0.110933065414, 0.110732078552, 0.113990068436, 0.11116909980799999, 0.111049890518, 0.11179399490399999, 0.11199402809099999, 0.110781908035, 0.111202001572, 0.111080884933, 0.110934972763, 0.111124992371, 0.110868930817, 0.11123108863800001, 0.112289190292], "base_times": [0.48088502883899997, 0.47923994064300002, 0.48160886764499999, 0.47763085365300001, 0.47884607315099997, 0.48438310623199998, 0.48113894462599999, 0.47794389724699998, 0.47896790504499998, 0.48052501678499998, 0.47819399833699999, 0.47891688346900002, 0.48100805282600001, 0.48024415969799999, 0.47854995727499999, 0.47899198532100001, 0.47972202301, 0.47723197937, 0.48147606849699998, 0.47865796089200002, 0.47799110412599999, 0.47740817070000002, 0.47738909721400002, 0.47732496261599999, 0.47762703895600001, 0.48322010040300001, 0.47945380210900002, 0.47645592689499999, 0.479950904846, 0.47760391235400002, 0.47802209854099997, 0.47923922538800001, 0.477582931519, 0.47682905197100001, 0.47848796844500002, 0.47821402549699998, 0.47620606422400003, 0.48374009132399998, 0.481714963913, 0.47922301292399999, 0.47735190391499999, 0.48569798469499997, 0.47899794578600002, 0.477490901947, 0.47891783714300001, 0.47670793533299999, 0.47655487060500001, 0.47783589363099999, 0.47823095321699999, 0.47739887237500001]}], ["telco", "RawResult", {"changed_times": [0.56803599999999999, 0.51203200000000004, 0.51603200000000005, 0.54403400000000002, 0.51203200000000004, 0.53603400000000001, 0.492031, 0.496031, 0.49203000000000002, 0.51203200000000004, 0.496031, 0.492031, 0.500031, 0.492031, 0.50403100000000001, 0.50403200000000004, 0.496031, 0.492031, 0.49203000000000002, 0.50403200000000004, 0.492031, 0.492031, 0.51203200000000004, 0.49203000000000002, 0.49203200000000002, 0.50403100000000001, 0.492031, 0.49203000000000002, 0.50403200000000004, 0.48803000000000002, 0.496031, 0.50403100000000001, 0.48803099999999999, 0.492031, 0.50003200000000003, 0.49203000000000002, 0.492031, 0.496031, 0.492031, 0.492031, 0.500031, 0.496031, 0.492031, 0.500031, 0.492031, 0.48803099999999999, 0.50403100000000001, 0.48803099999999999, 0.492031, 0.500031], "base_times": [1.22, 1.2, 1.21, 1.21, 1.2, 1.21, 1.22, 1.2, 1.2, 1.22, 1.2, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.2, 1.21, 1.22, 1.21, 1.21, 1.21, 1.2, 1.2, 1.22, 1.2, 1.21, 1.21, 1.2, 1.2, 1.21, 1.21, 1.2, 1.22, 1.21, 1.21, 1.2, 1.21, 1.21, 1.21, 1.21, 1.2, 1.2, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21]}], ["twisted_names", "RawResult", {"changed_times": [0.0074571215510812828, 0.0072046109510086453, 0.0071581961345740875, 0.0076745970836531079, 0.007215007215007215, 0.007446016381236039, 0.0072674418604651162, 0.0073909830007390983, 0.007320644216691069, 0.007246376811594203, 0.007446016381236039, 0.0071684587813620072, 0.0073691967575534268, 0.0070372976776917661, 0.0071530758226037196, 0.0074239049740163323, 0.0070721357850070717, 0.0070077084793272598, 0.0074794315632011965, 0.007052186177715092, 0.0075815011372251705, 0.0074850299401197605, 0.0069979006298110571, 0.0071174377224199285, 0.0075187969924812026, 0.0072833211944646759, 0.007763975155279503, 0.006939625260235947, 0.006939625260235947, 0.007331378299120235, 0.006954102920723227, 0.006954102920723227, 0.006939625260235947, 0.0073421439060205578, 0.0069156293222683261, 0.0068917987594762234, 0.0073746312684365781, 0.0069444444444444441, 0.006920415224913495, 0.0073637702503681884, 0.0069492703266157054, 0.0069300069300069298, 0.0070571630204657732, 0.0069108500345542506, 0.0068917987594762234, 0.0069348127600554789, 0.0073046018991964941, 0.0069637883008356544, 0.006954102920723227, 0.0073583517292126564], "base_times": [0.0096246390760346481, 0.0095877277085330784, 0.0095419847328244278, 0.0095877277085330784, 0.0095602294455066923, 0.0095785440613026813, 0.0095693779904306216, 0.0095510983763132766, 0.0095419847328244278, 0.0099009900990099011, 0.0095693779904306216, 0.0095419847328244278, 0.0096061479346781949, 0.0095693779904306216, 0.0095510983763132766, 0.0095969289827255271, 0.0095785440613026813, 0.0095602294455066923, 0.0095785440613026813, 0.0095969289827255271, 0.0095785440613026813, 0.0096061479346781949, 0.0095602294455066923, 0.0095877277085330784, 0.0095877277085330784, 0.0095602294455066923, 0.0095877277085330784, 0.0095510983763132766, 0.0095328884652049577, 0.0095877277085330784, 0.0096061479346781949, 0.0095693779904306216, 0.0096061479346781949, 0.0095785440613026813, 0.0095693779904306216, 0.0095693779904306216, 0.0095785440613026813, 0.0095510983763132766, 0.0095877277085330784, 0.0095877277085330784, 0.0095785440613026813, 0.0095785440613026813, 0.0095969289827255271, 0.0095693779904306216, 0.0095877277085330784, 0.0095602294455066923, 0.0095419847328244278, 0.0095877277085330784, 0.0095785440613026813, 0.0095785440613026813]}]], "branch": "trunk", "revision": 0}
\ No newline at end of file
+{"options": "", "results": [["crypto_pyaes", "RawResult", {"changed_times": [0.82904410362199998, 0.356333971024, 0.27036023139999998, 0.26120996475199998, 0.26673913001999999, 0.27246308326699997, 0.25749182701099999, 0.25644612312300002, 0.26470899581899998, 0.29816889762900001, 0.27409410476700002, 0.25924706459000002, 0.26701903343200001, 0.256891012192, 0.26991915702800001, 0.33074879646299998, 0.26318502426099999, 0.26682686805700001, 0.25480794906600002, 0.258436918259, 0.25471806526200003, 0.27756595611599999, 0.27438092231799999, 0.25547194480899998, 0.29641008377099998, 0.286099910736, 0.25961518287699997, 0.27814412116999998, 0.258821964264, 0.255064964294, 0.28223800659199999, 0.31574010849, 0.27696394920299999, 0.25558805465700002, 0.25136613845799999, 0.25414109229999998, 0.25539398193399998, 0.25389981269799999, 0.26006698608399997, 0.27383089065600003, 0.28252100944500003, 0.28729081153899999, 0.254612207413, 0.25178408622699999, 0.247875213623, 0.248955011368, 0.25435090064999999, 0.277958154678, 0.25217485427899999, 0.27278900146500001], "base_times": [2.76247382164, 2.75780797005, 2.7526109218600001, 2.7581448554999999, 2.7579169273400002, 2.7591660022700002, 2.7544469833399998, 2.75570702553, 2.7566180229200001, 2.7799978256200002, 2.75436115265, 2.7584359645799998, 2.7715408801999999, 2.7583799362199999, 2.75467610359, 2.7834770679499998, 2.7558917999300001, 2.7597181797000001, 2.75486803055, 2.7581379413599998, 2.7532849311800001, 2.7636089324999999, 2.75440192223, 2.7553980350499998, 2.7562680244400002, 2.7588288784000001, 2.75431799889, 2.7562849521600001, 2.7566390037500001, 2.75684499741, 2.7588860988600001, 2.7639617919899999, 2.758425951, 2.7611510753599999, 2.75679397583, 2.7544021606400002, 2.7587559223200002, 2.7592377662700001, 2.7555589675899999, 2.7590520381900001, 2.75563788414, 2.76086902618, 2.7588081359899999, 2.7565140724199999, 2.75627803802, 2.76230502129, 2.7579100132000001, 2.75721096992, 2.7558379173300001, 2.7606220245399999]}], ["django", "RawResult", {"changed_times": [0.30002903938300002, 0.25580286979700001, 0.261374950409, 0.26095104217499998, 0.25220704078700001, 0.26073884963999999, 0.26132106780999997, 0.254949092865, 0.26201701164199998, 0.26217198371900002, 0.25196504592899999, 0.26127791404700001, 0.26208996772799997, 0.254974842072, 0.261093854904, 0.261404037476, 0.25236296653700002, 0.26157402992200002, 0.26323699951200003, 0.25301718711900001, 0.26132202148400002, 0.26150584220899997, 0.25210714340200002, 0.26118707656899998, 0.26585507392899999, 0.25383305549599999, 0.26143312454200002, 0.26261687278700002, 0.267751932144, 0.26117897033699999, 0.264419078827, 0.252134799957, 0.26171708107000002, 0.26141405105600002, 0.25201201438900001, 0.261445045471, 0.26490807533299998, 0.25207686424300002, 0.26115393638599999, 0.26161694526700002, 0.25278902053800001, 0.261168003082, 0.26426982879599997, 0.25209307670600001, 0.261046171188, 0.26155900955200001, 0.25206208229100002, 0.261226177216, 0.26494908332799999, 0.252925872803], "base_times": [0.995049953461, 0.99419713020300005, 0.99113988876299997, 0.99470710754400005, 0.99212503433200006, 0.99265909194900004, 0.99358582496600001, 0.99400091171299998, 0.99305796623200004, 0.99362897872900002, 0.99429702758799998, 0.99161601066600003, 0.99425101280200001, 0.99299907684300004, 0.99139285087600004, 0.99515914917000003, 0.99177312850999999, 0.99377703666700001, 0.99326705932600001, 0.99304509162900001, 0.99348807334900002, 0.99273705482499996, 0.99258708953899999, 0.99477910995499996, 0.99403500556900004, 0.99051785469099995, 0.994111061096, 0.99377894401599998, 0.99141192436199999, 0.99447512626599999, 0.99370288848900001, 0.99225115776100004, 0.99282193183900003, 0.99233508110000002, 0.99293303489700002, 0.99670696258500002, 0.99345684051500005, 0.99477696418799999, 0.99382996559100001, 0.99122619628899999, 0.99395608901999999, 0.99324393272400002, 0.99102210998499995, 0.99499702453600003, 0.99375295639000005, 0.99156498909000002, 0.99277806282000003, 0.99424910545300005, 0.99084305763199998, 0.993386983871]}], ["fannkuch", "RawResult", {"changed_times": [0.52755904197699999, 0.47701597213699998, 0.46741414070100001, 0.47383809089700002, 0.46101403236400001, 0.46737408638, 0.47243213653600002, 0.47586488723800002, 0.46479701995799999, 0.46248412132299999, 0.46604609489400001, 0.46004605293299999, 0.46376204490700002, 0.46708703041100003, 0.47270798683199999, 0.472360134125, 0.47210407257100001, 0.47691607475300002, 0.47826886177099998, 0.47204399108900003, 0.47201299667399998, 0.46304917335500001, 0.46217513084400003, 0.46784305572500001, 0.460357189178, 0.46357607841499998, 0.46536898613, 0.46894693374599999, 0.46510815620399998, 0.46110606193499998, 0.46555995941200001, 0.46238088607799999, 0.46605896949800002, 0.46549701690700002, 0.45882105827300002, 0.45827507972699999, 0.473033905029, 0.47556495666499998, 0.460822105408, 0.47130393981899998, 0.476540803909, 0.46522593498199999, 0.46266603469799999, 0.47138500213599999, 0.46661305427600003, 0.46587181091300001, 0.47512102127099998, 0.47284221649199998, 0.47128200530999997, 0.47180795669600001], "base_times": [1.9883148670199999, 1.9792110919999999, 1.9909551143599999, 1.98928499222, 1.97898578644, 1.98664617538, 1.9864010810899999, 1.98417901993, 1.9872028827699999, 1.98194694519, 1.9878189563799999, 1.98948693275, 1.9811220169099999, 1.9890899658200001, 1.9894771575900001, 1.9801352024100001, 1.9894909858700001, 1.9891271591199999, 1.98107600212, 1.99568986893, 1.9895131587999999, 1.9803609847999999, 2.0069239139600001, 1.9916350841499999, 1.98022890091, 1.98729896545, 1.99073004723, 1.98082304001, 1.98834180832, 1.9909420013400001, 1.9793639183, 1.98839306831, 1.9902248382600001, 1.9822490215299999, 1.98810195923, 1.99047279358, 1.98221516609, 1.98903012276, 1.9900078773500001, 1.9820079803499999, 1.9892179966000001, 1.9895989894899999, 1.9800350665999999, 1.9895751476300001, 1.9891831874799999, 1.9804949760399999, 1.9896249771100001, 1.9898300170900001, 1.9807391166699999, 1.9895370006599999]}], ["go", "RawResult", {"changed_times": [0.46677207946799998, 0.51466798782300005, 0.388464927673, 0.50362396240200002, 0.36921596527099998, 0.41929197311400002, 0.37606406211900001, 0.428333997726, 0.36696696281399999, 0.33521819114700002, 0.35271787643399999, 0.47167301177999998, 0.36826205253599997, 0.33157110214199997, 0.39287805557299998, 0.30100107192999997, 0.31953597068799999, 0.34765505790700002, 0.31619405746500001, 0.32286810874900002, 0.37090992927600003, 0.328006982803, 0.31113696098299998, 0.355150938034, 0.30218696594200001, 0.31915521621699999, 0.31008601188700002, 0.34765815734900002, 0.30721187591600002, 0.32940602302600003, 0.30662703514099998, 0.33826303482100001, 0.29518413543700001, 0.32855200767499998, 0.37890100479099997, 0.295953989029, 0.28935694694500003, 0.32658910751300002, 0.31018304824800003, 0.32643103599500001, 0.28621220588700003, 0.29142880439800001, 0.28649687766999998, 0.301717042923, 0.288222074509, 0.353245973587, 0.33032798767100002, 0.285399913788, 0.29600501060500001, 0.28842616081200001], "base_times": [0.94673204422000001, 0.94645810127300001, 0.94507908821099995, 0.944802045822, 0.941561937332, 0.944491147995, 0.94291400909400003, 0.95224094390900005, 0.94410514831500003, 0.943855047226, 0.94802284240699997, 0.94546699523900002, 0.94773101806600002, 0.94944882392899999, 0.94337701797499995, 0.94636011123700003, 0.94657802581799999, 0.94187402725199998, 0.94828701019299999, 0.94231510162400001, 0.94628596305799995, 0.94881415367099997, 0.94279098510699999, 0.94889903068500003, 0.95280098915099998, 0.94382286071800003, 0.94861888885500001, 0.94221186637900001, 0.94600486755400004, 0.94160294532800004, 0.94753003120400003, 0.94430899620099995, 0.94843101501500005, 0.96007490158099995, 0.94707822799700003, 0.94710588455199995, 0.95057606697100006, 0.94280695915199997, 0.94800686836199999, 0.94509601593000003, 0.96038508415199997, 0.94446587562600004, 0.94845819473299997, 0.94734406471300003, 0.94510412216200002, 0.94387602806100002, 0.94524908065799995, 0.94992303848299997, 0.94564318656900004, 0.94311594963099998]}], ["html5lib", "RawResult", {"changed_times": [15.457784891099999, 11.3277301788, 10.600764036199999, 10.4216821194, 10.2325599194, 10.2733259201, 10.3756709099, 10.156955957399999, 10.101744890200001, 10.133589983, 10.206784963600001, 10.1694169044, 10.0392251015, 10.155544042600001, 10.1885080338, 10.058653831499999, 10.2810080051, 10.2360329628, 10.039163112600001, 10.1998898983, 10.119593143499999, 10.1641438007, 10.0413920879, 10.107867002500001, 10.135586977000001, 10.1675360203, 10.1493079662, 10.060842990899999, 10.178154945399999, 10.1074330807, 10.170939922300001, 10.047384023699999, 10.0127818584, 10.050449132900001, 10.161669015899999, 10.037492990500001, 10.0900728703, 10.190579891200001, 9.9983761310600006, 10.1199560165, 10.003987073899999, 10.119729042099999, 10.1512379646, 10.054825067499999, 9.9258110523200003, 10.0976631641, 10.0940468311, 10.122640132900001, 9.9850409030900007, 10.099215984300001], "base_times": [13.800020933200001, 13.988759994500001, 13.945523977300001, 13.9515860081, 13.8968560696, 14.0439591408, 14.0257151127, 13.9940800667, 13.977200985, 13.965268135100001, 13.8857018948, 14.0525372028, 14.022065877899999, 14.0086979866, 13.961042880999999, 13.921565771099999, 13.924687862400001, 14.0379269123, 14.256644964199999, 13.9928328991, 14.0013000965, 13.9456880093, 13.914921999000001, 14.061727047, 14.0501580238, 14.0101730824, 13.9787597656, 13.921334981899999, 13.889928102500001, 14.0468928814, 14.0285110474, 14.0453250408, 13.9559979439, 13.9637241364, 13.9317610264, 14.0579519272, 14.0324690342, 14.0096049309, 13.9675040245, 13.964776992799999, 13.924336910199999, 14.0519120693, 14.0543570518, 14.0083789825, 13.9841370583, 13.954766988799999, 13.9033069611, 13.8949439526, 14.038803100599999, 14.0201179981]}], ["meteor-contest", "RawResult", {"changed_times": [0.47674798965499998, 0.403456926346, 0.39809894561800002, 0.40778803825400001, 0.39824509620699999, 0.39766597747799998, 0.40741491317700002, 0.39641904830899999, 0.39668798446699999, 0.39379596710199999, 0.39647078514099998, 0.393444061279, 0.40611696243299999, 0.39961814880399998, 0.39285993576, 0.39224314689599998, 0.39388489723199999, 0.39147710800199997, 0.39165306091300001, 0.39033508300800002, 0.39437389373800003, 0.39175200462300003, 0.391242027283, 0.39103412628200002, 0.39511799812300002, 0.398463010788, 0.39204406738300002, 0.39300107955899999, 0.39334487914999999, 0.391426801682, 0.39687800407399998, 0.39431095123299997, 0.39070391654999997, 0.39106798172000001, 0.39058089256299999, 0.39391088485699999, 0.39834094047500002, 0.39057397842399999, 0.39198994636500001, 0.39368200302099998, 0.39073801040599998, 0.39122891426099998, 0.39064598083500002, 0.39390277862500001, 0.390840053558, 0.39128398895299998, 0.39098405838, 0.39335107803300001, 0.39786505699199998, 0.39080500602700002], "base_times": [0.348359107971, 0.34692883491499998, 0.346405982971, 0.34667205810500001, 0.34701514244100001, 0.34753990173299998, 0.346569061279, 0.34635281562800002, 0.346190929413, 0.34796619415300001, 0.34590101242100002, 0.34643101692200001, 0.34670686721799998, 0.34775090217600002, 0.34620094299300003, 0.34622883796699999, 0.34623694419899997, 0.34779596328700002, 0.348699092865, 0.34642219543500002, 0.347030878067, 0.34581995010400002, 0.347792863846, 0.34766483306899998, 0.34653806686400002, 0.34683513641399999, 0.34936118125900001, 0.34839010238599999, 0.346399068832, 0.34670805931100002, 0.34576678276099998, 0.348170042038, 0.34661507606500003, 0.34650683402999999, 0.34621381759600001, 0.34878396987900001, 0.34638595581100001, 0.34623980522199999, 0.34593510627700003, 0.34620404243500003, 0.34790611267100002, 0.346742153168, 0.34607505798299998, 0.34633517265300001, 0.34812307357799999, 0.34604907035799998, 0.34677386283900002, 0.34667706489599998, 0.34659886360199998, 0.34881806373599999]}], ["nbody_modified", "RawResult", {"changed_times": [0.12817502021800001, 0.114922046661, 0.105006933212, 0.10339498519900001, 0.104254007339, 0.103782892227, 0.103751897812, 0.105894088745, 0.104180812836, 0.103867053986, 0.103599071503, 0.103564023972, 0.103793859482, 0.103470087051, 0.103558063507, 0.103418111801, 0.10340905189500001, 0.10347700119, 0.103801012039, 0.10350394249, 0.103738069534, 0.10368895530699999, 0.106331110001, 0.103569984436, 0.10441899299600001, 0.103644132614, 0.103709936142, 0.10339307785, 0.103735923767, 0.10362410545300001, 0.103760004044, 0.103440999985, 0.10381603240999999, 0.103622913361, 0.10393595695500001, 0.103415966034, 0.103775978088, 0.106086969376, 0.103862047195, 0.103460073471, 0.103792905807, 0.10356497764600001, 0.103854894638, 0.103960990906, 0.103703975677, 0.103446006775, 0.10406708717300001, 0.103525876999, 0.103811979294, 0.10510087013199999], "base_times": [0.64283800125099999, 0.64096808433499997, 0.64577603340099998, 0.64423298835800002, 0.64046311378499998, 0.63754677772500001, 0.63747310638400001, 0.63725614547700005, 0.63658809661899995, 0.64228487014799995, 0.64188194274900001, 0.64234304428099998, 0.63921594619800004, 0.63971495628399999, 0.64251685142500004, 0.64014887809800003, 0.64212799072299998, 0.64027500152600003, 0.63313889503499998, 0.63153290748599999, 0.63076686859099995, 0.63370203971899997, 0.63236403465299995, 0.63347291946399997, 0.63246107101399995, 0.63299584388700003, 0.63348007202099998, 0.63026380538899995, 0.632724046707, 0.63243818283099995, 0.63531398773199999, 0.63819217681899998, 0.63529610633900002, 0.63803815841699996, 0.63671016693100002, 0.64081311225899995, 0.63594484329199996, 0.64017796516400005, 0.64193677902199997, 0.64040708541900004, 0.64205718040500004, 0.64145493507399998, 0.640824079514, 0.65274095535300003, 0.64189887046799998, 0.64417505264300001, 0.64191007614100004, 0.640285015106, 0.63908886909499996, 0.64495897293100002]}], ["pyflate-fast", "RawResult", {"changed_times": [1.83701682091, 1.8073658943199999, 1.8406269550300001, 1.82018709183, 1.8337500095399999, 1.8074929714200001, 1.84810090065, 1.8087329864499999, 1.8338890075700001, 1.81683397293, 1.78763699532, 1.79593896866, 1.81039500237, 1.7986569404599999, 1.8007159233100001, 1.8014991283399999, 1.7972960472099999, 1.80028700829, 1.8007249832200001, 1.85811805725, 1.7931230068199999, 1.82364487648, 1.7935469150500001, 1.82844901085, 1.7920200824700001, 1.82318806648, 1.7929878234900001, 1.82861304283, 1.79402518272, 1.82314896584, 1.79180502892, 1.8225588798500001, 1.80868387222, 1.80998492241, 1.8257219791399999, 1.8084599971799999, 1.81027412415, 1.80907607079, 1.80926918983, 1.82390213013, 1.8229339122799999, 1.9030389785799999, 1.9522318839999999, 1.8275589942899999, 1.81771707535, 1.8227140903500001, 1.8171408176399999, 1.8353171348599999, 1.8287210464500001, 1.83105301857], "base_times": [3.1601369381, 3.17563796043, 3.1705720424699999, 3.17048382759, 3.1654319763199998, 3.1680698394800002, 3.1681051254299999, 3.1718590259599999, 3.1830987930300001, 3.1775770187400001, 3.1700208187099999, 3.1612432002999999, 3.15727186203, 3.1732730865500001, 3.1751198768600002, 3.1624131202700001, 3.1771841049199998, 3.1661839485200001, 3.16439390182, 3.1774570941900002, 3.1796469688400002, 3.16518807411, 3.1729710102099999, 3.1790311336500001, 3.17372894287, 3.1641249656700001, 3.1653139591200001, 3.16444206238, 3.1757740974400002, 3.16623592377, 3.1762218475299999, 3.1651339530899998, 3.1656432151799998, 3.1724681854200001, 3.1735718250299998, 3.1667320728299999, 3.1666581630700001, 3.1662278175399998, 3.1711161136600001, 3.1634638309500001, 3.1635270118699998, 3.1630849838300001, 3.1776990890499999, 3.17032909393, 3.1747660636899999, 3.1702299117999999, 3.17133522034, 3.1701278686499998, 3.1603660583500002, 3.1653149128]}], ["raytrace-simple", "RawResult", {"changed_times": [0.659919023514, 0.66626596450800002, 0.65816497802700002, 0.65398693084699999, 0.63991713523899996, 0.66545104980500003, 0.65175795555100002, 0.63982892036399996, 0.66220998764000005, 0.65392279624899996, 0.63932204246500002, 0.65032505989099998, 0.65231609344499997, 0.63614416122399997, 0.65278100967399999, 0.63543891906700001, 0.65301513671900002, 0.647172212601, 0.63230013847400002, 0.64931488037100005, 0.689571857452, 0.63732600212099999, 0.67306900024399996, 0.63961911201499999, 0.650694131851, 0.63388586044299999, 0.65063095092800005, 0.64781498909000002, 0.63850116729700002, 0.65920114517200001, 0.68630504608200005, 0.63646793365499998, 0.64802885055500004, 0.64920401573200004, 0.63225889205899999, 0.65038490295399998, 0.63108992576599998, 0.64513182640099997, 0.63317394256600001, 0.64602303504900005, 0.64737510681199995, 0.63001990318300005, 0.64657092094400004, 0.63210487365699997, 0.645840883255, 0.65991592407199995, 0.62948513030999997, 0.65090513229400004, 0.63588213920600001, 0.66070103645300005], "base_times": [2.6739389896399999, 2.6792252063799999, 2.67714691162, 2.6755440235100001, 2.6839830875400001, 2.67896699905, 2.6759178638500001, 2.6771140098599999, 2.6831061840100001, 2.6781611442600002, 2.6843791008000002, 2.67555809021, 2.67742395401, 2.68145084381, 2.6856698989900001, 2.6754090786, 2.6820809841200002, 2.7008218765300001, 2.6860649585699998, 2.68976688385, 2.7091708183300001, 2.6831979751600001, 2.6825587749499999, 2.7229101657900001, 2.7227280139899999, 2.7074718475299999, 2.7032160758999999, 2.69513916969, 2.7023360729200001, 2.7511570453599998, 2.6899399757400002, 2.6838908195500002, 2.7002210617100002, 2.8071360588099998, 2.6863989830000001, 2.6864531040199999, 2.69359779358, 2.7453310489699998, 2.7076761722599998, 3.06150817871, 2.7332241535200001, 2.7483820915199999, 2.7639210224199999, 3.4124541282699998, 2.7048859596299999, 2.6892578601800001, 2.6912610530899999, 2.6821250915500001, 2.68154597282, 2.6890358924900002]}], ["richards", "RawResult", {"changed_times": [0.12052297592199999, 0.093158006668099994, 0.091630935668900002, 0.091804027557399998, 0.091670036315899997, 0.091012954711900004, 0.091180086135899993, 0.102787017822, 0.090846061706500003, 0.090148925781199998, 0.093178987503099997, 0.090260028839100004, 0.090500116348300005, 0.098414897918700006, 0.090541839599599994, 0.090975999832200005, 0.090515136718800002, 0.088860034942600002, 0.089382886886599996, 0.087617158889799998, 0.087574005127000004, 0.088225841522200005, 0.087996006012, 0.087512969970700002, 0.0879878997803, 0.096507072448699993, 0.087088108062699995, 0.090749025344800005, 0.087268829345700002, 0.087903976440400003, 0.087549924850500005, 0.087819099426300007, 0.087373971939100001, 0.087810039520300004, 0.087095975875899997, 0.0871419906616, 0.088514089584400005, 0.087375879287700006, 0.086944818496699994, 0.087454080581699997, 0.087461948394799996, 0.097897052764900003, 0.087609052658099998, 0.087203979492200004, 0.089391946792599999, 0.088086128234900002, 0.086940050125100005, 0.087315082550000006, 0.086781978607199994, 0.086982011794999994], "base_times": [0.35359501838700003, 0.35667300224300003, 0.347419977188, 0.35246205329899999, 0.35532999038700003, 0.35530591011000001, 0.35107088088999999, 0.35549902915999998, 0.35110902786300002, 0.35400104522699999, 0.35261011123699998, 0.35541200637800002, 0.35351705551099999, 0.35617089271500002, 0.34958410263099998, 0.35414409637499999, 0.35661983489999999, 0.35693001747100001, 0.35174703598000001, 0.34876704215999998, 0.35491299629200002, 0.35479497909500002, 0.35416102409400002, 0.355372905731, 0.35764288902300001, 0.35487508773799997, 0.35283422470100001, 0.35428977012599999, 0.35132908821100001, 0.35751414299000001, 0.362582206726, 0.35600781440700002, 0.35644507408100001, 0.35086202621500001, 0.35400605201699997, 0.34989190101599998, 0.35055398940999999, 0.35048794746400003, 0.35601019859299998, 0.35249996185299998, 0.35182499885599999, 0.35495114326499999, 0.35285496711699998, 0.355787038803, 0.35039997100800002, 0.35504388809199999, 0.35427594184900002, 0.35611510276800001, 0.35387182235699999, 0.34967398643499997]}], ["spambayes", "RawResult", {"changed_times": [0.48457980155899999, 0.56999397277800001, 0.53518199920700005, 0.54824614524799997, 0.49303603172299998, 0.58462905883799998, 0.51067399978600003, 0.49086999893200001, 0.48435497283899998, 0.42737793922400003, 0.494954109192, 0.43986797332799998, 0.40522003173799997, 0.40721011161800003, 0.57795190811200003, 0.48255205154399999, 0.462779998779, 0.44051218032799999, 0.39301896095299999, 0.39432501792899999, 0.39888882637, 0.43233299255399998, 0.42077803611800002, 0.39932012558000002, 0.40041995048500001, 0.409831047058, 0.44888401031500003, 0.40377283096299998, 0.39395999908399998, 0.40184497833299998, 0.41734480857799999, 0.49758601188700002, 0.39052391052199997, 0.392160177231, 0.39913892746000001, 0.42745113372799998, 0.36878705024699998, 0.38831090927099998, 0.37554693222000002, 0.40881800651599998, 0.37994885444600002, 0.37922286987300002, 0.39221501350400001, 0.39279913902300001, 0.34234404563900001, 0.37785792350800002, 0.38006091117899998, 0.37980985641499998, 0.375820875168, 0.345314979553], "base_times": [0.29905700683600001, 0.298835992813, 0.29825806617700001, 0.29831504821799998, 0.30105614662199998, 0.298439979553, 0.29832100868200001, 0.29835200309799997, 0.29867696762099999, 0.29994201660199998, 0.29847502708399998, 0.29867196083100001, 0.29891991615300001, 0.29807305336000001, 0.30059790611300002, 0.298420906067, 0.29828691482500003, 0.29897713661199998, 0.29811811447100001, 0.300410985947, 0.29847979545600001, 0.2982878685, 0.29861903190599998, 0.29877710342399999, 0.30105996131899998, 0.29816794395399998, 0.29835987091100002, 0.29860401153600002, 0.2985060215, 0.30007410049400002, 0.29955601692200001, 0.299430847168, 0.29854393005399998, 0.29915118217499997, 0.29966092109699999, 0.29980802536000001, 0.29794311523400002, 0.29930615425099999, 0.298702001572, 0.29842782020600001, 0.30247402191200001, 0.29896807670600001, 0.298580169678, 0.29869008064300001, 0.299312114716, 0.30094408988999999, 0.29845499992399999, 0.29868721962, 0.29998612403899999, 0.29916310310400002]}], ["spectral-norm", "RawResult", {"changed_times": [0.18530607223500001, 0.118788957596, 0.11839389801, 0.117673158646, 0.115185976028, 0.115034103394, 0.114280939102, 0.115108966827, 0.115477085114, 0.114944934845, 0.11555695533800001, 0.111644983292, 0.111333847046, 0.111480951309, 0.11155700683600001, 0.11224794387799999, 0.112046003342, 0.111447095871, 0.11138176918000001, 0.11152100563, 0.111184120178, 0.11187982559200001, 0.111309051514, 0.115062952042, 0.111434936523, 0.111573934555, 0.111452102661, 0.128764867783, 0.11133313179, 0.11140704155, 0.11155796051, 0.111167907715, 0.111421108246, 0.11097002029399999, 0.111446857452, 0.110933065414, 0.110732078552, 0.113990068436, 0.11116909980799999, 0.111049890518, 0.11179399490399999, 0.11199402809099999, 0.110781908035, 0.111202001572, 0.111080884933, 0.110934972763, 0.111124992371, 0.110868930817, 0.11123108863800001, 0.112289190292], "base_times": [0.48088502883899997, 0.47923994064300002, 0.48160886764499999, 0.47763085365300001, 0.47884607315099997, 0.48438310623199998, 0.48113894462599999, 0.47794389724699998, 0.47896790504499998, 0.48052501678499998, 0.47819399833699999, 0.47891688346900002, 0.48100805282600001, 0.48024415969799999, 0.47854995727499999, 0.47899198532100001, 0.47972202301, 0.47723197937, 0.48147606849699998, 0.47865796089200002, 0.47799110412599999, 0.47740817070000002, 0.47738909721400002, 0.47732496261599999, 0.47762703895600001, 0.48322010040300001, 0.47945380210900002, 0.47645592689499999, 0.479950904846, 0.47760391235400002, 0.47802209854099997, 0.47923922538800001, 0.477582931519, 0.47682905197100001, 0.47848796844500002, 0.47821402549699998, 0.47620606422400003, 0.48374009132399998, 0.481714963913, 0.47922301292399999, 0.47735190391499999, 0.48569798469499997, 0.47899794578600002, 0.477490901947, 0.47891783714300001, 0.47670793533299999, 0.47655487060500001, 0.47783589363099999, 0.47823095321699999, 0.47739887237500001]}], ["telco", "RawResult", {"changed_times": [0.56803599999999999, 0.51203200000000004, 0.51603200000000005, 0.54403400000000002, 0.51203200000000004, 0.53603400000000001, 0.492031, 0.496031, 0.49203000000000002, 0.51203200000000004, 0.496031, 0.492031, 0.500031, 0.492031, 0.50403100000000001, 0.50403200000000004, 0.496031, 0.492031, 0.49203000000000002, 0.50403200000000004, 0.492031, 0.492031, 0.51203200000000004, 0.49203000000000002, 0.49203200000000002, 0.50403100000000001, 0.492031, 0.49203000000000002, 0.50403200000000004, 0.48803000000000002, 0.496031, 0.50403100000000001, 0.48803099999999999, 0.492031, 0.50003200000000003, 0.49203000000000002, 0.492031, 0.496031, 0.492031, 0.492031, 0.500031, 0.496031, 0.492031, 0.500031, 0.492031, 0.48803099999999999, 0.50403100000000001, 0.48803099999999999, 0.492031, 0.500031], "base_times": [1.22, 1.2, 1.21, 1.21, 1.2, 1.21, 1.22, 1.2, 1.2, 1.22, 1.2, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21, 1.2, 1.21, 1.22, 1.21, 1.21, 1.21, 1.2, 1.2, 1.22, 1.2, 1.21, 1.21, 1.2, 1.2, 1.21, 1.21, 1.2, 1.22, 1.21, 1.21, 1.2, 1.21, 1.21, 1.21, 1.21, 1.2, 1.2, 1.21, 1.21, 1.21, 1.21, 1.21, 1.21]}], ["twisted_names", "RawResult", {"changed_times": [0.0074571215510812828, 0.0072046109510086453, 0.0071581961345740875, 0.0076745970836531079, 0.007215007215007215, 0.007446016381236039, 0.0072674418604651162, 0.0073909830007390983, 0.007320644216691069, 0.007246376811594203, 0.007446016381236039, 0.0071684587813620072, 0.0073691967575534268, 0.0070372976776917661, 0.0071530758226037196, 0.0074239049740163323, 0.0070721357850070717, 0.0070077084793272598, 0.0074794315632011965, 0.007052186177715092, 0.0075815011372251705, 0.0074850299401197605, 0.0069979006298110571, 0.0071174377224199285, 0.0075187969924812026, 0.0072833211944646759, 0.007763975155279503, 0.006939625260235947, 0.006939625260235947, 0.007331378299120235, 0.006954102920723227, 0.006954102920723227, 0.006939625260235947, 0.0073421439060205578, 0.0069156293222683261, 0.0068917987594762234, 0.0073746312684365781, 0.0069444444444444441, 0.006920415224913495, 0.0073637702503681884, 0.0069492703266157054, 0.0069300069300069298, 0.0070571630204657732, 0.0069108500345542506, 0.0068917987594762234, 0.0069348127600554789, 0.0073046018991964941, 0.0069637883008356544, 0.006954102920723227, 0.0073583517292126564], "base_times": [0.0096246390760346481, 0.0095877277085330784, 0.0095419847328244278, 0.0095877277085330784, 0.0095602294455066923, 0.0095785440613026813, 0.0095693779904306216, 0.0095510983763132766, 0.0095419847328244278, 0.0099009900990099011, 0.0095693779904306216, 0.0095419847328244278, 0.0096061479346781949, 0.0095693779904306216, 0.0095510983763132766, 0.0095969289827255271, 0.0095785440613026813, 0.0095602294455066923, 0.0095785440613026813, 0.0095969289827255271, 0.0095785440613026813, 0.0096061479346781949, 0.0095602294455066923, 0.0095877277085330784, 0.0095877277085330784, 0.0095602294455066923, 0.0095877277085330784, 0.0095510983763132766, 0.0095328884652049577, 0.0095877277085330784, 0.0096061479346781949, 0.0095693779904306216, 0.0096061479346781949, 0.0095785440613026813, 0.0095693779904306216, 0.0095693779904306216, 0.0095785440613026813, 0.0095510983763132766, 0.0095877277085330784, 0.0095877277085330784, 0.0095785440613026813, 0.0095785440613026813, 0.0095969289827255271, 0.0095693779904306216, 0.0095877277085330784, 0.0095602294455066923, 0.0095419847328244278, 0.0095877277085330784, 0.0095785440613026813, 0.0095785440613026813]}]], "branch": "trunk", "revision": 0}
From cfbolz at codespeak.net Wed Oct 13 22:32:07 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Wed, 13 Oct 2010 22:32:07 +0200 (CEST)
Subject: [pypy-svn] r77893 - pypy/extradoc/talk/pepm2011
Message-ID: <20101013203207.C9ABA282BDB@codespeak.net>
Author: cfbolz
Date: Wed Oct 13 22:32:06 2010
New Revision: 77893
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
re-import benchmark times, fix one XXX
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Wed Oct 13 22:32:06 2010
@@ -989,6 +989,7 @@
removed guards are an effect of the optimization described here.
\begin{figure*}
+\begin{center}
\begin{tabular}{|l||r|rr|rr|rr|rr|}
\hline
&num loops &new &removed &get/set &removed &guard &removed &all ops &removed\\
@@ -1011,6 +1012,7 @@
total &2240 &49709 &70\% &1227447 &14\% &222569 &93\% &3395548 &89\%\\
\hline
\end{tabular}
+\end{center}
\caption{Number of Operations and Percentage Removed By Optimization}
\label{fig:numops}
\end{figure*}
@@ -1022,29 +1024,35 @@
confidence level \cite{georges_statistically_2007}. The results are reported in Figure~\ref{fig:times}.
With the optimization turned on, PyPy's Python interpreter outperforms CPython
in all benchmarks except spambayes (which heavily relies on regular expression
-performance XXX). All benchmarks are improved by the allocation removal
-optimization, some as much as XXX. XXX Psyco
+performance and thus is not helped much by our Python JIT) and meteor-contest.
+All benchmarks are improved by the allocation removal optimization, some as much
+as XXX. XXX Psyco
XXX runtimes of the algorithm somehow?
\begin{figure*}
-\begin{tabular}{lrrrr}
- &CPython &JIT no optimizations &JIT allocation removal &JIT full\\
-crypto\_pyaes &1958.40 $\pm$ 0.32 &782.14 $\pm$ 3.08 &150.86 $\pm$ 3.62 &125.68 $\pm$ 3.02\\
-django &630.91 $\pm$ 0.13 &365.90 $\pm$ 1.57 &124.41 $\pm$ 0.88 &117.36 $\pm$ 0.91\\
-fannkuch &1344.15 $\pm$ 0.36 &378.18 $\pm$ 0.36 &309.04 $\pm$ 1.56 &299.12 $\pm$ 0.25\\
-go &595.31 $\pm$ 0.51 &1067.26 $\pm$ 12.20 &121.76 $\pm$ 4.15 &125.57 $\pm$ 3.90\\
-html5lib &8678.67 $\pm$ 23.37 &17665.16 $\pm$ 2695.45 &6739.95 $\pm$ 1225.25 &6699.24 $\pm$ 1295.92\\
-meteor-contest &241.53 $\pm$ 0.09 &291.11 $\pm$ 0.86 &274.09 $\pm$ 0.37 &272.77 $\pm$ 0.30\\
-nbody &396.55 $\pm$ 0.24 &84.77 $\pm$ 0.14 &69.01 $\pm$ 0.08 &68.71 $\pm$ 0.09\\
-pyflate-fast &1991.33 $\pm$ 1.90 &1818.70 $\pm$ 4.30 &1062.03 $\pm$ 4.56 &1052.02 $\pm$ 3.24\\
-raytrace-simple &1598.84 $\pm$ 1.07 &1126.93 $\pm$ 11.42 &93.45 $\pm$ 4.75 &92.17 $\pm$ 4.15\\
-richards &216.47 $\pm$ 0.20 &169.84 $\pm$ 1.21 &11.81 $\pm$ 0.09 &11.90 $\pm$ 0.29\\
-spambayes &193.30 $\pm$ 0.03 &331.37 $\pm$ 14.93 &238.83 $\pm$ 6.10 &237.23 $\pm$ 5.97\\
-spectral-norm &375.91 $\pm$ 0.08 &221.24 $\pm$ 0.07 &31.96 $\pm$ 0.06 &28.79 $\pm$ 0.09\\
-telco &789.67 $\pm$ 1.14 &622.67 $\pm$ 1.86 &155.33 $\pm$ 1.82 &154.67 $\pm$ 1.82\\
-twisted\_names &6.61 $\pm$ 0.00 &8.85 $\pm$ 0.09 &4.35 $\pm$ 0.05 &4.35 $\pm$ 0.04\\
+\begin{center}
+\begin{tabular}{|l||r|r|r|r|}
+\hline
+ &Cpython &Psyco &PyPy w/o optimizations &PyPy w/ optimizations \\
+\hline
+crypto\_pyaes &2757.80 $\pm$ 0.98 &67.90 $\pm$ 0.47 &1652.00 $\pm$ 4.00 &266.86 $\pm$ 5.94 \\
+django &993.19 $\pm$ 0.50 &913.51 $\pm$ 4.22 &694.73 $\pm$ 2.86 &259.53 $\pm$ 1.79 \\
+fannkuch &1987.22 $\pm$ 2.02 &944.44 $\pm$ 0.61 &566.99 $\pm$ 1.06 &466.87 $\pm$ 1.85 \\
+go &947.21 $\pm$ 1.58 &445.96 $\pm$ 0.68 &2197.71 $\pm$ 25.21 &316.15 $\pm$ 9.33 \\
+html5lib &13987.12 $\pm$ 19.51 &17398.25 $\pm$ 36.50 &27194.45 $\pm$ 46.62 &10092.19 $\pm$ 23.50 \\
+meteor-contest &346.98 $\pm$ 0.35 &215.66 $\pm$ 0.23 &433.04 $\pm$ 1.45 &392.85 $\pm$ 0.87 \\
+nbody\_modified &637.90 $\pm$ 1.82 &256.78 $\pm$ 0.18 &135.55 $\pm$ 0.33 &103.93 $\pm$ 0.25 \\
+pyflate-fast &3169.35 $\pm$ 1.89 &1278.16 $\pm$ 3.13 &3285.89 $\pm$ 8.51 &1822.36 $\pm$ 11.52 \\
+raytrace-simple &2744.60 $\pm$ 51.72 &1072.66 $\pm$ 1.08 &2778.27 $\pm$ 15.13 &647.24 $\pm$ 5.44 \\
+richards &354.06 $\pm$ 1.00 &63.48 $\pm$ 0.15 &383.93 $\pm$ 3.28 &88.32 $\pm$ 0.91 \\
+spambayes &299.16 $\pm$ 0.35 &338.68 $\pm$ 3.14 &580.90 $\pm$ 24.68 &397.37 $\pm$ 10.60 \\
+spectral-norm &478.63 $\pm$ 0.80 &139.83 $\pm$ 1.54 &353.51 $\pm$ 1.39 &112.10 $\pm$ 1.17 \\
+telco &1207.67 $\pm$ 2.03 &730.00 $\pm$ 2.66 &1296.08 $\pm$ 4.37 &495.23 $\pm$ 2.14 \\
+twisted\_names &9.58 $\pm$ 0.01 &10.43 $\pm$ 0.01 &17.99 $\pm$ 0.27 &7.13 $\pm$ 0.09 \\
+\hline
\end{tabular}
+\end{center}
\caption{Benchmark Times}
\label{fig:times}
\end{figure*}
From cfbolz at codespeak.net Wed Oct 13 22:41:51 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Wed, 13 Oct 2010 22:41:51 +0200 (CEST)
Subject: [pypy-svn] r77894 - pypy/extradoc/talk/pepm2011
Message-ID: <20101013204151.1A148282B9E@codespeak.net>
Author: cfbolz
Date: Wed Oct 13 22:41:48 2010
New Revision: 77894
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
give a unit
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Wed Oct 13 22:41:48 2010
@@ -1053,7 +1053,7 @@
\hline
\end{tabular}
\end{center}
-\caption{Benchmark Times}
+\caption{Benchmark Times in Milliseconds}
\label{fig:times}
\end{figure*}
From afa at codespeak.net Thu Oct 14 00:59:28 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 14 Oct 2010 00:59:28 +0200 (CEST)
Subject: [pypy-svn] r77895 - in
pypy/branch/fast-forward/pypy/module/_multiprocessing: . test
Message-ID: <20101013225928.DBAB4282B9E@codespeak.net>
Author: afa
Date: Thu Oct 14 00:59:27 2010
New Revision: 77895
Modified:
pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_semaphore.py
pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_semaphore.py
Log:
Test and fix with recursive mutex
Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_semaphore.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_semaphore.py (original)
+++ pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_semaphore.py Thu Oct 14 00:59:27 2010
@@ -354,6 +354,7 @@
def semlock_release(self, space):
if self.kind == RECURSIVE_MUTEX:
+ sem_post(self.handle)
return
if HAVE_BROKEN_SEM_GETVALUE:
# We will only check properly the maxvalue == 1 case
Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_semaphore.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_semaphore.py (original)
+++ pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_semaphore.py Thu Oct 14 00:59:27 2010
@@ -7,6 +7,7 @@
space = gettestobjspace(usemodules=('_multiprocessing', 'thread'))
cls.space = space
cls.w_SEMAPHORE = space.wrap(SEMAPHORE)
+ cls.w_RECURSIVE = space.wrap(RECURSIVE_MUTEX)
def test_semaphore(self):
from _multiprocessing import SemLock
@@ -31,6 +32,26 @@
sem.release()
assert sem._count() == 0
+ def test_recursive(self):
+ from _multiprocessing import SemLock
+ kind = self.RECURSIVE
+ value = 1
+ maxvalue = 1
+ sem = SemLock(kind, value, maxvalue)
+
+ sem.acquire()
+ sem.release()
+ assert sem._count() == 0
+ sem.acquire()
+ sem.release()
+
+ # now recursively
+ sem.acquire()
+ sem.acquire()
+ assert sem._count() == 2
+ sem.release()
+ sem.release()
+
def test_semaphore_wait(self):
from _multiprocessing import SemLock
kind = self.SEMAPHORE
From afa at codespeak.net Thu Oct 14 08:52:11 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 14 Oct 2010 08:52:11 +0200 (CEST)
Subject: [pypy-svn] r77896 - in pypy/branch/fast-forward/pypy/objspace/std:
. test
Message-ID: <20101014065211.3E58B282BE3@codespeak.net>
Author: afa
Date: Thu Oct 14 08:52:06 2010
New Revision: 77896
Modified:
pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py
Log:
Enhance bytearray constructor
Modified: pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py Thu Oct 14 08:52:06 2010
@@ -1,6 +1,7 @@
import sys
from pypy.interpreter import gateway
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
+from pypy.interpreter.error import OperationError
from pypy.objspace.std.register_all import register_all
from pypy.objspace.std.stdtypedef import StdTypeDef, SMM
@@ -14,15 +15,66 @@
str_join, str_split, str_rsplit, str_partition, str_rpartition,
str_splitlines)
- at gateway.unwrap_spec(ObjSpace, W_Root, W_Root, W_Root, W_Root)
-def descr__new__(space, w_bytearraytype,
- w_source='', w_encoding=None, w_errors=None):
+def _getbytevalue(space, w_value):
+ if space.isinstance_w(w_value, space.w_str):
+ string = space.str_w(w_value)
+ if len(string) != 1:
+ raise OperationError(space.w_ValueError, space.wrap(
+ "string must be of size 1"))
+ return string[0]
+
+ value = space.getindex_w(w_value, None)
+ if not 0 <= value < 256:
+ # this includes the OverflowError in case the long is too large
+ raise OperationError(space.w_ValueError, space.wrap(
+ "byte must be in range(0, 256)"))
+ return chr(value)
+
+def new_bytearray(space, w_bytearraytype, data):
from pypy.objspace.std.bytearrayobject import W_BytearrayObject
- data = [c for c in space.str_w(w_source)]
w_obj = space.allocate_instance(W_BytearrayObject, w_bytearraytype)
W_BytearrayObject.__init__(w_obj, data)
return w_obj
+ at gateway.unwrap_spec(ObjSpace, W_Root, W_Root, W_Root, W_Root)
+def descr__new__(space, w_bytearraytype,
+ w_source='', w_encoding=None, w_errors=None):
+
+ data = []
+ # String-like argument
+ try:
+ string = space.str_w(w_source)
+ except OperationError, e:
+ if not e.match(space, space.w_TypeError):
+ raise
+ else:
+ data = [c for c in string]
+ return new_bytearray(space, w_bytearraytype, data)
+
+ # Is it an int?
+ try:
+ count = space.int_w(w_source)
+ except OperationError, e:
+ if not e.match(space, space.w_TypeError):
+ raise
+ else:
+ data = ['\0'] * count
+ return new_bytearray(space, w_bytearraytype, data)
+
+ # sequence of bytes
+ w_iter = space.iter(w_source)
+ while True:
+ try:
+ w_item = space.next(w_iter)
+ except OperationError, e:
+ if not e.match(space, space.w_StopIteration):
+ raise
+ break
+ value = _getbytevalue(space, w_item)
+ data.append(value)
+
+ return new_bytearray(space, w_bytearraytype, data)
+
# ____________________________________________________________
bytearray_typedef = StdTypeDef("bytearray",
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py Thu Oct 14 08:52:06 2010
@@ -5,6 +5,16 @@
assert type(b) is bytearray
assert b.__class__ is bytearray
+ def test_constructor(self):
+ assert bytearray() == ""
+ assert bytearray('abc') == "abc"
+ assert bytearray(['a', 'b', 'c']) == "abc"
+ assert bytearray([65, 66, 67]) == "ABC"
+ assert bytearray(5) == '\0' * 5
+ raises(ValueError, bytearray, ['a', 'bc'])
+ raises(ValueError, bytearray, [65, -3])
+ raises(TypeError, bytearray, [65.0])
+
def test_len(self):
b = bytearray('test')
assert len(b) == 4
From cfbolz at codespeak.net Thu Oct 14 11:18:33 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 11:18:33 +0200 (CEST)
Subject: [pypy-svn] r77898 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014091833.1F507282BE3@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 11:18:31 2010
New Revision: 77898
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
two things from samuele, regen pdf
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 11:18:31 2010
@@ -469,8 +469,7 @@
comparison (``greater than''), respectively.
\end{itemize}
-Note how the functions that are called by \lstinline{f} are automatically inlined
-into the trace. The method calls are always preceded by a \lstinline{guard_class}
+The method calls in the trace are always preceded by a \lstinline{guard_class}
operation, to check that the class of the receiver is the same as the one that
was observed during tracing.\footnote{\lstinline{guard_class} performs a precise
class check, not checking for subclasses.} These guards make the trace specific
@@ -549,6 +548,9 @@
\subsection{Static Objects}
+XXX add a footnote saying that "static" is meant in the sense of PE, not static
+allocation
+
The main insight to improve the code shown in the last section is that objects
in category 1 don't survive very long -- they are used only inside the loop and
nobody else in the program stores a reference to them. The idea for improving
From cfbolz at codespeak.net Thu Oct 14 11:20:39 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 11:20:39 +0200 (CEST)
Subject: [pypy-svn] r77899 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014092039.95A20282BE3@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 11:20:38 2010
New Revision: 77899
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
another comment by samuele
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 11:20:38 2010
@@ -484,6 +484,8 @@
In the rest of the paper we will see how this trace can be optimized using
partial evaluation.
+XXX add comment that guards are not only slow, but also use lots of memory
+
\section{Object Lifetimes in a Tracing JIT}
\label{sec:lifetimes}
From fijal at codespeak.net Thu Oct 14 11:32:17 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Thu, 14 Oct 2010 11:32:17 +0200 (CEST)
Subject: [pypy-svn] r77900 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014093217.A6F83282BE3@codespeak.net>
Author: fijal
Date: Thu Oct 14 11:32:16 2010
New Revision: 77900
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
Add a comment
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 11:32:16 2010
@@ -134,6 +134,10 @@
dispatching. Those are problems that are usually not present or at least less
severe in statically typed languages.
+XXX [fijal] I would say that the other goal of the JIT in case of dynamic
+languages is to compile only a common scenario and leave a guard (instead
+of proving that something will never ever happen)
+
Boxing of primitive types is necessary because dynamic languages need to be able to handle
all objects, even integers, floats, booleans etc. in the same way as user-defined
instances. Thus those primitive types are usually \emph{boxed}, i.e. a small
From arigo at codespeak.net Thu Oct 14 11:35:56 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 14 Oct 2010 11:35:56 +0200 (CEST)
Subject: [pypy-svn] r77901 - in pypy/branch/32ptr-on-64bit/pypy/rpython:
lltypesystem test
Message-ID: <20101014093556.1536C282BE3@codespeak.net>
Author: arigo
Date: Thu Oct 14 11:35:54 2010
New Revision: 77901
Modified:
pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py
pypy/branch/32ptr-on-64bit/pypy/rpython/test/test_rcompressed.py
Log:
Test and fix: GCREF may point to a var-sized object, so it cannot be compressed.
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/lltypesystem/rcompressed.py Thu Oct 14 11:35:54 2010
@@ -14,6 +14,8 @@
# mmap()-within-32GB-of-RAM.
if baserepr.lowleveltype.TO._is_varsize():
return baserepr
+ if baserepr.lowleveltype == llmemory.GCREF:
+ return baserepr # a GCREF may contain a non-fixed-size object
try:
comprmgr = rtyper.compressed_gcref_manager
except AttributeError:
Modified: pypy/branch/32ptr-on-64bit/pypy/rpython/test/test_rcompressed.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/rpython/test/test_rcompressed.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/rpython/test/test_rcompressed.py Thu Oct 14 11:35:54 2010
@@ -167,6 +167,12 @@
assert er is r_L
assert ir.lowleveltype == llmemory.HiddenGcRef32
+ def test_rerased(self):
+ from pypy.rlib.rerased import ErasedRepr
+ r_E = ErasedRepr(self.rtyper)
+ er, ir = rmodel.externalvsinternal(self. rtyper, r_E)
+ assert er is ir is r_E
+
class TestLLtype64(MixinCompressed64, test_rclass.TestLLtype):
From arigo at codespeak.net Thu Oct 14 11:36:31 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 14 Oct 2010 11:36:31 +0200 (CEST)
Subject: [pypy-svn] r77902 - pypy/branch/32ptr-on-64bit/pypy/translator/c/src
Message-ID: <20101014093631.ABFED282BE3@codespeak.net>
Author: arigo
Date: Thu Oct 14 11:36:30 2010
New Revision: 77902
Modified:
pypy/branch/32ptr-on-64bit/pypy/translator/c/src/linuxmemchk.c
Log:
Update the limit to a recently-useful number.
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/src/linuxmemchk.c
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/src/linuxmemchk.c (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/src/linuxmemchk.c Thu Oct 14 11:36:30 2010
@@ -7,7 +7,7 @@
#include
#define PAGESIZE 4096
#ifndef MALLOC_BIGBUFFER
-# define MALLOC_BIGBUFFER (PAGESIZE*32768) /* 128MB */
+# define MALLOC_BIGBUFFER (PAGESIZE*32768*14) /* 1.75GB */
#endif
From arigo at codespeak.net Thu Oct 14 11:38:26 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 14 Oct 2010 11:38:26 +0200 (CEST)
Subject: [pypy-svn] r77903 - in
pypy/branch/32ptr-on-64bit/pypy/objspace/std: . test
Message-ID: <20101014093826.A0F61282BE3@codespeak.net>
Author: arigo
Date: Thu Oct 14 11:38:25 2010
New Revision: 77903
Modified:
pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py
pypy/branch/32ptr-on-64bit/pypy/objspace/std/test/test_mapdict.py
Log:
Update mapdicts to only use rerased for the last field,
which allows all other fields to be compressed.
Modified: pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py Thu Oct 14 11:38:25 2010
@@ -413,11 +413,14 @@
rangen = unroll.unrolling_iterable(range(n))
nmin1 = n - 1
rangenmin1 = unroll.unrolling_iterable(range(nmin1))
+ #
class subcls(ObjectMixin, BaseMapdictObject, supercls):
+ _nmin1 = nmin1
+ for _i in rangenmin1:
+ locals()["_value%s" % _i] = None
+ locals()["_value%s" % nmin1] = rerased.erase(None)
+
def _init_empty(self, map):
- from pypy.rlib.debug import make_sure_not_resized
- for i in rangen:
- setattr(self, "_value%s" % i, rerased.erase(None))
self.map = map
def _has_storage_list(self):
@@ -430,22 +433,21 @@
def _mapdict_read_storage(self, index):
for i in rangenmin1:
if index == i:
- erased = getattr(self, "_value%s" % i)
- return rerased.unerase(erased, W_Root)
+ return getattr(self, "_value%s" % i)
if self._has_storage_list():
return self._mapdict_get_storage_list()[index - nmin1]
erased = getattr(self, "_value%s" % nmin1)
return rerased.unerase(erased, W_Root)
def _mapdict_write_storage(self, index, value):
- erased = rerased.erase(value)
for i in rangenmin1:
if index == i:
- setattr(self, "_value%s" % i, erased)
+ setattr(self, "_value%s" % i, value)
return
if self._has_storage_list():
self._mapdict_get_storage_list()[index - nmin1] = value
return
+ erased = rerased.erase(value)
setattr(self, "_value%s" % nmin1, erased)
def _mapdict_storage_length(self):
@@ -458,10 +460,10 @@
len_storage = len(storage)
for i in rangenmin1:
if i < len_storage:
- erased = rerased.erase(storage[i])
+ value = storage[i]
else:
- erased = rerased.erase(None)
- setattr(self, "_value%s" % i, erased)
+ value = None
+ setattr(self, "_value%s" % i, value)
has_storage_list = self._has_storage_list()
if len_storage < n:
assert not has_storage_list
Modified: pypy/branch/32ptr-on-64bit/pypy/objspace/std/test/test_mapdict.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/objspace/std/test/test_mapdict.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/objspace/std/test/test_mapdict.py Thu Oct 14 11:38:25 2010
@@ -372,12 +372,18 @@
obj = objectcls()
obj.user_setup(space, cls)
obj.setdictvalue(space, "a", w1)
- assert rerased.unerase(obj._value0, W_Root) is w1
+ if objectcls._nmin1 == 0:
+ assert rerased.unerase(obj._value0, W_Root) is w1
+ else:
+ assert obj._value0 is w1
assert obj.getdictvalue(space, "a") is w1
assert obj.getdictvalue(space, "b") is None
assert obj.getdictvalue(space, "c") is None
obj.setdictvalue(space, "a", w2)
- assert rerased.unerase(obj._value0, W_Root) is w2
+ if objectcls._nmin1 == 0:
+ assert rerased.unerase(obj._value0, W_Root) is w2
+ else:
+ assert obj._value0 is w2
assert obj.getdictvalue(space, "a") == w2
assert obj.getdictvalue(space, "b") is None
assert obj.getdictvalue(space, "c") is None
@@ -395,7 +401,10 @@
res = obj.deldictvalue(space, "a")
assert res
- assert rerased.unerase(obj._value0, W_Root) is w4
+ if objectcls._nmin1 == 0:
+ assert rerased.unerase(obj._value0, W_Root) is w4
+ else:
+ assert obj._value0 is w4
assert obj.getdictvalue(space, "a") is None
assert obj.getdictvalue(space, "b") is w4
assert obj.getdictvalue(space, "c") is None
From arigo at codespeak.net Thu Oct 14 11:40:19 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 14 Oct 2010 11:40:19 +0200 (CEST)
Subject: [pypy-svn] r77904 - in
pypy/branch/32ptr-on-64bit/pypy/objspace/std: . test
Message-ID: <20101014094019.ABFB2282BE3@codespeak.net>
Author: arigo
Date: Thu Oct 14 11:40:18 2010
New Revision: 77904
Modified:
pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py
pypy/branch/32ptr-on-64bit/pypy/objspace/std/test/test_mapdict.py
Log:
Upgrade the previous hack by not using 'rerased' at all in the compressptr
case. Instead, point to a small W_Root subclass that itself contains the
pointer to the list.
Modified: pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py Thu Oct 14 11:40:18 2010
@@ -359,8 +359,8 @@
assert isinstance(weakreflifeline, WeakrefLifeline)
self._get_mapdict_map().write(self, ("weakref", SPECIAL), weakreflifeline)
-class ObjectMixin(object):
- _mixin_ = True
+class Object(BaseMapdictObject, W_Root):
+ # mainly for tests
def _init_empty(self, map):
from pypy.rlib.debug import make_sure_not_resized
self.map = map
@@ -376,18 +376,13 @@
self.storage = storage
self.map = map
-class Object(ObjectMixin, BaseMapdictObject, W_Root):
- pass # mainly for tests
-
def get_subclass_of_correct_size(space, cls, w_type):
assert space.config.objspace.std.withmapdict
map = w_type.terminator
classes = memo_get_subclass_of_correct_size(space, cls)
size = map.size_estimate()
- if not size:
- size = 1
try:
- return classes[size - 1]
+ return classes[size]
except IndexError:
return classes[-1]
get_subclass_of_correct_size._annspecialcase_ = "specialize:arg(1)"
@@ -401,24 +396,48 @@
except KeyError:
assert not hasattr(supercls, "__del__")
result = []
- for i in range(1, NUM_SUBCLASSES+1):
- result.append(_make_subclass_size_n(supercls, i))
+ if space.config.translation.compressptr:
+ # with 'compressptr', there are two issues: first we only need
+ # half as many classes, because there is no point in having a
+ # class that contains an odd number of hiddengcrefs (there is
+ # one too for the map, so there is no point in having an even
+ # number of attributes); and we avoid using rlib.rerased because
+ # such an erased pointer cannot be compressed, as it might point
+ # to a list.
+ for i in range(0, NUM_SUBCLASSES // 2 + 1):
+ subcls = _make_subclass_size_n(supercls, i * 2 + 1,
+ use_erased=False)
+ result.append(subcls)
+ result.append(subcls)
+ else:
+ # common case
+ for i in range(1, NUM_SUBCLASSES+1):
+ result.append(_make_subclass_size_n(supercls, i,
+ use_erased=True))
+ result.insert(0, result[0])
_subclass_cache[key] = result
return result
memo_get_subclass_of_correct_size._annspecialcase_ = "specialize:memo"
_subclass_cache = {}
-def _make_subclass_size_n(supercls, n):
- from pypy.rlib import unroll, rerased
+def _make_subclass_size_n(supercls, n, use_erased):
+ from pypy.rlib import unroll
rangen = unroll.unrolling_iterable(range(n))
nmin1 = n - 1
rangenmin1 = unroll.unrolling_iterable(range(nmin1))
+ if use_erased:
+ from pypy.rlib import rerased
+ erase = rerased.erase
+ unerase = rerased.unerase
+ else:
+ erase = lambda x: x
+ unerase = lambda x, t: x
#
- class subcls(ObjectMixin, BaseMapdictObject, supercls):
+ class subcls(BaseMapdictObject, supercls):
_nmin1 = nmin1
for _i in rangenmin1:
locals()["_value%s" % _i] = None
- locals()["_value%s" % nmin1] = rerased.erase(None)
+ locals()["_value%s" % nmin1] = erase(None)
def _init_empty(self, map):
self.map = map
@@ -428,7 +447,11 @@
def _mapdict_get_storage_list(self):
erased = getattr(self, "_value%s" % nmin1)
- return rerased.unerase_fixedsizelist(erased, W_Root)
+ if use_erased:
+ return rerased.unerase_fixedsizelist(erased, W_Root)
+ else:
+ assert isinstance(erased, ExtraAttributes)
+ return erased.storage
def _mapdict_read_storage(self, index):
for i in rangenmin1:
@@ -437,7 +460,7 @@
if self._has_storage_list():
return self._mapdict_get_storage_list()[index - nmin1]
erased = getattr(self, "_value%s" % nmin1)
- return rerased.unerase(erased, W_Root)
+ return unerase(erased, W_Root)
def _mapdict_write_storage(self, index, value):
for i in rangenmin1:
@@ -447,7 +470,7 @@
if self._has_storage_list():
self._mapdict_get_storage_list()[index - nmin1] = value
return
- erased = rerased.erase(value)
+ erased = erase(value)
setattr(self, "_value%s" % nmin1, erased)
def _mapdict_storage_length(self):
@@ -467,25 +490,33 @@
has_storage_list = self._has_storage_list()
if len_storage < n:
assert not has_storage_list
- erased = rerased.erase(None)
+ erased = erase(None)
elif len_storage == n:
assert not has_storage_list
- erased = rerased.erase(storage[nmin1])
+ erased = erase(storage[nmin1])
elif not has_storage_list:
# storage is longer than self.map.length() only due to
# overallocation
- erased = rerased.erase(storage[nmin1])
+ erased = erase(storage[nmin1])
# in theory, we should be ultra-paranoid and check all entries,
# but checking just one should catch most problems anyway:
assert storage[n] is None
else:
storage_list = storage[nmin1:]
- erased = rerased.erase_fixedsizelist(storage_list, W_Root)
+ if use_erased:
+ erased = rerased.erase_fixedsizelist(storage_list, W_Root)
+ else:
+ erased = ExtraAttributes(storage_list)
setattr(self, "_value%s" % nmin1, erased)
subcls.__name__ = supercls.__name__ + "Size%s" % n
return subcls
+class ExtraAttributes(W_Root):
+ def __init__(self, storage):
+ from pypy.rlib.debug import make_sure_not_resized
+ self.storage = make_sure_not_resized(storage)
+
# ____________________________________________________________
# dict implementation
Modified: pypy/branch/32ptr-on-64bit/pypy/objspace/std/test/test_mapdict.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/objspace/std/test/test_mapdict.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/objspace/std/test/test_mapdict.py Thu Oct 14 11:40:18 2010
@@ -1,6 +1,7 @@
from pypy.conftest import gettestobjspace, option
from pypy.objspace.std.test.test_dictmultiobject import FakeSpace, W_DictMultiObject
from pypy.objspace.std.mapdict import *
+from pypy.objspace.std.mapdict import _subclass_cache
space = FakeSpace()
@@ -357,9 +358,15 @@
# check specialized classes
-def test_specialized_class():
+def test_specialized_class(compressptr=False):
from pypy.objspace.std.objectobject import W_ObjectObject
from pypy.rlib import rerased
+ space = FakeSpace()
+ class translation:
+ pass
+ translation.compressptr = compressptr
+ space.config.translation = translation
+ _subclass_cache.clear()
classes = memo_get_subclass_of_correct_size(space, W_ObjectObject)
w1 = W_Root()
w2 = W_Root()
@@ -372,7 +379,7 @@
obj = objectcls()
obj.user_setup(space, cls)
obj.setdictvalue(space, "a", w1)
- if objectcls._nmin1 == 0:
+ if objectcls._nmin1 == 0 and not compressptr:
assert rerased.unerase(obj._value0, W_Root) is w1
else:
assert obj._value0 is w1
@@ -380,7 +387,7 @@
assert obj.getdictvalue(space, "b") is None
assert obj.getdictvalue(space, "c") is None
obj.setdictvalue(space, "a", w2)
- if objectcls._nmin1 == 0:
+ if objectcls._nmin1 == 0 and not compressptr:
assert rerased.unerase(obj._value0, W_Root) is w2
else:
assert obj._value0 is w2
@@ -401,7 +408,7 @@
res = obj.deldictvalue(space, "a")
assert res
- if objectcls._nmin1 == 0:
+ if objectcls._nmin1 == 0 and not compressptr:
assert rerased.unerase(obj._value0, W_Root) is w4
else:
assert obj._value0 is w4
@@ -417,6 +424,9 @@
assert obj2.getdictvalue(space, "b") is w6
assert obj2.map is abmap
+def test_specialized_class_compressptr():
+ test_specialized_class(compressptr=True)
+
# ___________________________________________________________
# integration tests
From afa at codespeak.net Thu Oct 14 11:42:03 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 14 Oct 2010 11:42:03 +0200 (CEST)
Subject: [pypy-svn] r77905 - in
pypy/branch/fast-forward/pypy/module/_hashlib: . test
Message-ID: <20101014094203.0699F282BE3@codespeak.net>
Author: afa
Date: Thu Oct 14 11:42:02 2010
New Revision: 77905
Modified:
pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py
pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py
Log:
Hash value was truncated to the first \0 :-(
Test and fix.
Modified: pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py (original)
+++ pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py Thu Oct 14 11:42:02 2010
@@ -71,7 +71,7 @@
try:
ropenssl.EVP_DigestFinal(ctx, digest, None)
- return rffi.charp2strn(digest, digest_size)
+ return rffi.charpsize2str(digest, digest_size)
finally:
lltype.free(digest, flavor='raw')
Modified: pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py (original)
+++ pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py Thu Oct 14 11:42:02 2010
@@ -8,6 +8,7 @@
def test_simple(self):
import _hashlib
assert isinstance(_hashlib.new('md5'), _hashlib.HASH)
+ assert len(_hashlib.new('md5').hexdigest()) == 32
def test_attributes(self):
import hashlib
From arigo at codespeak.net Thu Oct 14 11:43:47 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 14 Oct 2010 11:43:47 +0200 (CEST)
Subject: [pypy-svn] r77906 - pypy/branch/32ptr-on-64bit/pypy/objspace/std
Message-ID: <20101014094347.A086E282BE3@codespeak.net>
Author: arigo
Date: Thu Oct 14 11:43:46 2010
New Revision: 77906
Modified:
pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py
Log:
Revert this part of the change. ObjectMixin is needed
by interpreter/typedef.py.
Modified: pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py Thu Oct 14 11:43:46 2010
@@ -359,8 +359,8 @@
assert isinstance(weakreflifeline, WeakrefLifeline)
self._get_mapdict_map().write(self, ("weakref", SPECIAL), weakreflifeline)
-class Object(BaseMapdictObject, W_Root):
- # mainly for tests
+class ObjectMixin(object):
+ _mixin_ = True
def _init_empty(self, map):
from pypy.rlib.debug import make_sure_not_resized
self.map = map
@@ -376,6 +376,9 @@
self.storage = storage
self.map = map
+class Object(ObjectMixin, BaseMapdictObject, W_Root):
+ pass # mainly for tests
+
def get_subclass_of_correct_size(space, cls, w_type):
assert space.config.objspace.std.withmapdict
map = w_type.terminator
From antocuni at codespeak.net Thu Oct 14 11:47:43 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Thu, 14 Oct 2010 11:47:43 +0200 (CEST)
Subject: [pypy-svn] r77907 - in pypy/branch/jitffi/pypy/module/_ffi: . test
Message-ID: <20101014094743.F10CF282BE3@codespeak.net>
Author: antocuni
Date: Thu Oct 14 11:47:42 2010
New Revision: 77907
Modified:
pypy/branch/jitffi/pypy/module/_ffi/interp_ffi.py
pypy/branch/jitffi/pypy/module/_ffi/test/test__ffi.py
Log:
add a way to the the actual address of the underlying function. Useful for testing
Modified: pypy/branch/jitffi/pypy/module/_ffi/interp_ffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/_ffi/interp_ffi.py (original)
+++ pypy/branch/jitffi/pypy/module/_ffi/interp_ffi.py Thu Oct 14 11:47:42 2010
@@ -96,9 +96,17 @@
assert voidres is None
return space.w_None
+ @unwrap_spec('self', ObjSpace)
+ def getaddr(self, space):
+ """
+ Return the physical address in memory of the function
+ """
+ return space.wrap(rffi.cast(rffi.LONG, self.func.funcsym))
+
W_FuncPtr.typedef = TypeDef(
'FuncPtr',
- __call__ = interp2app(W_FuncPtr.call)
+ __call__ = interp2app(W_FuncPtr.call),
+ getaddr = interp2app(W_FuncPtr.getaddr),
)
Modified: pypy/branch/jitffi/pypy/module/_ffi/test/test__ffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/_ffi/test/test__ffi.py (original)
+++ pypy/branch/jitffi/pypy/module/_ffi/test/test__ffi.py Thu Oct 14 11:47:42 2010
@@ -32,13 +32,19 @@
def setup_class(cls):
- from pypy.rlib.libffi import get_libc_name
+ from pypy.rpython.lltypesystem import rffi
+ from pypy.rlib.libffi import get_libc_name, CDLL, types
from pypy.rlib.test.test_libffi import get_libm_name
space = gettestobjspace(usemodules=('_ffi',))
cls.space = space
cls.w_libfoo_name = space.wrap(cls.prepare_c_example())
cls.w_libc_name = space.wrap(get_libc_name())
- cls.w_libm_name = space.wrap(get_libm_name(sys.platform))
+ libm_name = get_libm_name(sys.platform)
+ cls.w_libm_name = space.wrap(libm_name)
+ libm = CDLL(libm_name)
+ pow = libm.getpointer('pow', [], types.void)
+ pow_addr = rffi.cast(rffi.LONG, pow.funcsym)
+ cls.w_pow_addr = space.wrap(pow_addr)
def test_libload(self):
import _ffi
@@ -58,6 +64,12 @@
libm = CDLL(self.libm_name)
pow = libm.getfunc('pow', [types.double, types.double], types.double)
assert pow(2, 3) == 8
+
+ def test_getaddr(self):
+ from _ffi import CDLL, types
+ libm = CDLL(self.libm_name)
+ pow = libm.getfunc('pow', [types.double, types.double], types.double)
+ assert pow.getaddr() == self.pow_addr
def test_int_args(self):
"""
From arigo at codespeak.net Thu Oct 14 11:57:09 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 14 Oct 2010 11:57:09 +0200 (CEST)
Subject: [pypy-svn] r77908 - pypy/branch/32ptr-on-64bit/pypy/objspace/std
Message-ID: <20101014095709.47FC4282BE3@codespeak.net>
Author: arigo
Date: Thu Oct 14 11:57:07 2010
New Revision: 77908
Modified:
pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py
Log:
Flow graph fix.
Modified: pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/objspace/std/mapdict.py Thu Oct 14 11:57:07 2010
@@ -433,6 +433,7 @@
erase = rerased.erase
unerase = rerased.unerase
else:
+ rerased = None # don't use in that case
erase = lambda x: x
unerase = lambda x, t: x
#
From afa at codespeak.net Thu Oct 14 12:04:09 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 14 Oct 2010 12:04:09 +0200 (CEST)
Subject: [pypy-svn] r77909 - in pypy/branch/fast-forward/pypy:
module/_hashlib rlib
Message-ID: <20101014100409.925C4282BE3@codespeak.net>
Author: afa
Date: Thu Oct 14 12:04:07 2010
New Revision: 77909
Modified:
pypy/branch/fast-forward/pypy/module/_hashlib/__init__.py
pypy/branch/fast-forward/pypy/rlib/ropenssl.py
Log:
"OpenSSL_add_all_digests()" is neeeded to call other hashlib functions,
difficult to test because CPython does it already in the same process.
Modified: pypy/branch/fast-forward/pypy/module/_hashlib/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_hashlib/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/_hashlib/__init__.py Thu Oct 14 12:04:07 2010
@@ -13,3 +13,7 @@
for name in algorithms:
interpleveldefs[name] = 'interp_hashlib.new_%s' % (name,)
+
+ def startup(self, space):
+ from pypy.rlib.ropenssl import init_digests
+ init_digests()
Modified: pypy/branch/fast-forward/pypy/rlib/ropenssl.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/ropenssl.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/ropenssl.py Thu Oct 14 12:04:07 2010
@@ -128,6 +128,8 @@
EVP_MD_CTX = rffi.COpaquePtr('EVP_MD_CTX', compilation_info=eci)
EVP_MD = rffi.COpaquePtr('EVP_MD')
+OpenSSL_add_all_digests = external(
+ 'OpenSSL_add_all_digests', [], lltype.Void)
EVP_get_digestbyname = external(
'EVP_get_digestbyname',
[rffi.CCHARP], EVP_MD)
@@ -149,3 +151,5 @@
libssl_SSL_load_error_strings()
libssl_SSL_library_init()
+def init_digests():
+ OpenSSL_add_all_digests()
From afa at codespeak.net Thu Oct 14 12:05:57 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 14 Oct 2010 12:05:57 +0200 (CEST)
Subject: [pypy-svn] r77910 - in pypy/branch/fast-forward: lib_pypy
lib_pypy/ctypes_config_cache/test lib_pypy/pypy_test
pypy/module/_md5 pypy/module/_sha
Message-ID: <20101014100557.B9256282BE3@codespeak.net>
Author: afa
Date: Thu Oct 14 12:05:55 2010
New Revision: 77910
Modified:
pypy/branch/fast-forward/lib_pypy/_md5.py
pypy/branch/fast-forward/lib_pypy/_sha.py
pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/test/test_cache.py
pypy/branch/fast-forward/lib_pypy/pypy_test/test_md5_extra.py
pypy/branch/fast-forward/lib_pypy/pypy_test/test_sha_extra.py
pypy/branch/fast-forward/pypy/module/_md5/__init__.py
pypy/branch/fast-forward/pypy/module/_md5/interp_md5.py
pypy/branch/fast-forward/pypy/module/_sha/__init__.py
pypy/branch/fast-forward/pypy/module/_sha/interp_sha.py
Log:
Line up md5 and sha modules with CPython.
Should fix a warning in hmac.py
+ fix tests after rename.
Modified: pypy/branch/fast-forward/lib_pypy/_md5.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/_md5.py (original)
+++ pypy/branch/fast-forward/lib_pypy/_md5.py Thu Oct 14 12:05:55 2010
@@ -116,6 +116,7 @@
"An implementation of the MD5 hash function in pure Python."
digest_size = digestsize = 16
+ block_size = 64
def __init__(self):
"Initialisation."
@@ -369,24 +370,10 @@
# ======================================================================
# Mimic Python top-level functions from standard library API
-# for consistency with the md5 module of the standard library.
+# for consistency with the _md5 module of the standard library.
# ======================================================================
-digest_size = digestsize = 16
-blocksize = 1
-
-def new(arg=None):
- """Return a new md5 crypto object.
-
- If arg is present, the method call update(arg) is made.
- """
-
- crypto = MD5Type()
- if arg:
- crypto.update(arg)
-
- return crypto
-
+digest_size = 16
def md5(arg=None):
"""Same as new().
Modified: pypy/branch/fast-forward/lib_pypy/_sha.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/_sha.py (original)
+++ pypy/branch/fast-forward/lib_pypy/_sha.py Thu Oct 14 12:05:55 2010
@@ -118,6 +118,7 @@
"An implementation of the MD5 hash function in pure Python."
digest_size = digestsize = 20
+ block_size = 1
def __init__(self):
"Initialisation."
@@ -328,13 +329,14 @@
# ======================================================================
# Mimic Python top-level functions from standard library API
-# for consistency with the md5 module of the standard library.
+# for consistency with the _sha module of the standard library.
# ======================================================================
# These are mandatory variables in the module. They have constant values
# in the SHA standard.
-digest_size = digestsize = 20
+digest_size = 20
+digestsize = 20
blocksize = 1
def new(arg=None):
Modified: pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/test/test_cache.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/test/test_cache.py (original)
+++ pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/test/test_cache.py Thu Oct 14 12:05:55 2010
@@ -27,10 +27,6 @@
d = run('syslog.ctc.py', '_syslog_cache.py')
assert 'LOG_NOTICE' in d
-def test_hashlib():
- d = run('hashlib.ctc.py', '_hashlib_cache.py')
- assert hasattr(d['EVP_MD_CTX'], 'digest')
-
def test_resource():
d = run('resource.ctc.py', '_resource_cache.py')
assert 'RLIM_NLIMITS' in d
Modified: pypy/branch/fast-forward/lib_pypy/pypy_test/test_md5_extra.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/pypy_test/test_md5_extra.py (original)
+++ pypy/branch/fast-forward/lib_pypy/pypy_test/test_md5_extra.py Thu Oct 14 12:05:55 2010
@@ -6,7 +6,7 @@
from __future__ import absolute_import
import md5 # CPython's implementation in C.
-from .. import md5 as pymd5
+from .. import _md5 as pymd5
# Helpers...
Modified: pypy/branch/fast-forward/lib_pypy/pypy_test/test_sha_extra.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/pypy_test/test_sha_extra.py (original)
+++ pypy/branch/fast-forward/lib_pypy/pypy_test/test_sha_extra.py Thu Oct 14 12:05:55 2010
@@ -4,7 +4,7 @@
# Publication 180-1, Secure Hash Standard, 1995 April 17
# http://www.itl.nist.gov/div897/pubs/fip180-1.htm
from __future__ import absolute_import
-from .. import sha
+from .. import _sha
class TestSHA:
def check(self, data, digest):
Modified: pypy/branch/fast-forward/pypy/module/_md5/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_md5/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/_md5/__init__.py Thu Oct 14 12:05:55 2010
@@ -19,12 +19,9 @@
far using the digest() method."""
interpleveldefs = {
- 'md5': 'interp_md5.W_MD5',
'new': 'interp_md5.W_MD5',
'MD5Type': 'interp_md5.W_MD5',
'digest_size': 'space.wrap(16)',
- 'digestsize': 'space.wrap(16)',
- 'blocksize': 'space.wrap(1)',
}
appleveldefs = {
Modified: pypy/branch/fast-forward/pypy/module/_md5/interp_md5.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_md5/interp_md5.py (original)
+++ pypy/branch/fast-forward/pypy/module/_md5/interp_md5.py Thu Oct 14 12:05:55 2010
@@ -49,6 +49,7 @@
copy = interp2app(W_MD5.copy_w, unwrap_spec=['self']),
digest_size = 16,
digestsize = 16,
+ block_size = 64,
__doc__ = """md5(arg) -> return new md5 object.
If arg is present, the method call update(arg) is made.""")
Modified: pypy/branch/fast-forward/pypy/module/_sha/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_sha/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/_sha/__init__.py Thu Oct 14 12:05:55 2010
@@ -19,7 +19,6 @@
bits instead of MD5's 128 bits."""
interpleveldefs = {
- 'sha': 'interp_sha.W_SHA',
'new': 'interp_sha.W_SHA',
'SHAType': 'interp_sha.W_SHA',
'blocksize': 'space.wrap(1)',
Modified: pypy/branch/fast-forward/pypy/module/_sha/interp_sha.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_sha/interp_sha.py (original)
+++ pypy/branch/fast-forward/pypy/module/_sha/interp_sha.py Thu Oct 14 12:05:55 2010
@@ -49,6 +49,7 @@
copy = interp2app(W_SHA.copy_w, unwrap_spec=['self']),
digest_size = 20,
digestsize = 20,
+ block_size = 64,
__doc__ = """sha(arg) -> return new sha object.
If arg is present, the method call update(arg) is made.""")
From pedronis at codespeak.net Thu Oct 14 12:19:53 2010
From: pedronis at codespeak.net (pedronis at codespeak.net)
Date: Thu, 14 Oct 2010 12:19:53 +0200 (CEST)
Subject: [pypy-svn] r77911 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014101953.BD83A282BE3@codespeak.net>
Author: pedronis
Date: Thu Oct 14 12:19:49 2010
New Revision: 77911
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
the escaping is not needed
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 12:19:49 2010
@@ -467,9 +467,9 @@
\item \lstinline{new} corresponds to object creation.
\item \lstinline{get} correspond to attribute reads.
\item \lstinline{set} correspond to attribute writes.
- \item \lstinline{guard\_class} correspond to method calls and are followed by
+ \item \lstinline{guard_class} correspond to method calls and are followed by
the trace of the called method.
- \item \lstinline{int\_add} and \lstinline{int\_get} are integer addition and
+ \item \lstinline{int_add} and \lstinline{int_get} are integer addition and
comparison (``greater than''), respectively.
\end{itemize}
@@ -484,7 +484,7 @@
using the interpreter.
The trace shows the inefficiencies of \lstinline{f} clearly, if one looks at the
-number of \lstinline{new}, \lstinline{set/get} and \lstinline{guard\_class} operations.
+number of \lstinline{new}, \lstinline{set/get} and \lstinline{guard_class} operations.
In the rest of the paper we will see how this trace can be optimized using
partial evaluation.
From pedronis at codespeak.net Thu Oct 14 12:21:07 2010
From: pedronis at codespeak.net (pedronis at codespeak.net)
Date: Thu, 14 Oct 2010 12:21:07 +0200 (CEST)
Subject: [pypy-svn] r77912 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014102107.00169282BE3@codespeak.net>
Author: pedronis
Date: Thu Oct 14 12:21:06 2010
New Revision: 77912
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
typo+tweak
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 12:21:06 2010
@@ -1110,8 +1110,8 @@
\section{Conclusion}
\label{sec:conclusion}
-In this paper, we used a approach based on online partial evaluation to
-optimize allocations in the traces of a tracing JIT. In this context a
+In this paper, we used an approach based on online partial evaluation to
+optimize away allocations in the traces of a tracing JIT. In this context a
simple approach to partial evaluation gives good results. This is due
to the fact that the tracing JIT itself is responsible for all control
issues, which are usually the hardest part of partial evaluation: the
From pedronis at codespeak.net Thu Oct 14 12:34:01 2010
From: pedronis at codespeak.net (pedronis at codespeak.net)
Date: Thu, 14 Oct 2010 12:34:01 +0200 (CEST)
Subject: [pypy-svn] r77913 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014103401.E92B3282BE3@codespeak.net>
Author: pedronis
Date: Thu Oct 14 12:34:00 2010
New Revision: 77913
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
Log:
regen pdf
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
From afa at codespeak.net Thu Oct 14 12:59:48 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 14 Oct 2010 12:59:48 +0200 (CEST)
Subject: [pypy-svn] r77914 - in
pypy/branch/fast-forward/pypy/module/_multiprocessing: . test
Message-ID: <20101014105948.EF014282BE9@codespeak.net>
Author: afa
Date: Thu Oct 14 12:59:47 2010
New Revision: 77914
Modified:
pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_semaphore.py
pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_semaphore.py
Log:
Add context manager to _multiprocessing.SemLock
Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_semaphore.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_semaphore.py (original)
+++ pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_semaphore.py Thu Oct 14 12:59:47 2010
@@ -1,6 +1,6 @@
from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root
from pypy.interpreter.typedef import TypeDef, GetSetProperty
-from pypy.interpreter.gateway import interp2app, unwrap_spec
+from pypy.interpreter.gateway import interp2app, Arguments, unwrap_spec
from pypy.interpreter.error import wrap_oserror, OperationError
from pypy.rpython.lltypesystem import rffi, lltype
from pypy.rlib.rarithmetic import r_uint
@@ -493,6 +493,14 @@
self.__init__(handle_w(space, w_handle), kind, maxvalue)
return space.wrap(self)
+ @unwrap_spec('self', ObjSpace)
+ def enter(self, space):
+ return self.acquire(space, w_timeout=space.w_None)
+
+ @unwrap_spec('self', ObjSpace, Arguments)
+ def exit(self, space, __args__):
+ self.release(space)
+
@unwrap_spec(ObjSpace, W_Root, int, int, int)
def descr_new(space, w_subtype, kind, value, maxvalue):
if kind != RECURSIVE_MUTEX and kind != SEMAPHORE:
@@ -525,5 +533,7 @@
acquire = interp2app(W_SemLock.acquire),
release = interp2app(W_SemLock.release),
_rebuild = interp2app(W_SemLock.rebuild.im_func, as_classmethod=True),
+ __enter__=interp2app(W_SemLock.enter),
+ __exit__=interp2app(W_SemLock.exit),
SEM_VALUE_MAX=SEM_VALUE_MAX,
)
Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_semaphore.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_semaphore.py (original)
+++ pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_semaphore.py Thu Oct 14 12:59:47 2010
@@ -71,3 +71,14 @@
sem2 = SemLock._rebuild(sem.handle, kind, value)
assert sem.handle == sem2.handle
+
+ def test_semaphore_contextmanager(self):
+ from _multiprocessing import SemLock
+ kind = self.SEMAPHORE
+ value = 1
+ maxvalue = 1
+ sem = SemLock(kind, value, maxvalue)
+
+ with sem:
+ assert sem._count() == 1
+ assert sem._count() == 0
From antocuni at codespeak.net Thu Oct 14 13:34:33 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Thu, 14 Oct 2010 13:34:33 +0200 (CEST)
Subject: [pypy-svn] r77915 - pypy/branch/jitffi/pypy/module/pypyjit/test
Message-ID: <20101014113433.0B8F3282BE9@codespeak.net>
Author: antocuni
Date: Thu Oct 14 13:34:32 2010
New Revision: 77915
Modified:
pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py
Log:
add a test to check that the applevel _ffi call is optimized
Modified: pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py Thu Oct 14 13:34:32 2010
@@ -79,8 +79,11 @@
class PyPyCJITTests(object):
- def run_source(self, source, expected_max_ops, *testcases):
+ def run_source(self, source, expected_max_ops, *testcases, **kwds):
assert isinstance(expected_max_ops, int)
+ threshold = kwds.pop('threshold', 3)
+ if kwds:
+ raise TypeError, 'Unsupported keyword arguments: %s' % kwds.keys()
source = py.code.Source(source)
filepath = self.tmpdir.join('case%d.py' % self.counter)
logfilepath = filepath.new(ext='.log')
@@ -92,7 +95,7 @@
import sys
try: # make the file runnable by CPython
import pypyjit
- pypyjit.set_param(threshold=3)
+ pypyjit.set_param(threshold=%d)
except ImportError:
pass
@@ -102,7 +105,7 @@
print >> sys.stderr, 'got:', repr(result)
assert result == expected
assert type(result) is type(expected)
- """)
+ """ % threshold)
for testcase in testcases * 2:
print >> f, "check(%r, %r)" % testcase
print >> f, "print 'OK :-)'"
@@ -123,6 +126,7 @@
if self.total_ops > expected_max_ops:
assert 0, "too many operations: got %d, expected maximum %d" % (
self.total_ops, expected_max_ops)
+ return result
def parse_loops(self, opslogfile):
from pypy.jit.metainterp.test.oparser import parse
@@ -1130,6 +1134,36 @@
return sa
''', 88, ([], 1997001))
+ def test__ffi_call(self):
+ from pypy.rlib.test.test_libffi import get_libm_name
+ libm_name = get_libm_name(sys.platform)
+ out = self.run_source('''
+ def main():
+ from _ffi import CDLL, types
+ libm = CDLL('%(libm_name)s')
+ pow = libm.getfunc('pow', [types.double, types.double],
+ types.double)
+ print pow.getaddr()
+ i = 0
+ res = 0
+ while i < 2000:
+ res += pow(2, 3)
+ i += 1
+ return res
+ ''' % locals(),
+ 70, ([], 8.0*2000), threshold=1000)
+ pow_addr = int(out.splitlines()[0])
+ ops = self.get_by_bytecode('CALL_FUNCTION')
+ assert len(ops) == 2 # we get two loops, because of specialization
+ op = ops[0]
+ # XXX: there should be a guard_not_forced
+ assert op[-1].getopname() == 'guard_no_exception'
+ call = op[-2]
+ assert call.getopname() == 'call'
+ assert call.getarg(0).value == pow_addr
+ assert call.getarg(1).value == 2.0
+ assert call.getarg(2).value == 3.0
+
# test_circular
class AppTestJIT(PyPyCJITTests):
From afa at codespeak.net Thu Oct 14 13:34:44 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 14 Oct 2010 13:34:44 +0200 (CEST)
Subject: [pypy-svn] r77916 -
pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests
Message-ID: <20101014113444.77ADD282BEA@codespeak.net>
Author: afa
Date: Thu Oct 14 13:34:42 2010
New Revision: 77916
Modified:
pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_array.py
Log:
Typo in a test
Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_array.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_array.py (original)
+++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_array.py Thu Oct 14 13:34:42 2010
@@ -82,11 +82,11 @@
na = numarray(1, 2, 3, 4, 5)
values = [i for i in na]
- assert values, [1, 2, 3, 4 == 5]
+ assert values == [1, 2, 3, 4, 5]
na = numarray(*map(c_int, (1, 2, 3, 4, 5)))
values = [i for i in na]
- assert values, [1, 2, 3, 4 == 5]
+ assert values == [1, 2, 3, 4, 5]
def test_classcache(self):
assert not ARRAY(c_int, 3) is ARRAY(c_int, 4)
From afa at codespeak.net Thu Oct 14 13:43:03 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 14 Oct 2010 13:43:03 +0200 (CEST)
Subject: [pypy-svn] r77917 - in pypy/branch/fast-forward: lib_pypy/_ctypes
pypy/module/test_lib_pypy/ctypes_tests
Message-ID: <20101014114303.5A9E4282BEA@codespeak.net>
Author: afa
Date: Thu Oct 14 13:43:01 2010
New Revision: 77917
Modified:
pypy/branch/fast-forward/lib_pypy/_ctypes/array.py
pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_array.py
Log:
Error in slicing ctypes arrays: a[:9999] would fail.
Test and fix.
Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/array.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/_ctypes/array.py (original)
+++ pypy/branch/fast-forward/lib_pypy/_ctypes/array.py Thu Oct 14 13:43:01 2010
@@ -109,8 +109,18 @@
def array_get_slice_params(self, index):
if index.step is not None:
raise TypeError("3 arg slices not supported (for no reason)")
- start = index.start or 0
- stop = index.stop or self._length_
+ if index.start is not None:
+ start = index.start
+ if start < 0:
+ start = 0
+ else:
+ start = 0
+ if index.stop is not None:
+ stop = index.stop
+ if stop > self._length_:
+ stop = self._length_
+ else:
+ stop = self._length_
return start, stop
def array_slice_setitem(self, index, value):
Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_array.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_array.py (original)
+++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_array.py Thu Oct 14 13:43:01 2010
@@ -88,6 +88,16 @@
values = [i for i in na]
assert values == [1, 2, 3, 4, 5]
+ def test_slice(self):
+ values = range(5)
+ numarray = c_int * 5
+
+ na = numarray(*(c_int(x) for x in values))
+
+ assert list(na[0:0]) == []
+ assert list(na[:]) == values
+ assert list(na[:10]) == values
+
def test_classcache(self):
assert not ARRAY(c_int, 3) is ARRAY(c_int, 4)
assert ARRAY(c_int, 3) is ARRAY(c_int, 3)
From leuschel at codespeak.net Thu Oct 14 13:52:46 2010
From: leuschel at codespeak.net (leuschel at codespeak.net)
Date: Thu, 14 Oct 2010 13:52:46 +0200 (CEST)
Subject: [pypy-svn] r77918 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014115246.71B22282BEA@codespeak.net>
Author: leuschel
Date: Thu Oct 14 13:52:42 2010
New Revision: 77918
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
some changes to sect. 4
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 13:52:42 2010
@@ -560,15 +560,15 @@
The main insight to improve the code shown in the last section is that objects
in category 1 don't survive very long -- they are used only inside the loop and
nobody else in the program stores a reference to them. The idea for improving
-the code is thus to analyze which objects fall in category 1 and may thus
-not be allocated at all.
+the code is thus to analyze which objects fall in category 1 and thus do
+not have to be allocated at all.
This is a process that is usually called \emph{escape analysis}. In this paper we will
perform escape analysis by using partial evaluation. The partial evalution is a
-bit peculiar in that there are not actually any constant arguments to the trace,
-but it is only used to optimized operations within a trace.
+bit peculiar in that it receives no static input arguments for the trace,
+but it is only used to optimize operations within a trace.
-The partial evaluation works by walking the trace from beginning to end.
+The partial evaluation works by traversing the trace from beginning to end.
Whenever a \lstinline{new} operation is seen, the operation is removed and a static
object is constructed and associated with the variable that would have stored
the result of \lstinline{new}. The static object describes the shape of the
@@ -583,8 +583,9 @@
as well, because the shape description stores the type and thus the outcome of
the type check the guard does is statically known.
-In the example from last section, the following operations would produce two
-static objects, and be completely removed from the optimized trace:
+In the example from last section, the following operations in the upper half
+of Fig.~\ref{fig:unopt-trace} would produce two
+static objects, and would thus be completely removed from the optimized trace:
\begin{lstlisting}[mathescape,xleftmargin=20pt]
$p_{5}$ = new(BoxedInteger)
@@ -599,7 +600,8 @@
one associated with $p_{6}$ would know that it is a \lstinline{BoxedInteger}
whose \lstinline{intval} field contains the constant -100.
-The following operations on $p_{5}$ and $p_{6}$ could then be
+The subsequent operations in Fig.~\ref{fig:unopt-trace},
+ which use $p_{5}$ and $p_{6}$, could then be
optimized using that knowledge:
\begin{lstlisting}[mathescape,xleftmargin=20pt]
@@ -612,8 +614,8 @@
$i_{9}$ = int_add($i_{7}$, $i_{8}$)
\end{lstlisting}
-The \lstinline{guard_class} operations can be removed, because the classes of $p_{5}$ and
-$p_{6}$ are known to be \lstinline{BoxedInteger}. The \lstinline{get} operations can be removed
+First, the \lstinline{guard_class} operations can be removed, because the classes of $p_{5}$ and
+$p_{6}$ are known to be \lstinline{BoxedInteger}. Second, the \lstinline{get} operations can be removed
and $i_{7}$ and $i_{8}$ are just replaced by $i_{4}$ and -100. Thus the only
remaining operation in the optimized trace would be:
@@ -621,24 +623,24 @@
$i_{9}$ = int_add($i_{4}$, -100)
\end{lstlisting}
-The rest of the trace is optimized similarly.
+The rest of the trace from Fig.~\ref{fig:unopt-trace} is optimized similarly.
-So far we have only described what happens when static objects are used in
-operations that read and write their fields and in guards. When the static
-object is used in any other operation, it cannot stay static. For example, when
-a static object is stored in a globally accessible place, the object needs to
-actually be allocated, as it might live longer than one iteration of the loop
+So far we have only described what happens when static objects are used in guards and in
+operations that read and write fields. When the static
+object is used in any other operation, it cannot remain static. For example, when
+a static object is stored in a globally accessible place, the object has to
+be allocated, as it might live longer than one iteration of the loop
and because the partial evaluator looses track of it. This means that the static
object needs to be turned into a dynamic one, \ie lifted. This makes it
-necessary to put operations into the residual code that actually allocate the
+necessary to put operations into the residual code that allocate the
static object at runtime.
-This is what happens at the end of the trace in Figure~\ref{fig:unopt-trace}, when the \lstinline{jump} operation
+This is what happens at the end of the trace in Fig.~\ref{fig:unopt-trace}, when the \lstinline{jump} operation
is hit. The arguments of the jump are at this point static objects. Before the
jump is emitted, they are \emph{lifted}. This means that the optimizer produces code
that allocates a new object of the right type and sets its fields to the field
values that the static object has (if the static object points to other static
-objects, those need to be lifted as well, recursively) This means that instead of the jump,
+objects, those need to be lifted as well, recursively). This means that instead of a simple jump,
the following operations are emitted:
\begin{lstlisting}[mathescape,xleftmargin=20pt]
@@ -649,10 +651,11 @@
jump($p_{15}$, $p_{10}$)
\end{lstlisting}
-Note how the operations for creating these two instances have been moved down the
-trace. It looks like for these operations we actually didn't win much, because
-the objects are still allocated at the end. However, the optimization was still
-worthwhile even in this case, because some operations that have been performed
+Observe how the operations for creating these two instances have been moved to later point in the
+trace.
+At first sight, it may look like for these operations we didn't gain much, as
+the objects are still allocated in the end. However, our optimizations were still
+worthwhile, because some operations that have been performed
on the lifted static objects have been removed (some \lstinline{get} operations
and \lstinline{guard_class} operations).
From cfbolz at codespeak.net Thu Oct 14 14:06:34 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 14:06:34 +0200 (CEST)
Subject: [pypy-svn] r77919 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014120634.6A8CF282BEA@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 14:06:32 2010
New Revision: 77919
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
try to have a marginally acceptable author block
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 14:06:32 2010
@@ -1,4 +1,4 @@
-\documentclass{sigplanconf}
+\documentclass[preprint]{sigplanconf}
\usepackage{ifthen}
\usepackage{fancyvrb}
@@ -72,16 +72,23 @@
\begin{tabular}{c}}
%
\begin{document}
-\conferenceinfo{PEPM'11,} {XXX}
-\CopyrightYear{XXX}
-\copyrightdata{XXX}
+\conferenceinfo{PEPM}{'11, Austin, USA}
+\CopyrightYear{2011}
+\copyrightdata{[to be supplied]}
\title{Allocation Removal by Partial Evaluation in a Tracing JIT}
-\authorinfo{Carl Friedrich Bolz \and Antonio Cuni \and Maciej Fija?kowski \and Michael Leuschel \and Samuele Pedroni \and Armin Rigo}
- {Heinrich-Heine-Universit?t D?sseldorf, STUPS Group, Germany XXX}
+\authorinfo{Carl Friedrich Bolz$^a$ \and Antonio Cuni$^a$ \and Maciej Fija?kowski$^b$ \and Michael Leuschel$^a$ \and \\
+ Samuele Pedroni$^c$ \and Armin Rigo$^a$}
+ {$^a$Heinrich-Heine-Universit?t D?sseldorf, STUPS Group, Germany
+
+ $^b$merlinux GmbH, Hildesheim, Germany
+
+ $^c$Open End, G?teborg, Sweden
+ }
{cfbolz at gmx.de, anto.cuni at gmail.com, fijal at merlinux.eu,
leuschel at cs.uni-duesseldorf.de, samuele.pedroni at gmail.com, arigo at tunes.org}
+% {Heinrich-Heine-Universit?t D?sseldorf, STUPS Group, Germany} \and {merlinux GmbH, Hildesheim, Germany} \and {Open End, G?teborg, Sweden}
%\numberofauthors{3}
%\author{
@@ -1133,8 +1140,8 @@
\section*{Acknowledgements}
-The authors would like to thank Stefan Hallerstede and Thomas Stiehl for
-fruitful discussions during the writing of the paper.
+The authors would like to thank Stefan Hallerstede, David Schneider and Thomas
+Stiehl for fruitful discussions during the writing of the paper.
\bibliographystyle{abbrv}
\bibliography{paper}
From cfbolz at codespeak.net Thu Oct 14 14:07:58 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 14:07:58 +0200 (CEST)
Subject: [pypy-svn] r77920 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014120758.7176B282BEA@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 14:07:56 2010
New Revision: 77920
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
better email addresses
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 14:07:56 2010
@@ -86,9 +86,8 @@
$^c$Open End, G?teborg, Sweden
}
- {cfbolz at gmx.de, anto.cuni at gmail.com, fijal at merlinux.eu,
- leuschel at cs.uni-duesseldorf.de, samuele.pedroni at gmail.com, arigo at tunes.org}
-% {Heinrich-Heine-Universit?t D?sseldorf, STUPS Group, Germany} \and {merlinux GmbH, Hildesheim, Germany} \and {Open End, G?teborg, Sweden}
+ {cfbolz at gmx.de \and anto.cuni at gmail.com \and fijal at merlinux.eu,
+ leuschel at cs.uni-duesseldorf.de \and samuele.pedroni at gmail.com \and arigo at tunes.org}
%\numberofauthors{3}
%\author{
From arigo at codespeak.net Thu Oct 14 14:20:37 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 14 Oct 2010 14:20:37 +0200 (CEST)
Subject: [pypy-svn] r77921 - pypy/branch/32ptr-on-64bit/pypy/translator/c/src
Message-ID: <20101014122037.6AC55282BEA@codespeak.net>
Author: arigo
Date: Thu Oct 14 14:20:35 2010
New Revision: 77921
Modified:
pypy/branch/32ptr-on-64bit/pypy/translator/c/src/stack.h
Log:
When compiling on gcc with -O0, we really need more stack.
Thanks antocuni for finding out the proper #ifdefs.
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/src/stack.h
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/src/stack.h (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/src/stack.h Thu Oct 14 14:20:35 2010
@@ -3,7 +3,11 @@
/*** C header subsection: stack operations ***/
#ifndef MAX_STACK_SIZE
+# if defined(__GNUC__) && !defined(__OPTIMIZE__)
+# define MAX_STACK_SIZE (7 << 20) /* 7 MB */
+# else
# define MAX_STACK_SIZE (3 << 18) /* 768 kb */
+# endif
#endif
/* This include must be done in any case to initialise
From cfbolz at codespeak.net Thu Oct 14 14:24:13 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 14:24:13 +0200 (CEST)
Subject: [pypy-svn] r77922 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014122413.7CCC4282BEA@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 14:24:12 2010
New Revision: 77922
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
fix an XXX, add a new one
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 14:24:12 2010
@@ -183,8 +183,7 @@
The core of our trace optimization technique can be
viewed as partial evaluation: the partial evaluation
performs a form of escape analysis \cite{bruno_blanchet_escape_2003} on the traces and make some
-objects that are allocated in the trace \emph{static}\footnote{These objects are
-called \emph{virtual} in Psyco \cite{rigo_representation-based_2004}.} which
+objects that are allocated in the trace \emph{static} which
means that they do not occur any more in the optimized trace. This technique is
informally described in Section~\ref{sec:statics}, a more formal description is
given in Section~\ref{sec:formal}.
@@ -560,9 +559,6 @@
\subsection{Static Objects}
-XXX add a footnote saying that "static" is meant in the sense of PE, not static
-allocation
-
The main insight to improve the code shown in the last section is that objects
in category 1 don't survive very long -- they are used only inside the loop and
nobody else in the program stores a reference to them. The idea for improving
@@ -575,8 +571,11 @@
but it is only used to optimize operations within a trace.
The partial evaluation works by traversing the trace from beginning to end.
-Whenever a \lstinline{new} operation is seen, the operation is removed and a static
-object is constructed and associated with the variable that would have stored
+Whenever a \lstinline{new} operation is seen, the operation is removed and a \empth{static
+object}\footnote{Here ``static'' is meant in the sense of partial
+evaluation, \ie known at partial evaluation time, not in the sense of static
+allocation or static method.} is constructed and associated with the variable
+that would have stored
the result of \lstinline{new}. The static object describes the shape of the
original object, \eg where the values that would be stored in the fields of the
allocated object come from, as well as the type of the object. Whenever the
@@ -1115,6 +1114,9 @@
xxx: relation to compile-time garbage collection \cite{mazur_practical_2001}; separation logic; John Hughes: type specialization
+XXX Psyco \cite{rigo_representation-based_2004}
+
+XXX cite SELF paper for the type propagation effects
\section{Conclusion}
\label{sec:conclusion}
From david at codespeak.net Thu Oct 14 14:39:11 2010
From: david at codespeak.net (david at codespeak.net)
Date: Thu, 14 Oct 2010 14:39:11 +0200 (CEST)
Subject: [pypy-svn] r77923 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101014123911.52F33282BEA@codespeak.net>
Author: david
Date: Thu Oct 14 14:39:09 2010
New Revision: 77923
Added:
pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py
Log:
Add ARM regalloc
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Thu Oct 14 14:39:09 2010
@@ -0,0 +1,34 @@
+from pypy.jit.backend.llsupport.regalloc import FrameManager, \
+ RegisterManager, compute_vars_longevity
+from pypy.jit.backend.arm import registers as r
+
+class ARMRegisterManager(RegisterManager):
+ all_regs = r.all_regs
+ box_types = None # or a list of acceptable types
+ no_lower_byte_regs = r.all_regs
+ save_around_call_regs = all_regs
+
+ def __init__(self, longevity, frame_manager=None, assembler=None):
+ RegisterManager.__init__(self, longevity, frame_manager, assembler)
+
+class ARMFrameManager(FrameManager):
+ @staticmethod
+ def frame_pos(loc, type):
+ pass
+
+#class RegAlloc(object):
+# def __init__(self, assembler, translate_support_code=False):
+# self.assembler = assembler
+# self.translate_support_code = translate_support_code
+# self.fm = None
+#
+# def _prepare(self, inputargs, operations):
+# longevity = compute_vars_longevity(inputargs, operations)
+# self.rm = ARMRegisterManager(longevity, self.fm)
+#
+# def prepare_loop(self, inputargs, operations, looptoken):
+# self._prepare(inputargs, operations)
+#
+# def force_allocate_reg(self, v, forbidden_vars=[], selected_reg=None,
+# need_lower_byte=False):
+# return self.rm.force_allocate_reg(v, forbidden_vars, selected_reg, need_lower_byte)
From fijal at codespeak.net Thu Oct 14 14:39:57 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Thu, 14 Oct 2010 14:39:57 +0200 (CEST)
Subject: [pypy-svn] r77924 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014123957.BA05D5080B@codespeak.net>
Author: fijal
Date: Thu Oct 14 14:39:56 2010
New Revision: 77924
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
Two more comments
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 14:39:56 2010
@@ -931,6 +931,9 @@
because every time an object is stored into a local variable, it is stored into
the frame-object, which makes it escape.
+XXX [fijal] Well, in fact that's *a lot* of support for debugger from the VM.
+The support is just generic instead of specific.
+
This problem is solved by making it possible to the interpreter author to add
some hints into the source code to declare instances of one class as frame
objects. The JIT will then fill these objects only lazily when they are actually
@@ -952,6 +955,9 @@
benchmarks we used are small-to-medium Python programs, some synthetic
benchmarks, some real applications.
+XXX [fijal] note that some of those have cpython-specific hacks removed or
+old versions used (nbody mostly I think).
+
Some of them are from the Computer Language Benchmark
Game\footnote{\texttt{http://shootout.alioth.debian.org/}}: \textbf{fannkuch},
\textbf{nbody}, \textbf{meteor-contest}, \textbf{spectral-norm}.
From cfbolz at codespeak.net Thu Oct 14 14:43:50 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 14:43:50 +0200 (CEST)
Subject: [pypy-svn] r77925 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014124350.7AD0C282BEA@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 14:43:49 2010
New Revision: 77925
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
fix another an XXX and a typo
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 14:43:49 2010
@@ -488,13 +488,17 @@
first \lstinline{guard_class} instruction will fail and execution will continue
using the interpreter.
-The trace shows the inefficiencies of \lstinline{f} clearly, if one looks at the
-number of \lstinline{new}, \lstinline{set/get} and \lstinline{guard_class} operations.
+The trace shows the inefficiencies of \lstinline{f} clearly, if one looks at
+the number of \lstinline{new}, \lstinline{set/get} and \lstinline{guard_class}
+operations. The number of \lstinline{guard_class} operation is particularly
+problematic, not only because of the time it takes to run them. All guards also
+have additional information attached that makes it possible to return to the
+interpreter, should the guard fail. This means that many guard operations also
+lead to a memory problem.
+
In the rest of the paper we will see how this trace can be optimized using
partial evaluation.
-XXX add comment that guards are not only slow, but also use lots of memory
-
\section{Object Lifetimes in a Tracing JIT}
\label{sec:lifetimes}
@@ -571,7 +575,7 @@
but it is only used to optimize operations within a trace.
The partial evaluation works by traversing the trace from beginning to end.
-Whenever a \lstinline{new} operation is seen, the operation is removed and a \empth{static
+Whenever a \lstinline{new} operation is seen, the operation is removed and a \emph{static
object}\footnote{Here ``static'' is meant in the sense of partial
evaluation, \ie known at partial evaluation time, not in the sense of static
allocation or static method.} is constructed and associated with the variable
From david at codespeak.net Thu Oct 14 14:45:47 2010
From: david at codespeak.net (david at codespeak.net)
Date: Thu, 14 Oct 2010 14:45:47 +0200 (CEST)
Subject: [pypy-svn] r77926 - in
pypy/branch/arm-backend/pypy/jit/backend/arm: . test
Message-ID: <20101014124547.7CB4136C231@codespeak.net>
Author: david
Date: Thu Oct 14 14:45:45 2010
New Revision: 77926
Added:
pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
Log:
Refactor parts of the codebuilder, build load and store instructions from a table
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Thu Oct 14 14:45:45 2010
@@ -1,6 +1,7 @@
import conditions as cond
from pypy.rlib.rmmap import alloc
from pypy.rpython.lltypesystem import lltype, rffi
+from pypy.jit.backend.arm.instruction_builder import define_instructions
class ARMv7Builder(object):
@@ -8,21 +9,6 @@
self._data = alloc(1024)
self._pos = 0
- def LDR_ri(self, rt, rn, imm=0, cond=cond.AL):
- # XXX W and P bits are not encoded yet
- p = 1
- w = 0
- u, imm = self._encode_imm(imm)
- self.write32(cond << 28
- | 0x1 << 26
- | (p & 0x1) << 24
- | (u & 0x1) << 23
- | (w & 0x1) << 21
- | 0x1 << 20
- | (rn & 0xF) << 16
- | (rt & 0xF) << 12
- | (imm & 0xFFF))
-
def ADD_ri(self, rt, rn, imm, cond=cond.AL):
# XXX S bit
self.write32(cond << 28
@@ -58,20 +44,6 @@
| (rt & 0xF) << 12
| (imm & 0xFFF))
- def STR_ri(self, rt, rn, imm=0, cond=cond.AL):
- # XXX W and P bits are not encoded yet
- p = 1
- w = 0
- u, imm = self._encode_imm(imm)
- self.write32(cond << 28
- | 0x1 << 26
- | (p & 0x1) << 24
- | (u & 0x1) << 23
- | (w & 0x1) << 21
- | (rn & 0xF) << 16
- | (rt & 0xF) << 12
- | (imm & 0xFFF))
-
def ASR_ri(self, rd, rm, imm=0, cond=cond.AL, s=0):
self.write32(cond << 28
| 0xD << 21
@@ -139,4 +111,4 @@
def curraddr(self):
return self.baseaddr() + self._pos
-
+define_instructions(ARMv7Builder)
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py Thu Oct 14 14:45:45 2010
@@ -0,0 +1,43 @@
+from pypy.jit.backend.arm import conditions as cond
+from pypy.jit.backend.arm import instructions
+
+def define_load_store_func(target, name, table):
+ # XXX W and P bits are not encoded yet
+ n = (0x1 << 26
+ | (table['A'] & 0x1) << 25
+ | (table['op1'] & 0x1F) << 20
+ | (table['B'] & 0x1) << 4)
+ if table['imm']:
+ def f(self, rt, rn, imm=0, cond=cond.AL):
+ p = 1
+ w = 0
+ u, imm = self._encode_imm(imm)
+ self.write32(n
+ | cond << 28
+ | (p & 0x1) << 24
+ | (u & 0x1) << 23
+ | (w & 0x1) << 21
+ | (rn & 0xFF) << 16
+ | (rt & 0xFF) << 12
+ | (imm & 0xFFF))
+ else:
+ def f(self, rt, rn, rm, imm=0, cond=cond.AL, s=0, shifttype=0):
+ p = 1
+ w = 0
+ u, imm = self._encode_imm(imm)
+ self.write32(n
+ | cond << 28
+ | (p & 0x1) << 24
+ | (u & 0x1) << 23
+ | (w & 0x1) << 21
+ | (rn & 0xFF) << 16
+ | (rt & 0xFF) << 12
+ | (imm & 0x1F) << 7
+ | (shifttype & 0x3) << 5
+ | (rm & 0xFF))
+
+ setattr(target, name, f)
+
+def define_instructions(target):
+ for key, val in instructions.load_store.iteritems():
+ define_load_store_func(target, key, val)
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py Thu Oct 14 14:45:45 2010
@@ -0,0 +1,12 @@
+# XXX ensure b value is as expected
+# XXX add not assertions for op1
+load_store = {
+ 'STR_ri': {'A':0, 'op1': 0x0, 'B': 0, 'imm': True},
+ 'STR_rr': {'A':1, 'op1': 0x0, 'B': 0, 'imm': False},
+ 'LDR_ri': {'A':0, 'op1': 0x1, 'B': 0, 'imm': True},
+ 'LDR_rr': {'A':1, 'op1': 0x1, 'B': 0, 'imm': False},
+ 'STRB_ri': {'A':0, 'op1': 0x4, 'B': 0, 'imm': True},
+ 'STRB_rr': {'A':1, 'op1': 0x4, 'B': 0, 'imm': False},
+ 'LDRB_ri': {'A':0, 'op1': 0x5, 'B': 0, 'imm': True},
+ 'LDRB_rr': {'A':1, 'op1': 0x5, 'B': 0, 'imm': False},
+}
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Thu Oct 14 14:45:45 2010
@@ -1,5 +1,6 @@
from pypy.jit.backend.arm import registers as r
from pypy.jit.backend.arm import codebuilder
+from pypy.jit.backend.arm import instructions
from pypy.jit.backend.arm.test.support import requires_arm_as
from gen import assemble
import py
@@ -15,8 +16,11 @@
def hexdump(self):
return ''.join(self.buffer)
+class ASMTest(object):
+ def assert_equal(self, asm):
+ assert self.cb.hexdump() == assemble(asm)
-class TestInstrCodeBuilder(object):
+class TestInstrCodeBuilder(ASMTest):
def setup_method(self, ffuu_method):
self.cb = CodeBuilder()
@@ -110,6 +114,31 @@
self.cb.CMP(r.r3, 123)
self.assert_equal('CMP r3, #123')
- def assert_equal(self, asm):
- assert self.cb.hexdump() == assemble(asm)
+class TestInstrCodeBuilderForGeneratedInstr(ASMTest):
+ def setup_method(self, ffuu_method):
+ self.cb = CodeBuilder()
+
+def build_tests():
+ for key, value in instructions.load_store.iteritems():
+ if value['imm']:
+ f = gen_test_imm_func
+ else:
+ f = gen_test_reg_func
+ test = f(key, value)
+ setattr(TestInstrCodeBuilderForGeneratedInstr, 'test_%s' % key, test)
+
+def gen_test_imm_func(name, table):
+ def f(self):
+ func = getattr(self.cb, name)
+ func(r.r3, r.r7, 23)
+ self.assert_equal('%s r3, [r7, #23]' % name[:name.index('_')])
+ return f
+
+def gen_test_reg_func(name, table):
+ def f(self):
+ func = getattr(self.cb, name)
+ func(r.r3, r.r7, r.r12)
+ self.assert_equal('%s r3, [r7, r12]' % name[:name.index('_')])
+ return f
+build_tests()
From antocuni at codespeak.net Thu Oct 14 14:56:22 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Thu, 14 Oct 2010 14:56:22 +0200 (CEST)
Subject: [pypy-svn] r77927 - pypy/branch/jitffi/pypy/jit/metainterp/test
Message-ID: <20101014125622.DBEAE36C231@codespeak.net>
Author: antocuni
Date: Thu Oct 14 14:56:21 2010
New Revision: 77927
Added:
pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py (contents, props changed)
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
move ffi tests to their own file
Added: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
==============================================================================
--- (empty file)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py Thu Oct 14 14:56:21 2010
@@ -0,0 +1,81 @@
+from pypy.rpython.lltypesystem import llmemory
+from pypy.rlib.libffi import Func, types
+from pypy.jit.metainterp.history import AbstractDescr
+from pypy.jit.codewriter.effectinfo import EffectInfo
+from pypy.jit.metainterp.test.test_optimizeopt import BaseTestOptimizeOpt, LLtypeMixin
+
+class MyCallDescr(AbstractDescr):
+ """
+ Fake calldescr to be used inside the tests.
+
+ The particularity is that it provides an __eq__ method, so that it
+ comparses by value by comparing the arg_types and typeinfo fields, so you
+ can check that the signature of a call is really what you want.
+ """
+
+ def __init__(self, arg_types, typeinfo):
+ self.arg_types = arg_types
+ self.typeinfo = typeinfo # return type
+
+ def __eq__(self, other):
+ return self.arg_types == other.arg_types and self.typeinfo == other.typeinfo
+
+class FakeLLObject(object):
+
+ def __init__(self, **kwds):
+ self.__dict__.update(kwds)
+ self._TYPE = llmemory.GCREF
+
+ def _identityhash(self):
+ return id(self)
+
+
+class TestFfiCall(BaseTestOptimizeOpt, LLtypeMixin):
+
+ class namespace:
+ cpu = LLtypeMixin.cpu
+ FUNC = LLtypeMixin.FUNC
+ int_float__int = MyCallDescr('if', 'i')
+ funcptr = FakeLLObject()
+ func = FakeLLObject(_fake_class=Func,
+ argtypes=[types.sint, types.double],
+ restype=types.sint)
+ #
+ def calldescr(cpu, FUNC, oopspecindex):
+ einfo = EffectInfo([], [], [], oopspecindex=oopspecindex)
+ return cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, einfo)
+ #
+ libffi_prepare = calldescr(cpu, FUNC, EffectInfo.OS_LIBFFI_PREPARE)
+ libffi_push_arg = calldescr(cpu, FUNC, EffectInfo.OS_LIBFFI_PUSH_ARG)
+ libffi_call = calldescr(cpu, FUNC, EffectInfo.OS_LIBFFI_CALL)
+
+ namespace = namespace.__dict__
+
+ def test_ffi_call_opt(self):
+ ops = """
+ [i0, f1]
+ call(0, ConstPtr(func), descr=libffi_prepare)
+ call(0, ConstPtr(func), i0, descr=libffi_push_arg)
+ call(0, ConstPtr(func), f1, descr=libffi_push_arg)
+ i3 = call(0, ConstPtr(func), 12345, descr=libffi_call)
+ jump(i3, f1)
+ """
+ expected = """
+ [i0, f1]
+ i3 = call(12345, i0, f1, descr=int_float__int)
+ jump(i3, f1)
+ """
+ loop = self.optimize_loop(ops, 'Not, Not', expected)
+
+ def test_ffi_call_nonconst(self):
+ ops = """
+ [i0, f1, p2]
+ call(0, p2, descr=libffi_prepare)
+ call(0, p2, i0, descr=libffi_push_arg)
+ call(0, p2, f1, descr=libffi_push_arg)
+ i3 = call(0, p2, 12345, descr=libffi_call)
+ jump(i3, f1, p2)
+ """
+ expected = ops
+ loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
+
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py Thu Oct 14 14:56:21 2010
@@ -4499,87 +4499,6 @@
# XXX str2unicode
-# ------------------------------------------------
-from pypy.rpython.lltypesystem import llmemory
-from pypy.rlib.libffi import Func, types
-from pypy.jit.metainterp.history import AbstractDescr
-from pypy.jit.codewriter.effectinfo import EffectInfo
-
-class MyCallDescr(AbstractDescr):
- """
- Fake calldescr to be used inside the tests.
-
- The particularity is that it provides an __eq__ method, so that it
- comparses by value by comparing the arg_types and typeinfo fields, so you
- can check that the signature of a call is really what you want.
- """
-
- def __init__(self, arg_types, typeinfo):
- self.arg_types = arg_types
- self.typeinfo = typeinfo # return type
-
- def __eq__(self, other):
- return self.arg_types == other.arg_types and self.typeinfo == other.typeinfo
-
-class FakeLLObject(object):
-
- def __init__(self, **kwds):
- self.__dict__.update(kwds)
- self._TYPE = llmemory.GCREF
-
- def _identityhash(self):
- return id(self)
-
-
-class TestFfiCall(BaseTestOptimizeOpt, LLtypeMixin):
-
- class namespace:
- cpu = LLtypeMixin.cpu
- FUNC = LLtypeMixin.FUNC
- int_float__int = MyCallDescr('if', 'i')
- funcptr = FakeLLObject()
- func = FakeLLObject(_fake_class=Func,
- argtypes=[types.sint, types.double],
- restype=types.sint)
- #
- def calldescr(cpu, FUNC, oopspecindex):
- einfo = EffectInfo([], [], [], oopspecindex=oopspecindex)
- return cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, einfo)
- #
- libffi_prepare = calldescr(cpu, FUNC, EffectInfo.OS_LIBFFI_PREPARE)
- libffi_push_arg = calldescr(cpu, FUNC, EffectInfo.OS_LIBFFI_PUSH_ARG)
- libffi_call = calldescr(cpu, FUNC, EffectInfo.OS_LIBFFI_CALL)
-
- namespace = namespace.__dict__
-
- def test_ffi_call_opt(self):
- ops = """
- [i0, f1]
- call(0, ConstPtr(func), descr=libffi_prepare)
- call(0, ConstPtr(func), i0, descr=libffi_push_arg)
- call(0, ConstPtr(func), f1, descr=libffi_push_arg)
- i3 = call(0, ConstPtr(func), 12345, descr=libffi_call)
- jump(i3, f1)
- """
- expected = """
- [i0, f1]
- i3 = call(12345, i0, f1, descr=int_float__int)
- jump(i3, f1)
- """
- loop = self.optimize_loop(ops, 'Not, Not', expected)
-
- def test_ffi_call_nonconst(self):
- ops = """
- [i0, f1, p2]
- call(0, p2, descr=libffi_prepare)
- call(0, p2, i0, descr=libffi_push_arg)
- call(0, p2, f1, descr=libffi_push_arg)
- i3 = call(0, p2, 12345, descr=libffi_call)
- jump(i3, f1, p2)
- """
- expected = ops
- loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
-
##class TestOOtype(OptimizeOptTest, OOtypeMixin):
From arigo at codespeak.net Thu Oct 14 14:58:42 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 14 Oct 2010 14:58:42 +0200 (CEST)
Subject: [pypy-svn] r77928 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014125842.B371E5080B@codespeak.net>
Author: arigo
Date: Thu Oct 14 14:58:41 2010
New Revision: 77928
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
Add the word "modern" here, and a footnote.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 14:58:41 2010
@@ -157,8 +157,11 @@
applicable to the objects at hand when doing a generic operation on them. An
example would be the addition of two objects: The addition needs to check what
the concrete objects that should be added are, and choose the implementation
-that is fitting for them. Type dispatching is a very common operation in a
-dynamic language because no types are known at compile time, so all operations
+that is fitting for them. Type dispatching is a very common operation in
+modern\footnote{For languages in the LISP family, basic arithmetic operations
+are typically not overloaded; even in Smalltalk, type dispatching is much
+simpler than in Python or JavaScript.}
+dynamic languages because no types are known at compile time, so all operations
need it.
A recently popular approach to implementing just-in-time compilers for dynamic
From arigo at codespeak.net Thu Oct 14 15:09:36 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 14 Oct 2010 15:09:36 +0200 (CEST)
Subject: [pypy-svn] r77929 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014130936.02B80282BEB@codespeak.net>
Author: arigo
Date: Thu Oct 14 15:09:34 2010
New Revision: 77929
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
Typos.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 15:09:34 2010
@@ -169,7 +169,7 @@
program and recording its hot spots into linear execution traces, which are then turned into
machine code. One reason for the popularity of tracing JITs is their relative
simplicity. They can often be added to an interpreter and a lot of the
-infrastructure of an interpreter can be reused. They give some important
+infrastructure of the interpreter can be reused. They give some important
optimizations like inlining and constant-folding for free. A tracing JIT always
produces linear pieces of code, which simplifies many optimizations that are usually
hard in a compiler, such as register allocation.
@@ -186,9 +186,9 @@
The core of our trace optimization technique can be
viewed as partial evaluation: the partial evaluation
performs a form of escape analysis \cite{bruno_blanchet_escape_2003} on the traces and make some
-objects that are allocated in the trace \emph{static} which
+objects that are allocated in the trace \emph{static,} which
means that they do not occur any more in the optimized trace. This technique is
-informally described in Section~\ref{sec:statics}, a more formal description is
+informally described in Section~\ref{sec:statics}; a more formal description is
given in Section~\ref{sec:formal}.
In Section~\ref{sec:support} we describe some supporting techniques that are not
@@ -997,11 +997,12 @@
2.6.35. We compared the performance of various Python implementations on the
benchmarks. As a baseline, we used the standard Python implementation in C,
CPython 2.6.6\footnote{\texttt{http://python.org}}, which uses a bytecode-based
-interpreter. Furthermore we compared against Psyco 1.6
-\cite{rigo_representation-based_2004}, an extension to CPython which is a
+interpreter. Furthermore we compared against
+Psyco\cite{rigo_representation-based_2004} 1.6,
+an extension to CPython which is a
just-in-time compiler that produces machine code at run-time. It is not based
on traces. Finally, we used two versions of PyPy's Python interpreter (revision
-77823 of SVN trunk\footnote{\texttt{https://codespeak.net/svn/pypy/trunk}}): one
+77823 of SVN trunk\footnote{\texttt{http://codespeak.net/svn/pypy/trunk}}): one
including the JIT but not optimizing the traces, and one using the allocation
removal optimizations (as well as some minor other optimizations, such as
constant folding).
@@ -1063,7 +1064,7 @@
\begin{center}
\begin{tabular}{|l||r|r|r|r|}
\hline
- &Cpython &Psyco &PyPy w/o optimizations &PyPy w/ optimizations \\
+ &CPython &Psyco &PyPy w/o optimizations &PyPy w/ optimizations \\
\hline
crypto\_pyaes &2757.80 $\pm$ 0.98 &67.90 $\pm$ 0.47 &1652.00 $\pm$ 4.00 &266.86 $\pm$ 5.94 \\
django &993.19 $\pm$ 0.50 &913.51 $\pm$ 4.22 &694.73 $\pm$ 2.86 &259.53 $\pm$ 1.79 \\
From arigo at codespeak.net Thu Oct 14 15:22:01 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 14 Oct 2010 15:22:01 +0200 (CEST)
Subject: [pypy-svn] r77930 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014132201.8608836C231@codespeak.net>
Author: arigo
Date: Thu Oct 14 15:22:00 2010
New Revision: 77930
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
A short paragraph about Psyco in Related Work.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 15:22:00 2010
@@ -1117,6 +1117,12 @@
\cite{michael_bebenita_spur:_2010}. However, no details for the approach and its
implementation are given.
+Psyco \cite{rigo_representation-based_2004} is a (non-tracing) JIT for Python
+that fully implements allocation removal (the \emph{static} objects are called
+\emph{virtual} objects). It is a hand-written extension module for CPython.
+Historically, PyPy's JIT can be seen as some successor of Psyco for a
+general context.
+
partial evaluation:
Prolog:
@@ -1128,8 +1134,6 @@
xxx: relation to compile-time garbage collection \cite{mazur_practical_2001}; separation logic; John Hughes: type specialization
-XXX Psyco \cite{rigo_representation-based_2004}
-
XXX cite SELF paper for the type propagation effects
\section{Conclusion}
From antocuni at codespeak.net Thu Oct 14 15:23:12 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Thu, 14 Oct 2010 15:23:12 +0200 (CEST)
Subject: [pypy-svn] r77931 - in pypy/branch/jitffi/pypy: jit/codewriter
jit/metainterp/optimizeopt jit/metainterp/test rlib/test
Message-ID: <20101014132312.F3339282BEA@codespeak.net>
Author: antocuni
Date: Thu Oct 14 15:23:11 2010
New Revision: 77931
Modified:
pypy/branch/jitffi/pypy/jit/codewriter/jtransform.py
pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
Log:
attach the correct effectinfo to the libffi oopspec, and adapt the optimizer
and the tests: now the ffi call is done through CALL_MAY_FORCE, not simply CALL.
It might force in case of callbacks
Modified: pypy/branch/jitffi/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/branch/jitffi/pypy/jit/codewriter/jtransform.py Thu Oct 14 15:23:11 2010
@@ -1030,8 +1030,10 @@
# ----------
# Strings and Unicodes.
- def _handle_oopspec_call(self, op, args, oopspecindex):
+ def _handle_oopspec_call(self, op, args, oopspecindex, extraeffect=None):
calldescr = self.callcontrol.getcalldescr(op, oopspecindex)
+ if extraeffect:
+ calldescr.get_extra_info().extraeffect = extraeffect
if isinstance(op.args[0].value, str):
pass # for tests only
else:
@@ -1129,13 +1131,16 @@
def _handle_libffi_call(self, op, oopspec_name, args):
if oopspec_name == 'libffi_prepare_call':
oopspecindex = EffectInfo.OS_LIBFFI_PREPARE
+ extraeffect = EffectInfo.EF_CANNOT_RAISE
elif oopspec_name.startswith('libffi_push_'):
oopspecindex = EffectInfo.OS_LIBFFI_PUSH_ARG
+ extraeffect = EffectInfo.EF_CANNOT_RAISE
elif oopspec_name.startswith('libffi_call_'):
oopspecindex = EffectInfo.OS_LIBFFI_CALL
+ extraeffect = EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE
else:
assert False, 'unsupported oopspec: %s' % oopspec_name
- return self._handle_oopspec_call(op, args, oopspecindex)
+ return self._handle_oopspec_call(op, args, oopspecindex, extraeffect)
def rewrite_op_jit_force_virtual(self, op):
return self._do_builtin_call(op)
Modified: pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py Thu Oct 14 15:23:11 2010
@@ -88,6 +88,8 @@
# normal case
self.emit_operation(op)
+ optimize_CALL_MAY_FORCE = optimize_CALL
+
def _get_funcval(self, op):
funcval = self.getvalue(op.getarg(1))
if not funcval.is_constant():
@@ -113,7 +115,7 @@
for push_op in info.opargs:
argval = self.getvalue(push_op.getarg(2))
arglist.append(argval.force_box())
- newop = ResOperation(rop.CALL, arglist, op.result, descr=info.descr)
+ newop = ResOperation(rop.CALL_MAY_FORCE, arglist, op.result, descr=info.descr)
del self.func_infos[funcval]
return newop
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py Thu Oct 14 15:23:11 2010
@@ -57,12 +57,16 @@
call(0, ConstPtr(func), descr=libffi_prepare)
call(0, ConstPtr(func), i0, descr=libffi_push_arg)
call(0, ConstPtr(func), f1, descr=libffi_push_arg)
- i3 = call(0, ConstPtr(func), 12345, descr=libffi_call)
+ i3 = call_may_force(0, ConstPtr(func), 12345, descr=libffi_call)
+ guard_not_forced() []
+ guard_no_exception() []
jump(i3, f1)
"""
expected = """
[i0, f1]
- i3 = call(12345, i0, f1, descr=int_float__int)
+ i3 = call_may_force(12345, i0, f1, descr=int_float__int)
+ guard_not_forced() []
+ guard_no_exception() []
jump(i3, f1)
"""
loop = self.optimize_loop(ops, 'Not, Not', expected)
Modified: pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_libffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_libffi.py Thu Oct 14 15:23:11 2010
@@ -133,8 +133,9 @@
res = self.call(func, [38, 4.2], rffi.LONG)
assert res == 42
self.check_loops({
- 'call': 1,
+ 'call_may_force': 1,
'guard_no_exception': 1,
+ 'guard_not_forced': 1,
'int_add': 1,
'int_lt': 1,
'guard_true': 1,
@@ -145,7 +146,7 @@
func = (libm, 'pow', [types.double, types.double], types.double)
res = self.call(func, [2.0, 3.0], rffi.DOUBLE, init_result=0.0)
assert res == 8.0
- self.check_loops(call=1)
+ self.check_loops(call_may_force=1, guard_no_exception=1, guard_not_forced=1)
def test_cast_result(self):
"""
@@ -158,7 +159,7 @@
func = (libfoo, 'cast_to_uchar_and_ovf', [types.sint], types.uchar)
res = self.call(func, [0], rffi.UCHAR)
assert res == 200
- self.check_loops(call=1)
+ self.check_loops(call_may_force=1, guard_no_exception=1, guard_not_forced=1)
def test_cast_argument(self):
"""
From pedronis at codespeak.net Thu Oct 14 15:27:08 2010
From: pedronis at codespeak.net (pedronis at codespeak.net)
Date: Thu, 14 Oct 2010 15:27:08 +0200 (CEST)
Subject: [pypy-svn] r77932 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014132708.F302E5080B@codespeak.net>
Author: pedronis
Date: Thu Oct 14 15:27:07 2010
New Revision: 77932
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
expand for clarification
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 15:27:07 2010
@@ -928,27 +928,30 @@
\cfbolz{probably can be cut in case of space problems}
-One problem to the successful application of the allocation removal techniques
-described in the previous sections is the presence of frame-introspection
-features in many dynamic languages. Languages such as Python and Smalltalk allow
-the programmer to get access to the frames object that the interpreter uses to
-store local variables. This is a useful feature, as makes the implementation of
-a debugger possible in Python without needing much support from the VM level. On
-the other hand, it severely hinders the effectiveness of allocation removal,
-because every time an object is stored into a local variable, it is stored into
-the frame-object, which makes it escape.
-
-XXX [fijal] Well, in fact that's *a lot* of support for debugger from the VM.
-The support is just generic instead of specific.
-
-This problem is solved by making it possible to the interpreter author to add
-some hints into the source code to declare instances of one class as frame
-objects. The JIT will then fill these objects only lazily when they are actually
-accessed (\eg because a debugger is used). Therefore in the common case, nothing
-is stored into the frame objects, making the problem of too much escaping go
-away. This is a common approach in VM implementations \cite{miranda_context_1999}, the only
-novelty in our approach lays in its generality, because most other JITs are just
-specifically written for one particular language.
+One problem to the successful application of the allocation removal
+techniques described in the previous sections is the presence of
+frame-introspection features in many dynamic languages. Languages such
+as Python and Smalltalk allow the programmer to get access to the
+frames object that the interpreter uses to store local variables. This
+is a useful feature, as makes the implementation of a debugger
+possible in Python without needing much explicit support from the VM
+level. On the other hand, it severely hinders the effectiveness of
+allocation removal, because every time an object is stored into a
+local variable, it is stored into the frame-object, which makes it
+escape.
+
+This problem is solved by making it possible to the interpreter author
+to add some hints into the source code to declare instances of one
+class as frame objects needing special treatment. The JIT will then
+fill these objects only lazily when they are actually accessed (\eg
+because a debugger is used) using values from JIT-emitted code runtime
+locations (typically the CPU stack). Therefore in the common case,
+nothing is stored into the frame objects, making the problem of too
+much escaping go away. This is a common approach in VM implementations
+\cite{miranda_context_1999}, the only novelty in our approach lays in
+its generality, all the delicate support code for this is generated, because
+most other JITs are just specifically written for one particular
+language.
% subsection Virtualizables (end)
From antocuni at codespeak.net Thu Oct 14 15:32:23 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Thu, 14 Oct 2010 15:32:23 +0200 (CEST)
Subject: [pypy-svn] r77933 - pypy/branch/jitffi/pypy/jit/metainterp/test
Message-ID: <20101014133223.A97D9282BEA@codespeak.net>
Author: antocuni
Date: Thu Oct 14 15:32:22 2010
New Revision: 77933
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
Log:
update this test too
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py Thu Oct 14 15:32:22 2010
@@ -77,7 +77,9 @@
call(0, p2, descr=libffi_prepare)
call(0, p2, i0, descr=libffi_push_arg)
call(0, p2, f1, descr=libffi_push_arg)
- i3 = call(0, p2, 12345, descr=libffi_call)
+ i3 = call_may_force(0, p2, 12345, descr=libffi_call)
+ guard_not_forced() []
+ guard_no_exception() []
jump(i3, f1, p2)
"""
expected = ops
From cfbolz at codespeak.net Thu Oct 14 15:37:30 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 15:37:30 +0200 (CEST)
Subject: [pypy-svn] r77934 - in pypy/extradoc/talk/pepm2011: . benchmarks
Message-ID: <20101014133730.6995336E076@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 15:37:28 2010
New Revision: 77934
Modified:
pypy/extradoc/talk/pepm2011/benchmarks/bench.gnumeric
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
(cfbolz, arigo for some of it):
- add URLs for PyPy
- add factors to the timing table, to read it more easily
- change psyco description
- some fixes in the virtualizables section
Modified: pypy/extradoc/talk/pepm2011/benchmarks/bench.gnumeric
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 15:37:28 2010
@@ -211,7 +211,7 @@
\subsection{PyPy}
\label{sub:PyPy}
-The work described in this paper was done in the context of the PyPy project
+The work described in this paper was done in the context of the PyPy project\footnote{\texttt{http://pypy.org}}
\cite{armin_rigo_pypys_2006}. PyPy is an environment where dynamic languages can
be implemented in a simple yet efficient way.
When implementing a language with PyPy one writes an \emph{interpreter}
@@ -933,7 +933,7 @@
frame-introspection features in many dynamic languages. Languages such
as Python and Smalltalk allow the programmer to get access to the
frames object that the interpreter uses to store local variables. This
-is a useful feature, as makes the implementation of a debugger
+is a useful feature, as it makes the implementation of a debugger
possible in Python without needing much explicit support from the VM
level. On the other hand, it severely hinders the effectiveness of
allocation removal, because every time an object is stored into a
@@ -947,12 +947,14 @@
because a debugger is used) using values from JIT-emitted code runtime
locations (typically the CPU stack). Therefore in the common case,
nothing is stored into the frame objects, making the problem of too
-much escaping go away. This is a common approach in VM implementations
-\cite{miranda_context_1999}, the only novelty in our approach lays in
-its generality, all the delicate support code for this is generated, because
-most other JITs are just specifically written for one particular
+much escaping go away. This special handling of frame objects is a common approach in VM implementations
+\cite{miranda_context_1999}. The only novelty in our approach lays in
+its generality: all the delicate support code for this is generated, as opposed to
+most other JITs, which are just specifically written for one particular
language.
+XXX one of the JS tracing JITs does this as well
+
% subsection Virtualizables (end)
% section Supporting Techniques (end)
@@ -963,7 +965,10 @@
To evaluate the effectiveness of our allocation removal algorithm, we look at
the effectiveness when used in the tracing JIT of PyPy's Python interpreter. The
benchmarks we used are small-to-medium Python programs, some synthetic
-benchmarks, some real applications.
+benchmarks, some real applications.\footnote{All the source code of the
+benchmarks can be found at \texttt{http://codespeak.net/svn/pypy/benchmarks/}.
+There is also a website that monitors PyPy's performance nightly at
+\texttt{http://speed.pypy.org/}.}
XXX [fijal] note that some of those have cpython-specific hacks removed or
old versions used (nbody mostly I think).
@@ -991,6 +996,7 @@
framework\footnote{\texttt{http://twistedmatrix.com/}}.
\end{itemize}
+
We evaluate the allocation removal algorithm along two lines: first we want to
know how many allocations could be optimized away. On the other hand, we want
to know how much the run times of the benchmarks is improved.
@@ -1025,7 +1031,7 @@
\begin{center}
\begin{tabular}{|l||r|rr|rr|rr|rr|}
\hline
- &num loops &new &removed &get/set &removed &guard &removed &all ops &removed\\
+ &num loops &new &removed &get/set &removed &guard &removed &all ops &removed\\
\hline
crypto\_pyaes &78 &3088 &50\% &57148 &25\% &9055 &95\% &137189 &80\%\\
django &51 &673 &54\% &19318 &18\% &3876 &93\% &55682 &85\%\\
@@ -1061,32 +1067,30 @@
All benchmarks are improved by the allocation removal optimization, some as much
as XXX. XXX Psyco
-XXX runtimes of the algorithm somehow?
-
\begin{figure*}
\begin{center}
-\begin{tabular}{|l||r|r|r|r|}
+\begin{tabular}{|l||r|r||r|r||r|r||r|r|}
\hline
- &CPython &Psyco &PyPy w/o optimizations &PyPy w/ optimizations \\
+ &CPython [ms] & $\times$ &Psyco [ms] & $\times$ &PyPy w/o optim. [ms] & $\times$ &PyPy w/ optim. [ms]& $\times$ \\
\hline
-crypto\_pyaes &2757.80 $\pm$ 0.98 &67.90 $\pm$ 0.47 &1652.00 $\pm$ 4.00 &266.86 $\pm$ 5.94 \\
-django &993.19 $\pm$ 0.50 &913.51 $\pm$ 4.22 &694.73 $\pm$ 2.86 &259.53 $\pm$ 1.79 \\
-fannkuch &1987.22 $\pm$ 2.02 &944.44 $\pm$ 0.61 &566.99 $\pm$ 1.06 &466.87 $\pm$ 1.85 \\
-go &947.21 $\pm$ 1.58 &445.96 $\pm$ 0.68 &2197.71 $\pm$ 25.21 &316.15 $\pm$ 9.33 \\
-html5lib &13987.12 $\pm$ 19.51 &17398.25 $\pm$ 36.50 &27194.45 $\pm$ 46.62 &10092.19 $\pm$ 23.50 \\
-meteor-contest &346.98 $\pm$ 0.35 &215.66 $\pm$ 0.23 &433.04 $\pm$ 1.45 &392.85 $\pm$ 0.87 \\
-nbody\_modified &637.90 $\pm$ 1.82 &256.78 $\pm$ 0.18 &135.55 $\pm$ 0.33 &103.93 $\pm$ 0.25 \\
-pyflate-fast &3169.35 $\pm$ 1.89 &1278.16 $\pm$ 3.13 &3285.89 $\pm$ 8.51 &1822.36 $\pm$ 11.52 \\
-raytrace-simple &2744.60 $\pm$ 51.72 &1072.66 $\pm$ 1.08 &2778.27 $\pm$ 15.13 &647.24 $\pm$ 5.44 \\
-richards &354.06 $\pm$ 1.00 &63.48 $\pm$ 0.15 &383.93 $\pm$ 3.28 &88.32 $\pm$ 0.91 \\
-spambayes &299.16 $\pm$ 0.35 &338.68 $\pm$ 3.14 &580.90 $\pm$ 24.68 &397.37 $\pm$ 10.60 \\
-spectral-norm &478.63 $\pm$ 0.80 &139.83 $\pm$ 1.54 &353.51 $\pm$ 1.39 &112.10 $\pm$ 1.17 \\
-telco &1207.67 $\pm$ 2.03 &730.00 $\pm$ 2.66 &1296.08 $\pm$ 4.37 &495.23 $\pm$ 2.14 \\
-twisted\_names &9.58 $\pm$ 0.01 &10.43 $\pm$ 0.01 &17.99 $\pm$ 0.27 &7.13 $\pm$ 0.09 \\
+crypto\_pyaes &2757.80 $\pm$ 0.98 &10.33 &67.90 $\pm$ 0.47 &0.25 &1652.00 $\pm$ 4.00 &6.19 &266.86 $\pm$ 5.94 &1.00\\
+django &993.19 $\pm$ 0.50 &3.83 &913.51 $\pm$ 4.22 &3.52 &694.73 $\pm$ 2.86 &2.68 &259.53 $\pm$ 1.79 &1.00\\
+fannkuch &1987.22 $\pm$ 2.02 &4.26 &944.44 $\pm$ 0.61 &2.02 &566.99 $\pm$ 1.06 &1.21 &466.87 $\pm$ 1.85 &1.00\\
+go &947.21 $\pm$ 1.58 &3.00 &445.96 $\pm$ 0.68 &1.41 &2197.71 $\pm$ 25.21 &6.95 &316.15 $\pm$ 9.33 &1.00\\
+html5lib &13987.12 $\pm$ 19.51 &1.39 &17398.25 $\pm$ 36.50 &1.72 &27194.45 $\pm$ 46.62 &2.69 &10092.19 $\pm$ 23.50 &1.00\\
+meteor-contest &346.98 $\pm$ 0.35 &0.88 &215.66 $\pm$ 0.23 &0.55 &433.04 $\pm$ 1.45 &1.10 &392.85 $\pm$ 0.87 &1.00\\
+nbody\_modified &637.90 $\pm$ 1.82 &6.14 &256.78 $\pm$ 0.18 &2.47 &135.55 $\pm$ 0.33 &1.30 &103.93 $\pm$ 0.25 &1.00\\
+pyflate-fast &3169.35 $\pm$ 1.89 &1.74 &1278.16 $\pm$ 3.13 &0.70 &3285.89 $\pm$ 8.51 &1.80 &1822.36 $\pm$ 11.52 &1.00\\
+raytrace-simple &2744.60 $\pm$ 51.72 &4.24 &1072.66 $\pm$ 1.08 &1.66 &2778.27 $\pm$ 15.13 &4.29 &647.24 $\pm$ 5.44 &1.00\\
+richards &354.06 $\pm$ 1.00 &4.01 &63.48 $\pm$ 0.15 &0.72 &383.93 $\pm$ 3.28 &4.35 &88.32 $\pm$ 0.91 &1.00\\
+spambayes &299.16 $\pm$ 0.35 &0.75 &338.68 $\pm$ 3.14 &0.85 &580.90 $\pm$ 24.68 &1.46 &397.37 $\pm$ 10.60 &1.00\\
+spectral-norm &478.63 $\pm$ 0.80 &4.27 &139.83 $\pm$ 1.54 &1.25 &353.51 $\pm$ 1.39 &3.15 &112.10 $\pm$ 1.17 &1.00\\
+telco &1207.67 $\pm$ 2.03 &2.44 &730.00 $\pm$ 2.66 &1.47 &1296.08 $\pm$ 4.37 &2.62 &495.23 $\pm$ 2.14 &1.00\\
+twisted\_names &9.58 $\pm$ 0.01 &1.34 &10.43 $\pm$ 0.01 &1.46 &17.99 $\pm$ 0.27 &2.52 &7.13 $\pm$ 0.09 &1.00\\
\hline
\end{tabular}
\end{center}
-\caption{Benchmark Times in Milliseconds}
+\caption{Benchmark Times in Milliseconds, Together With Factor Over PyPy With Optimizations}
\label{fig:times}
\end{figure*}
@@ -1121,10 +1125,11 @@
implementation are given.
Psyco \cite{rigo_representation-based_2004} is a (non-tracing) JIT for Python
-that fully implements allocation removal (the \emph{static} objects are called
-\emph{virtual} objects). It is a hand-written extension module for CPython.
-Historically, PyPy's JIT can be seen as some successor of Psyco for a
-general context.
+that implements a more ad-hoc version of the allocation removal described here.
+Our static objects could be related to what are called \emph{virtual} objects
+in Psyco. It is a hand-written extension module for CPython. Historically,
+PyPy's JIT can be seen as some successor of Psyco for a general context (one of
+the authors of this paper is the author of Psyco).
partial evaluation:
From antocuni at codespeak.net Thu Oct 14 15:47:30 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Thu, 14 Oct 2010 15:47:30 +0200 (CEST)
Subject: [pypy-svn] r77935 - pypy/branch/jitffi/pypy/jit/metainterp/test
Message-ID: <20101014134730.E4FA3282BEC@codespeak.net>
Author: antocuni
Date: Thu Oct 14 15:47:29 2010
New Revision: 77935
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
try to use the full terminal width when displaying the oplists side-by-side
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py Thu Oct 14 15:47:29 2010
@@ -132,14 +132,20 @@
# ____________________________________________________________
def equaloplists(oplist1, oplist2, strict_fail_args=True, remap={}):
- print '-'*20, 'Comparing lists', '-'*20
+ # try to use the full width of the terminal to display the list
+ # unfortunately, does not work with the default capture method of py.test
+ # (which is fd), you you need to use either -s or --capture=sys, else you
+ # get the standard 80 columns width
+ totwidth = py.io.get_terminal_width()
+ width = totwidth / 2 - 1
+ print ' Comparing lists '.center(totwidth, '-')
for op1, op2 in zip(oplist1, oplist2):
txt1 = str(op1)
txt2 = str(op2)
while txt1 or txt2:
- print '%-39s| %s' % (txt1[:39], txt2[:39])
- txt1 = txt1[39:]
- txt2 = txt2[39:]
+ print '%s| %s' % (txt1[:width].ljust(width), txt2[:width])
+ txt1 = txt1[width:]
+ txt2 = txt2[width:]
assert op1.getopnum() == op2.getopnum()
assert op1.numargs() == op2.numargs()
for i in range(op1.numargs()):
From pedronis at codespeak.net Thu Oct 14 15:56:33 2010
From: pedronis at codespeak.net (pedronis at codespeak.net)
Date: Thu, 14 Oct 2010 15:56:33 +0200 (CEST)
Subject: [pypy-svn] r77936 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014135633.A408636E076@codespeak.net>
Author: pedronis
Date: Thu Oct 14 15:56:31 2010
New Revision: 77936
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
clarifications
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 15:56:31 2010
@@ -930,15 +930,15 @@
One problem to the successful application of the allocation removal
techniques described in the previous sections is the presence of
-frame-introspection features in many dynamic languages. Languages such
-as Python and Smalltalk allow the programmer to get access to the
-frames object that the interpreter uses to store local variables. This
-is a useful feature, as it makes the implementation of a debugger
-possible in Python without needing much explicit support from the VM
-level. On the other hand, it severely hinders the effectiveness of
-allocation removal, because every time an object is stored into a
-local variable, it is stored into the frame-object, which makes it
-escape.
+frame-introspection features in many dynamic languages and the fact
+that frames are heap allocated. Languages such as Python and Smalltalk
+allow the programmer to get access to the frames object that the
+interpreter uses to store local variables. This is a useful feature,
+as it makes the implementation of a debugger possible in Python
+without needing much explicit support from the VM level. On the other
+hand, it severely hinders the effectiveness of allocation removal,
+because every time an object is stored into a local variable, it is
+stored into the frame-object, which makes it escape.
This problem is solved by making it possible to the interpreter author
to add some hints into the source code to declare instances of one
@@ -1147,14 +1147,15 @@
\section{Conclusion}
\label{sec:conclusion}
-In this paper, we used an approach based on online partial evaluation to
-optimize away allocations in the traces of a tracing JIT. In this context a
-simple approach to partial evaluation gives good results. This is due
-to the fact that the tracing JIT itself is responsible for all control
-issues, which are usually the hardest part of partial evaluation: the
-tracing JIT selects the parts of the program that are worthwhile to
-optimize, and extracts linear paths through them, inlining functions as
-necessary. What is left to optimize is only those linear paths.
+In this paper, we used an approach based on online partial evaluation
+to optimize away allocations and type guards in the traces of a
+tracing JIT. In this context a simple approach to partial evaluation
+gives good results. This is due to the fact that the tracing JIT
+itself is responsible for all control issues, which are usually the
+hardest part of partial evaluation: the tracing JIT selects the parts
+of the program that are worthwhile to optimize, and extracts linear
+paths through them, inlining functions as necessary. What is left to
+optimize is only those linear paths.
We expect a similar result for other optimizations that usually require
a complex analysis phase and are thus normally too slow to use at
From cfbolz at codespeak.net Thu Oct 14 16:06:18 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 16:06:18 +0200 (CEST)
Subject: [pypy-svn] r77937 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014140618.6E577282BEA@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 16:06:14 2010
New Revision: 77937
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
fix XXX, rename section
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 16:06:14 2010
@@ -86,7 +86,7 @@
$^c$Open End, G?teborg, Sweden
}
- {cfbolz at gmx.de \and anto.cuni at gmail.com \and fijal at merlinux.eu,
+ {cfbolz at gmx.de \and anto.cuni at gmail.com \and fijal at merlinux.eu \and
leuschel at cs.uni-duesseldorf.de \and samuele.pedroni at gmail.com \and arigo at tunes.org}
%\numberofauthors{3}
@@ -191,8 +191,8 @@
informally described in Section~\ref{sec:statics}; a more formal description is
given in Section~\ref{sec:formal}.
-In Section~\ref{sec:support} we describe some supporting techniques that are not
-central to the approach, but are needed to improve the results. The introduced
+In Section~\ref{sec:frames} we describe some supporting techniques that are not
+central to the approach, but are essential to improve the results. The introduced
techniques are evaluated in Section~\ref{sec:Evaluation} using PyPy's Python
interpreter as a case study.
@@ -920,11 +920,8 @@
% section Escape Analysis in a Tracing JIT (end)
-\section{Supporting Techniques}
-\label{sec:support}
-
-\subsection{Virtualizables}
-\label{sub:Virtualizables}
+\section{Taming Frame Objects}
+\label{sec:frames}
\cfbolz{probably can be cut in case of space problems}
@@ -955,9 +952,7 @@
XXX one of the JS tracing JITs does this as well
-% subsection Virtualizables (end)
-
-% section Supporting Techniques (end)
+% section Taming Frame Objects (end)
\section{Evaluation}
\label{sec:Evaluation}
@@ -1060,12 +1055,18 @@
All benchmarks were run 50 times in the same process, to give the JIT time to produce machine
code. The arithmetic mean of the times of the last 30 runs were used as the
result. The errors were computed using a confidence interval with a 95\%
-confidence level \cite{georges_statistically_2007}. The results are reported in Figure~\ref{fig:times}.
+confidence level \cite{georges_statistically_2007}. The results are reported in
+Figure~\ref{fig:times}. In addition to the run times the table also reports the
+speedup that PyPy with optimization turned on achieves.
With the optimization turned on, PyPy's Python interpreter outperforms CPython
in all benchmarks except spambayes (which heavily relies on regular expression
performance and thus is not helped much by our Python JIT) and meteor-contest.
-All benchmarks are improved by the allocation removal optimization, some as much
-as XXX. XXX Psyco
+All benchmarks are improved by the allocation removal optimization, by at least
+20\% and by as much as a factor of 6.95.
+
+Psyco is able to outperform PyPy's JIT in five out of 14 benchmarks. We hope to
+overtake Psyco (which is no longer being actively developped) by adding some
+further optimizations.
\begin{figure*}
\begin{center}
From cfbolz at codespeak.net Thu Oct 14 16:07:07 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 16:07:07 +0200 (CEST)
Subject: [pypy-svn] r77938 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014140707.9F9E0282BEC@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 16:07:06 2010
New Revision: 77938
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
I think now that there is the explicit link to the source code of the
benchmarks, this XXX can go
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 16:07:06 2010
@@ -965,9 +965,6 @@
There is also a website that monitors PyPy's performance nightly at
\texttt{http://speed.pypy.org/}.}
-XXX [fijal] note that some of those have cpython-specific hacks removed or
-old versions used (nbody mostly I think).
-
Some of them are from the Computer Language Benchmark
Game\footnote{\texttt{http://shootout.alioth.debian.org/}}: \textbf{fannkuch},
\textbf{nbody}, \textbf{meteor-contest}, \textbf{spectral-norm}.
From cfbolz at codespeak.net Thu Oct 14 16:10:54 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 16:10:54 +0200 (CEST)
Subject: [pypy-svn] r77939 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014141054.CBD65282BEB@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 16:10:53 2010
New Revision: 77939
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
fix another XXX
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 16:10:53 2010
@@ -945,13 +945,11 @@
locations (typically the CPU stack). Therefore in the common case,
nothing is stored into the frame objects, making the problem of too
much escaping go away. This special handling of frame objects is a common approach in VM implementations
-\cite{miranda_context_1999}. The only novelty in our approach lays in
+\cite{miranda_context_1999,andreas_gal_trace-based_2009}. The only novelty in our approach lays in
its generality: all the delicate support code for this is generated, as opposed to
most other JITs, which are just specifically written for one particular
language.
-XXX one of the JS tracing JITs does this as well
-
% section Taming Frame Objects (end)
\section{Evaluation}
From cfbolz at codespeak.net Thu Oct 14 16:12:06 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 16:12:06 +0200 (CEST)
Subject: [pypy-svn] r77940 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014141206.44DB3282BEC@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 16:12:04 2010
New Revision: 77940
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
I decided to ignore this comment, because that is dealt with by tracing itself,
not by the optimization that the paper is about
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 16:12:04 2010
@@ -140,10 +140,6 @@
dispatching. Those are problems that are usually not present or at least less
severe in statically typed languages.
-XXX [fijal] I would say that the other goal of the JIT in case of dynamic
-languages is to compile only a common scenario and leave a guard (instead
-of proving that something will never ever happen)
-
Boxing of primitive types is necessary because dynamic languages need to be able to handle
all objects, even integers, floats, booleans etc. in the same way as user-defined
instances. Thus those primitive types are usually \emph{boxed}, i.e. a small
From arigo at codespeak.net Thu Oct 14 17:15:19 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 14 Oct 2010 17:15:19 +0200 (CEST)
Subject: [pypy-svn] r77941 - pypy/trunk/pypy/module/_rawffi
Message-ID: <20101014151519.4AED4282BEA@codespeak.net>
Author: arigo
Date: Thu Oct 14 17:15:12 2010
New Revision: 77941
Modified:
pypy/trunk/pypy/module/_rawffi/array.py
Log:
Add a workaround for the libffi x86_64 bug.
Modified: pypy/trunk/pypy/module/_rawffi/array.py
==============================================================================
--- pypy/trunk/pypy/module/_rawffi/array.py (original)
+++ pypy/trunk/pypy/module/_rawffi/array.py Thu Oct 14 17:15:12 2010
@@ -97,7 +97,14 @@
class W_ArrayInstance(W_DataInstance):
def __init__(self, space, shape, length, address=r_uint(0)):
- W_DataInstance.__init__(self, space, shape.size * length, address)
+ # XXX workaround for a bug in libffi on x86_64: make sure that
+ # we always have at least 8 bytes. For W_ArrayInstances that are
+ # used as the result value of a function call, ffi_call() writes
+ # 8 bytes into it even if the function's result type asks for less.
+ memsize = shape.size * length
+ if memsize < 8:
+ memsize = 8
+ W_DataInstance.__init__(self, space, memsize, address)
self.length = length
self.shape = shape
From arigo at codespeak.net Thu Oct 14 17:16:01 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 14 Oct 2010 17:16:01 +0200 (CEST)
Subject: [pypy-svn] r77942 - pypy/branch/32ptr-on-64bit/pypy/module/_rawffi
Message-ID: <20101014151601.F06FF282BEA@codespeak.net>
Author: arigo
Date: Thu Oct 14 17:15:59 2010
New Revision: 77942
Modified:
pypy/branch/32ptr-on-64bit/pypy/module/_rawffi/array.py
Log:
Merge r77941 from trunk.
Modified: pypy/branch/32ptr-on-64bit/pypy/module/_rawffi/array.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/module/_rawffi/array.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/module/_rawffi/array.py Thu Oct 14 17:15:59 2010
@@ -97,7 +97,14 @@
class W_ArrayInstance(W_DataInstance):
def __init__(self, space, shape, length, address=r_uint(0)):
- W_DataInstance.__init__(self, space, shape.size * length, address)
+ # XXX workaround for a bug in libffi on x86_64: make sure that
+ # we always have at least 8 bytes. For W_ArrayInstances that are
+ # used as the result value of a function call, ffi_call() writes
+ # 8 bytes into it even if the function's result type asks for less.
+ memsize = shape.size * length
+ if memsize < 8:
+ memsize = 8
+ W_DataInstance.__init__(self, space, memsize, address)
self.length = length
self.shape = shape
From cfbolz at codespeak.net Thu Oct 14 17:24:52 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 17:24:52 +0200 (CEST)
Subject: [pypy-svn] r77943 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014152452.8353B282BEA@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 17:24:50 2010
New Revision: 77943
Modified:
pypy/extradoc/talk/pepm2011/paper.bib
pypy/extradoc/talk/pepm2011/paper.tex
Log:
mention self
Modified: pypy/extradoc/talk/pepm2011/paper.bib
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.bib (original)
+++ pypy/extradoc/talk/pepm2011/paper.bib Thu Oct 14 17:24:50 2010
@@ -296,6 +296,19 @@
pages = {144--153}
},
+ at article{chambers_efficient_1989,
+ title = {An efficient implementation of {SELF} a dynamically-typed object-oriented language based on prototypes},
+ volume = {24},
+ url = {http://portal.acm.org/citation.cfm?id=74884},
+ doi = {10.1145/74878.74884},
+ abstract = {We have developed and implemented techniques that double the performance of dynamically-typed object-oriented languages. Our {SELF} implementation runs twice as fast as the fastest Smalltalk implementation, despite {SELF's} lack of classes and explicit variables. To compensate for the absence of classes, our system uses implementation-level maps to transparently group objects cloned from the same prototype, providing data type information and eliminating the apparent space overhead for prototype-based systems. To compensate for dynamic typing, user-defined control structures, and the lack of explicit variables, our system dynamically compiles multiple versions of a source method, each customized according to its receiver's map. Within each version the type of the receiver is fixed, and thus the compiler can statically bind and inline all messages sent to self. Message splitting and type prediction extract and preserve even more static type information, allowing the compiler to inline many other messages. Inlining dramatically improves performance and eliminates the need to hard-wire low-level methods such as +,==, and {ifTrue:.} Despite inlining and other optimizations, our system still supports interactive programming environments. The system traverses internal dependency lists to invalidate all compiled methods affected by a programming change. The debugger reconstructs inlined stack frames from compiler-generated debugging information, making inlining invisible to the {SELF} programmer.},
+ number = {10},
+ journal = {{SIGPLAN} Not.},
+ author = {C. Chambers and D. Ungar and E. Lee},
+ year = {1989},
+ pages = {49--70},
+},
+
@inproceedings{rigo_representation-based_2004,
address = {Verona, Italy},
title = {Representation-based just-in-time specialization and the Psyco prototype for Python},
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 17:24:50 2010
@@ -1123,6 +1123,13 @@
PyPy's JIT can be seen as some successor of Psyco for a general context (one of
the authors of this paper is the author of Psyco).
+The original SELF JIT compiler \cite{chambers_efficient_1989} used an algorithm for
+forward-propagating the types of variables as part of its optimizations. This
+makes it possible to remove all type checks on a variable but the first one.
+The optimization does not deal with removing the full object, if it is
+short-lived, but the type check removals are similar to what our optimization
+achieves.
+
partial evaluation:
Prolog:
@@ -1134,8 +1141,6 @@
xxx: relation to compile-time garbage collection \cite{mazur_practical_2001}; separation logic; John Hughes: type specialization
-XXX cite SELF paper for the type propagation effects
-
\section{Conclusion}
\label{sec:conclusion}
From antocuni at codespeak.net Thu Oct 14 17:25:39 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Thu, 14 Oct 2010 17:25:39 +0200 (CEST)
Subject: [pypy-svn] r77944 - in pypy/branch/jitffi/pypy/jit/metainterp:
optimizeopt test
Message-ID: <20101014152539.2C39F282BEC@codespeak.net>
Author: antocuni
Date: Thu Oct 14 17:25:37 2010
New Revision: 77944
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
disable the ffi call optimization if we find any operation between the various
prepare_call/push_arg/do_call (the code in rlib/libffi.py is written in such a
way that we should never get any operation in between)
Modified: pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py Thu Oct 14 17:25:37 2010
@@ -12,10 +12,11 @@
restype = None
descr = None
- def __init__(self, funcval, cpu):
+ def __init__(self, funcval, cpu, prepare_op):
self.opargs = []
argtypes, restype = self._get_signature(funcval)
self.descr = cpu.calldescrof_dynamic(argtypes, restype)
+ self.prepare_op = prepare_op
def _get_signature(self, funcval):
"""
@@ -64,7 +65,8 @@
class OptFfiCall(Optimization):
def __init__(self):
- self.func_infos = {}
+ self.funcval = None
+ self.funcinfo = None
def _get_oopspec(self, op):
effectinfo = op.getdescr().get_extra_info()
@@ -72,6 +74,18 @@
return effectinfo.oopspecindex
return EffectInfo.OS_NONE
+ def rollback(self):
+ self.emit_operation(self.funcinfo.prepare_op)
+ for op in self.funcinfo.opargs:
+ self.emit_operation(op)
+ self.funcval = None
+ self.funcinfo = None
+
+ def optimize_default(self, op):
+ if self.funcval:
+ self.rollback()
+ self.emit_operation(op)
+
def optimize_CALL(self, op):
oopspec = self._get_oopspec(op)
try:
@@ -92,31 +106,40 @@
def _get_funcval(self, op):
funcval = self.getvalue(op.getarg(1))
+ if self.funcval:
+ assert self.funcval is funcval # XXX do something nice
if not funcval.is_constant():
raise NonConstantFuncVal
return funcval
def do_prepare_call(self, op):
funcval = self._get_funcval(op)
- assert funcval not in self.func_infos # XXX: do something nice etc. etc.
- self.func_infos[funcval] = FuncInfo(funcval, self.optimizer.cpu)
+ assert self.funcval is None # XXX: do something nice etc. etc.
+ self.funcval = funcval
+ self.funcinfo = FuncInfo(funcval, self.optimizer.cpu, op)
def do_push_arg(self, op):
# we store the op in funcs because we might want to emit it later,
# in case we give up with the optimization
+ if self.funcval is None:
+ self.emit_operation(op)
+ return
funcval = self._get_funcval(op)
- self.func_infos[funcval].opargs.append(op)
+ self.funcinfo.opargs.append(op)
def do_call(self, op):
+ if self.funcval is None:
+ return op
funcval = self._get_funcval(op)
- info = self.func_infos[funcval]
+ info = self.funcinfo
funcsymval = self.getvalue(op.getarg(2))
arglist = [funcsymval.force_box()]
for push_op in info.opargs:
argval = self.getvalue(push_op.getarg(2))
arglist.append(argval.force_box())
newop = ResOperation(rop.CALL_MAY_FORCE, arglist, op.result, descr=info.descr)
- del self.func_infos[funcval]
+ self.funcval = None
+ self.funcinfo = None
return newop
def propagate_forward(self, op):
@@ -126,6 +149,6 @@
func(self, op)
break
else:
- self.emit_operation(op)
+ self.optimize_default(op)
optimize_ops = _findall(OptFfiCall, 'optimize_')
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py Thu Oct 14 17:25:37 2010
@@ -85,3 +85,17 @@
expected = ops
loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
+ def test_rollback_if_op_in_between(self):
+ ops = """
+ [i0, f1]
+ call(0, ConstPtr(func), descr=libffi_prepare)
+ call(0, ConstPtr(func), i0, descr=libffi_push_arg)
+ i1 = int_add(i0, 1)
+ call(0, ConstPtr(func), f1, descr=libffi_push_arg)
+ i3 = call_may_force(0, ConstPtr(func), 12345, descr=libffi_call)
+ guard_not_forced() []
+ guard_no_exception() []
+ jump(i3, f1)
+ """
+ expected = ops
+ loop = self.optimize_loop(ops, 'Not, Not', expected)
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py Thu Oct 14 17:25:37 2010
@@ -139,6 +139,7 @@
totwidth = py.io.get_terminal_width()
width = totwidth / 2 - 1
print ' Comparing lists '.center(totwidth, '-')
+ print '%s| %s' % ('optimized'.center(width), 'expected'.center(width))
for op1, op2 in zip(oplist1, oplist2):
txt1 = str(op1)
txt2 = str(op2)
From cfbolz at codespeak.net Thu Oct 14 17:27:07 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 17:27:07 +0200 (CEST)
Subject: [pypy-svn] r77945 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014152707.1720F282BEF@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 17:27:05 2010
New Revision: 77945
Modified:
pypy/extradoc/talk/pepm2011/paper.bib
Log:
full information for prolog JIT paper
Modified: pypy/extradoc/talk/pepm2011/paper.bib
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.bib (original)
+++ pypy/extradoc/talk/pepm2011/paper.bib Thu Oct 14 17:27:05 2010
@@ -38,15 +38,6 @@
pages = {152--160}
},
- at inproceedings{carl_friedrich_bolz_towards_2010,
- address = {Hagenberg, Austria},
- title = {Towards a Jitting {VM} for Prolog Execution},
- abstract = {Most Prolog implementations are implemented in low-level languages such as C and are based on a variation of the {WAM} instruction set, which enhances their performance but makes them hard to write. In addition, many of the more dynamic features of Prolog (like assert), despite their popularity, are not well supported. We present a high-level continuation-based Prolog interpreter based on the {PyPy} project. The {PyPy} project makes it possible to easily and efficiently implement dynamic languages. It provides tools that automatically generate a just-in-time compiler for a given interpreter of the target language, by using partial evaluation techniques. The resulting Prolog implementation is surprisingly efficient: it clearly outperforms existing interpreters of Prolog in high-level languages such as Java. Moreover, on some benchmarks, our system outperforms state-of-the-art {WAM-based} Prolog implementations. Our paper aims to show that declarative languages such as Prolog can indeed benefit from having a just-in-time compiler and that {PyPy} can form the basis for implementing programming languages other than Python.},
- author = {Carl Friedrich Bolz and Michael Leuschel and David Schneider},
- year = {2010},
- note = {accepted for publication}
-},
-
@misc{_property_????,
title = {Property cache - {MDC}},
url = {https://developer.mozilla.org/en/SpiderMonkey/Internals/Property_cache},
@@ -130,6 +121,20 @@
note = {Technical Report {DISI-TH-2010-05}}
},
+ at inproceedings{carl_friedrich_bolz_towards_2010,
+ address = {Hagenberg, Austria},
+ title = {Towards a Jitting {VM} for Prolog execution},
+ isbn = {978-1-4503-0132-9},
+ url = {http://portal.acm.org/citation.cfm?id=1836102},
+ doi = {10.1145/1836089.1836102},
+ abstract = {Most Prolog implementations are implemented in low-level languages such as C and are based on a variation of the {WAM} instruction set, which enhances their performance but makes them hard to write. In addition, many of the more dynamic features of Prolog (like assert), despite their popularity, are not well supported. We present a high-level continuation-based Prolog interpreter based on the {PyPy} project. The {PyPy} project makes it possible to easily and efficiently implement dynamic languages. It provides tools that automatically generate a just-in-time compiler for a given interpreter of the target language, by using partial evaluation techniques. The resulting Prolog implementation is surprisingly efficient: it clearly outperforms existing interpreters of Prolog in high-level languages such as Java. Moreover, on some benchmarks, our system outperforms state-of-the-art {WAM-based} Prolog implementations. Our paper aims to show that declarative languages such as Prolog can indeed benefit from having a just-in-time compiler and that {PyPy} can form the basis for implementing programming languages other than Python.},
+ booktitle = {Proceedings of the 12th international {ACM} {SIGPLAN} symposium on Principles and practice of declarative programming},
+ publisher = {{ACM}},
+ author = {Carl Friedrich Bolz and Michael Leuschel and David Schneider},
+ year = {2010},
+ pages = {99--108}
+},
+
@inproceedings{chang_tracing_2009,
address = {Washington, {DC,} {USA}},
title = {Tracing for Web 3.0: Trace Compilation for the Next Generation Web Applications},
From david at codespeak.net Thu Oct 14 17:37:19 2010
From: david at codespeak.net (david at codespeak.net)
Date: Thu, 14 Oct 2010 17:37:19 +0200 (CEST)
Subject: [pypy-svn] r77946 - in
pypy/branch/arm-backend/pypy/jit/backend/arm: . test
Message-ID: <20101014153719.CD52736E076@codespeak.net>
Author: david
Date: Thu Oct 14 17:37:17 2010
New Revision: 77946
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/gen.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
Log:
Encode register based data processing and generate functions for them.
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Thu Oct 14 17:37:17 2010
@@ -1,4 +1,5 @@
import conditions as cond
+import registers as reg
from pypy.rlib.rmmap import alloc
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.jit.backend.arm.instruction_builder import define_instructions
@@ -26,14 +27,6 @@
| (rd & 0xF) << 12
| (imm & 0xFFF))
-
- def MOV_rr(self, rd, rm, cond=cond.AL, s=0):
- self.write32(cond << 28
- | 0xD << 21
- | (s & 0x1) << 20
- | (rd & 0xFF) << 12
- | (rm & 0xFF))
-
def MOV_ri(self, rt, imm=0, cond=cond.AL):
# XXX Check the actual allowed size for imm
# XXX S bit
@@ -44,27 +37,8 @@
| (rt & 0xF) << 12
| (imm & 0xFFF))
- def ASR_ri(self, rd, rm, imm=0, cond=cond.AL, s=0):
- self.write32(cond << 28
- | 0xD << 21
- | (s & 0x1) << 20
- | (rd & 0xF) << 12
- | (imm & 0x1F) << 7
- | 0x4 << 4
- | (rm & 0xF))
-
- #XXX encode shifttype correctly
- def ORR_rr(self, rd, rn, rm, imm=0, cond=cond.AL, s=0, shifttype=0):
- self.write32(cond << 28
- | 0x3 << 23
- | (s & 0x1) << 20
- | (rn & 0xFF) << 16
- | (rd & 0xFF) << 12
- | (imm & 0x1F) << 7
- | (shifttype & 0x3) << 5
- | (rm & 0xFF))
-
def PUSH(self, regs, cond=cond.AL):
+ assert reg.sp not in regs
instr = self._encode_reg_list(cond << 28 | 0x92D << 16, regs)
self.write32(instr)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py Thu Oct 14 17:37:17 2010
@@ -1,7 +1,7 @@
from pypy.jit.backend.arm import conditions as cond
from pypy.jit.backend.arm import instructions
-def define_load_store_func(target, name, table):
+def define_load_store_func(name, table):
# XXX W and P bits are not encoded yet
n = (0x1 << 26
| (table['A'] & 0x1) << 25
@@ -30,14 +30,60 @@
| (p & 0x1) << 24
| (u & 0x1) << 23
| (w & 0x1) << 21
- | (rn & 0xFF) << 16
- | (rt & 0xFF) << 12
+ | reg_operation(rt, rn, rm, imm, s, shifttype))
+ return f
+
+def define_data_proc(name, table):
+ n = ((table['op1'] & 0x1F) << 20
+ | (table['op2'] & 0x1F) << 7
+ | (table['op3'] & 0x3) << 5)
+ if name[-2:] == 'ri':
+ def f(self, rd, rm, imm=0, cond=cond.AL, s=0):
+ if table['op2cond'] == '!0':
+ assert imm != 0
+ elif table['op2cond'] == '0':
+ assert imm == 0
+ self.write32(n
+ | cond << 28
+ | (s & 0x1) << 20
+ | (rd & 0xFF) << 12
| (imm & 0x1F) << 7
- | (shifttype & 0x3) << 5
| (rm & 0xFF))
- setattr(target, name, f)
+ elif not table['result']:
+ # ops without result
+ def f(self, rn, rm, imm=0, cond=cond.AL, s=0, shifttype=0):
+ self.write32(n
+ | cond << 28
+ | reg_operation(0, rn, rm, imm, s, shifttype))
+ elif not table['base']:
+ # ops without base register
+ def f(self, rd, rm, imm=0, cond=cond.AL, s=0, shifttype=0):
+ self.write32(n
+ | cond << 28
+ | reg_operation(rd, 0, rm, imm, s, shifttype))
+ else:
+ def f(self, rd, rn, rm, imm=0, cond=cond.AL, s=0, shifttype=0):
+ self.write32(n
+ | cond << 28
+ | reg_operation(rd, rn, rm, imm, s, shifttype))
+ return f
+
+
+def reg_operation(rt, rn, rm, imm, s, shifttype):
+ # XXX encode shiftype correctly
+ return ((s & 0x1 << 20)
+ | (rn & 0xFF) << 16
+ | (rt & 0xFF) << 12
+ | (imm & 0x1F) << 7
+ | (shifttype & 0x3) << 5
+ | (rm & 0xFF))
def define_instructions(target):
for key, val in instructions.load_store.iteritems():
- define_load_store_func(target, key, val)
+ f = define_load_store_func(key, val)
+ setattr(target, key, f)
+
+ for key, val in instructions.data_proc.iteritems():
+ f = define_data_proc(key, val)
+ setattr(target, key, f)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py Thu Oct 14 17:37:17 2010
@@ -10,3 +10,25 @@
'LDRB_ri': {'A':0, 'op1': 0x5, 'B': 0, 'imm': True},
'LDRB_rr': {'A':1, 'op1': 0x5, 'B': 0, 'imm': False},
}
+
+data_proc = {
+ 'AND_rr': {'op1':0x0, 'op2':0, 'op3':0, 'result':True, 'base':True},
+ 'EOR_rr': {'op1':0x2, 'op2':0, 'op3':0, 'result':True, 'base':True},
+ 'SUB_rr': {'op1':0x4, 'op2':0, 'op3':0, 'result':True, 'base':True},
+ 'RSB_rr': {'op1':0x6, 'op2':0, 'op3':0, 'result':True, 'base':True},
+ 'ADD_rr': {'op1':0x8, 'op2':0, 'op3':0, 'result':True, 'base':True},
+ 'ADC_rr': {'op1':0xA, 'op2':0, 'op3':0, 'result':True, 'base':True},
+ 'SBC_rr': {'op1':0xC, 'op2':0, 'op3':0, 'result':True, 'base':True},
+ 'RSC_rr': {'op1':0xE, 'op2':0, 'op3':0, 'result':True, 'base':True},
+ 'TST_rr': {'op1':0x11, 'op2':0, 'op3':0, 'result':False, 'base':True},
+ 'TEQ_rr': {'op1':0x13, 'op2':0, 'op3':0, 'result':False, 'base':True},
+ 'CMP_rr': {'op1':0x15, 'op2':0, 'op3':0, 'result':False, 'base':True},
+ 'CMN_rr': {'op1':0x17, 'op2':0, 'op3':0, 'result':False, 'base':True},
+ 'ORR_rr': {'op1':0x18, 'op2':0, 'op3':0, 'result':True, 'base':True},
+ 'MOV_rr': {'op1':0x1A, 'op2':0, 'op3':0, 'result':True, 'base':False},
+ 'LSL_ri': {'op1':0x1A, 'op2':0x0, 'op3':0, 'op2cond':'!0', 'result':False, 'base':True},
+ 'LSR_ri': {'op1':0x1A, 'op2':0, 'op3':0x1, 'op2cond':'', 'result':False, 'base':True},
+ 'ASR_ri': {'op1':0x1A, 'op2':0, 'op3':0x2, 'op2cond':'', 'result':False, 'base':True},
+ #'RRX_ri': {'op1':0x1A, 'op2':0, 'op3':0x3, 'op2cond':'0', 'result':False, 'base':True},
+ 'ROR_ri': {'op1':0x1A, 'op2':0x0, 'op3':0x3, 'op2cond':'!0', 'result':True, 'base':False},
+}
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/gen.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/gen.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/gen.py Thu Oct 14 17:37:17 2010
@@ -38,7 +38,7 @@
res = self.body % (self.instr)
self.file.write(res)
self.file.flush()
- os.system("%s %s %s -o %s/a.out" % (AS, self.asm_opts, self.name, self.tmpdir))
+ os.system("%s --fatal-warnings %s %s -o %s/a.out" % (AS, self.asm_opts, self.name, self.tmpdir))
def __del__(self):
self.file.close()
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Thu Oct 14 17:37:17 2010
@@ -82,8 +82,8 @@
self.assert_equal('PUSH {r1}')
def test_push_multiple(self):
- self.cb.PUSH([r.r3, r.r1, r.r6, r.r8, r.sp, r.pc])
- self.assert_equal('PUSH {r3, r1, r6, r8, sp, pc}')
+ self.cb.PUSH([r.r1, r.r3, r.r6, r.r8, r.pc])
+ self.assert_equal('PUSH {r1, r3, r6, r8, pc}')
def test_push_multiple2(self):
self.cb.PUSH([r.fp, r.ip, r.lr, r.pc])
@@ -126,7 +126,11 @@
else:
f = gen_test_reg_func
test = f(key, value)
- setattr(TestInstrCodeBuilderForGeneratedInstr, 'test_%s' % key, test)
+ setattr(TestInstrCodeBuilderForGeneratedInstr, 'test_%s' % key, test)
+
+ for key, value, in instructions.data_proc.iteritems():
+ test = gen_test_data_reg_func(key, value)
+ setattr(TestInstrCodeBuilderForGeneratedInstr, 'test_%s' % key, test)
def gen_test_imm_func(name, table):
def f(self):
@@ -141,4 +145,25 @@
func(r.r3, r.r7, r.r12)
self.assert_equal('%s r3, [r7, r12]' % name[:name.index('_')])
return f
+
+def gen_test_data_reg_func(name, table):
+ if name[-2:] == 'ri':
+ def f(self):
+ func = getattr(self.cb, name)
+ func(r.r3, r.r7, 12)
+ self.assert_equal('%s r3, r7, #12' % name[:name.index('_')])
+
+ elif table['base'] and table['result']:
+ def f(self):
+ func = getattr(self.cb, name)
+ func(r.r3, r.r7, r.r12)
+ self.assert_equal('%s r3, r7, r12' % name[:name.index('_')])
+ else:
+ def f(self):
+ func = getattr(self.cb, name)
+ func(r.r3, r.r7)
+ self.assert_equal('%s r3, r7' % name[:name.index('_')])
+
+ return f
+
build_tests()
From antocuni at codespeak.net Thu Oct 14 17:39:06 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Thu, 14 Oct 2010 17:39:06 +0200 (CEST)
Subject: [pypy-svn] r77947 -
pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt
Message-ID: <20101014153906.49CD436E076@codespeak.net>
Author: antocuni
Date: Thu Oct 14 17:39:04 2010
New Revision: 77947
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
Log:
simplify the logic, it's much more readable now
Modified: pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py Thu Oct 14 17:39:04 2010
@@ -59,9 +59,6 @@
return f.inst_argtypes, f.inst_restype
-class NonConstantFuncVal(Exception):
- pass
-
class OptFfiCall(Optimization):
def __init__(self):
@@ -88,18 +85,14 @@
def optimize_CALL(self, op):
oopspec = self._get_oopspec(op)
- try:
- if oopspec == EffectInfo.OS_LIBFFI_PREPARE:
- self.do_prepare_call(op)
- elif oopspec == EffectInfo.OS_LIBFFI_PUSH_ARG:
- self.do_push_arg(op)
- elif oopspec == EffectInfo.OS_LIBFFI_CALL:
- op = self.do_call(op)
- self.emit_operation(op)
- else:
- raise NonConstantFuncVal # it's not a libffi call
- except NonConstantFuncVal:
- # normal case
+ if oopspec == EffectInfo.OS_LIBFFI_PREPARE:
+ op = self.do_prepare_call(op)
+ elif oopspec == EffectInfo.OS_LIBFFI_PUSH_ARG:
+ op = self.do_push_arg(op)
+ elif oopspec == EffectInfo.OS_LIBFFI_CALL:
+ op = self.do_call(op)
+ #
+ if op:
self.emit_operation(op)
optimize_CALL_MAY_FORCE = optimize_CALL
@@ -109,21 +102,20 @@
if self.funcval:
assert self.funcval is funcval # XXX do something nice
if not funcval.is_constant():
- raise NonConstantFuncVal
+ return None
return funcval
def do_prepare_call(self, op):
funcval = self._get_funcval(op)
+ if not funcval:
+ return op
assert self.funcval is None # XXX: do something nice etc. etc.
self.funcval = funcval
self.funcinfo = FuncInfo(funcval, self.optimizer.cpu, op)
def do_push_arg(self, op):
- # we store the op in funcs because we might want to emit it later,
- # in case we give up with the optimization
if self.funcval is None:
- self.emit_operation(op)
- return
+ return op
funcval = self._get_funcval(op)
self.funcinfo.opargs.append(op)
From antocuni at codespeak.net Thu Oct 14 17:43:47 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Thu, 14 Oct 2010 17:43:47 +0200 (CEST)
Subject: [pypy-svn] r77948 - in pypy/branch/jitffi/pypy/jit/metainterp:
optimizeopt test
Message-ID: <20101014154347.95C7636E076@codespeak.net>
Author: antocuni
Date: Thu Oct 14 17:43:46 2010
New Revision: 77948
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
Log:
add one more case when to rollback
Modified: pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py Thu Oct 14 17:43:46 2010
@@ -100,7 +100,9 @@
def _get_funcval(self, op):
funcval = self.getvalue(op.getarg(1))
if self.funcval:
- assert self.funcval is funcval # XXX do something nice
+ if self.funcval is not funcval:
+ self.rollback()
+ return None
if not funcval.is_constant():
return None
return funcval
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py Thu Oct 14 17:43:46 2010
@@ -40,6 +40,9 @@
func = FakeLLObject(_fake_class=Func,
argtypes=[types.sint, types.double],
restype=types.sint)
+ func2 = FakeLLObject(_fake_class=Func,
+ argtypes=[types.sint, types.double],
+ restype=types.sint)
#
def calldescr(cpu, FUNC, oopspecindex):
einfo = EffectInfo([], [], [], oopspecindex=oopspecindex)
@@ -99,3 +102,21 @@
"""
expected = ops
loop = self.optimize_loop(ops, 'Not, Not', expected)
+
+ def test_rollback_multiple_calls(self):
+ ops = """
+ [i0, i2, f1]
+ call(0, ConstPtr(func), descr=libffi_prepare)
+ call(0, ConstPtr(func), i0, descr=libffi_push_arg)
+ call(0, ConstPtr(func2), descr=libffi_prepare) # culprit!
+ call(0, ConstPtr(func), f1, descr=libffi_push_arg)
+ i3 = call_may_force(0, ConstPtr(func), 12345, descr=libffi_call)
+ call(0, ConstPtr(func2), i0, descr=libffi_push_arg)
+ call(0, ConstPtr(func2), f1, descr=libffi_push_arg)
+ i4 = call_may_force(0, ConstPtr(func), 12345, descr=libffi_call)
+ guard_not_forced() []
+ guard_no_exception() []
+ jump(i3, i4, f1)
+ """
+ expected = ops
+ loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
From cfbolz at codespeak.net Thu Oct 14 17:45:10 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 17:45:10 +0200 (CEST)
Subject: [pypy-svn] r77949 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014154510.883B036E076@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 17:45:08 2010
New Revision: 77949
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
flesh out section about tracing
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 17:45:08 2010
@@ -1132,9 +1132,14 @@
partial evaluation:
-Prolog:
-partially static datastructures are already built-in to Prolog and was built-into partial
-evaluation from the beginning \cite{lloyd_partial_1991}.
+Partially known data structures are built directly into Prolog (via unbound
+logic variables) and thus the optimization of partially static data structures
+was part of partial evaluation of Prolog programs from the beginning
+\cite{lloyd_partial_1991}. One effect of unfolding in Prolog is that terms that
+are constructed and immediately matched again completely disappear in the
+residual program, which is equivalent to what our optimization does for an
+imperative language.
+
FP
partially static data structures: kenichi asai's thesis?
From cfbolz at codespeak.net Thu Oct 14 18:22:17 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 18:22:17 +0200 (CEST)
Subject: [pypy-svn] r77951 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014162217.5F81E282BEC@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 18:22:11 2010
New Revision: 77951
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
fixes by david
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 18:22:11 2010
@@ -132,11 +132,11 @@
The goal of a just-in-time (JIT) compiler for a dynamic language is obviously to
improve the speed of the language over an implementation of the language that
-uses interpretation. The first goal of a JIT is thus to remove the
+uses interpretation. The first goal of a JIT is therefore to remove the
interpretation overhead, i.e. the overhead of bytecode (or AST) dispatch and the
overhead of the interpreter's data structures, such as operand stack etc. The
second important problem that any JIT for a dynamic language needs to solve is
-how to deal with the overhead of boxing of primitive types and of type
+how to deal with the overhead of boxing primitive types and of type
dispatching. Those are problems that are usually not present or at least less
severe in statically typed languages.
@@ -167,7 +167,7 @@
simplicity. They can often be added to an interpreter and a lot of the
infrastructure of the interpreter can be reused. They give some important
optimizations like inlining and constant-folding for free. A tracing JIT always
-produces linear pieces of code, which simplifies many optimizations that are usually
+produces linear pieces of code, which simplifies many algorithms that are usually
hard in a compiler, such as register allocation.
The usage of a tracing JIT can remove the overhead of bytecode dispatch and that
@@ -262,7 +262,7 @@
The core idea of tracing JITs is to focus the optimization effort of the JIT
compiler on the hot paths of the core loops of the program and to just use an
interpreter for the less commonly executed parts. VMs that use a tracing JIT are
-thus mixed-mode execution environments, they contain both an interpreter and a
+mostly mixed-mode execution environments, they contain both an interpreter and a
JIT compiler. By default the interpreter is used to execute the program, doing
some light-weight profiling at the same time. This profiling is used to identify
the hot loops of the program. If a hot loop is found in that way, the
@@ -276,7 +276,7 @@
This trace of operations is then the basis of the generated code. The trace is
first optimized, and then turned into machine code. Both optimization
-and machine code generation is simple, because the traces are linear. This
+and machine code generation are simple, because the traces are linear. This
linearity makes many optimizations a lot more tractable, and the inlining that
happens gives the optimizations automatically more context to work with.
@@ -288,8 +288,9 @@
trace. As an example, if a loop contains an \lstinline{if} statement, the trace
will contain the execution of one of the paths only, which is the path that was
taken during the production of the trace. The trace will also contain a guard
-that checks that the condition of the \lstinline{if} statement is true, because if
-it isn't, the rest of the trace is not valid.
+that checks that the condition of the \lstinline{if} statement is the same as
+during tracing, because if
+it isn't, the rest of the trace is not valid. \cfbolz{The "if" shouldn't be bold}
When generating machine code, every guard is be turned into a quick check to
see whether the assumption still holds. When such a guard is hit during the
@@ -565,7 +566,7 @@
The main insight to improve the code shown in the last section is that objects
in category 1 don't survive very long -- they are used only inside the loop and
nobody else in the program stores a reference to them. The idea for improving
-the code is thus to analyze which objects fall in category 1 and thus do
+the code is to analyze which objects fall in category 1 and therefore do
not have to be allocated at all.
This is a process that is usually called \emph{escape analysis}. In this paper we will
@@ -835,7 +836,7 @@
The static heap is a partial function from $V^*$ into the
set of static objects, which are triples of a type and two elements of $V^*$.
A variable $v^*$ is in the domain of the static heap $S$ as long as the
-optimizer can fully keep track of the object. The image of $v^*$ is what is
+optimizer can fully keep track of the object. The object $S(v^*)$ is what is
statically known about the object stored in it, \ie its type and its fields. The
fields of objects in the static heap are also elements of $V^*$ (or null, for
short periods of time).
@@ -1048,7 +1049,8 @@
result. The errors were computed using a confidence interval with a 95\%
confidence level \cite{georges_statistically_2007}. The results are reported in
Figure~\ref{fig:times}. In addition to the run times the table also reports the
-speedup that PyPy with optimization turned on achieves.
+speedup that PyPy achieves when the optimization is turned on.
+
With the optimization turned on, PyPy's Python interpreter outperforms CPython
in all benchmarks except spambayes (which heavily relies on regular expression
performance and thus is not helped much by our Python JIT) and meteor-contest.
@@ -1089,7 +1091,7 @@
\section{Related Work}
\label{sec:related}
-There exists a large number of works on escape analysis, which is an program
+There exists a large number of works on escape analysis, which is a program
analysis that tries to find an upper bound for the lifetime of objects allocated
at specific program points
\cite{goldberg_higher_1990,park_escape_1992,choi_escape_1999,bruno_blanchet_escape_2003}.
@@ -1157,7 +1159,7 @@
hardest part of partial evaluation: the tracing JIT selects the parts
of the program that are worthwhile to optimize, and extracts linear
paths through them, inlining functions as necessary. What is left to
-optimize is only those linear paths.
+optimize are only those linear paths.
We expect a similar result for other optimizations that usually require
a complex analysis phase and are thus normally too slow to use at
From afa at codespeak.net Thu Oct 14 18:35:06 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 14 Oct 2010 18:35:06 +0200 (CEST)
Subject: [pypy-svn] r77952 - pypy/branch/fast-forward/pypy/module/thread/test
Message-ID: <20101014163506.6E982282BEA@codespeak.net>
Author: afa
Date: Thu Oct 14 18:35:04 2010
New Revision: 77952
Modified:
pypy/branch/fast-forward/pypy/module/thread/test/support.py
pypy/branch/fast-forward/pypy/module/thread/test/test_fork.py
Log:
A failing test which shows why test_multiprocessing hangs on startup...
Modified: pypy/branch/fast-forward/pypy/module/thread/test/support.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/thread/test/support.py (original)
+++ pypy/branch/fast-forward/pypy/module/thread/test/support.py Thu Oct 14 18:35:04 2010
@@ -1,5 +1,5 @@
import py
-import time, gc
+import time, gc, thread, os
from pypy.conftest import gettestobjspace, option
from pypy.interpreter.gateway import ObjSpace, W_Root, interp2app_temp
from pypy.module.thread import gil
@@ -21,6 +21,13 @@
print '*** timed out ***'
waitfor.unwrap_spec = [ObjSpace, W_Root, float]
+def timeout_killer(pid, delay):
+ def kill():
+ time.sleep(delay)
+ os.kill(pid, 9)
+ print "process %s killed!" % (pid,)
+ thread.start_new_thread(kill, ())
+timeout_killer.unwrap_spec = [int, float]
class GenericTestThread:
@@ -47,3 +54,4 @@
import time
return time.sleep
""")
+ cls.w_timeout_killer = space.wrap(interp2app_temp(timeout_killer))
Modified: pypy/branch/fast-forward/pypy/module/thread/test/test_fork.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/thread/test/test_fork.py (original)
+++ pypy/branch/fast-forward/pypy/module/thread/test/test_fork.py Thu Oct 14 18:35:04 2010
@@ -36,3 +36,20 @@
finally:
run = False
self.waitfor(lambda: done)
+
+ def test_forked_can_thread(self):
+ "Checks that a forked interpreter can start a thread"
+ import os, thread, time
+
+ # pre-allocate some locks
+ thread.start_new_thread(lambda: None, ())
+
+ pid = os.fork()
+ if pid == 0:
+ print 'in child'
+ thread.start_new_thread(lambda: None, ())
+ os._exit(0)
+ else:
+ self.timeout_killer(pid, 5)
+ exitcode = os.waitpid(pid, 0)[1]
+ assert exitcode == 0 # if 9, process was killed by timer!
From arigo at codespeak.net Thu Oct 14 18:35:50 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 14 Oct 2010 18:35:50 +0200 (CEST)
Subject: [pypy-svn] r77953 - pypy/trunk/pypy/module/_rawffi
Message-ID: <20101014163550.6F7E1282BEA@codespeak.net>
Author: arigo
Date: Thu Oct 14 18:35:48 2010
New Revision: 77953
Modified:
pypy/trunk/pypy/module/_rawffi/array.py
Log:
Fix the comment: remove the XXX and says "it's documented".
Modified: pypy/trunk/pypy/module/_rawffi/array.py
==============================================================================
--- pypy/trunk/pypy/module/_rawffi/array.py (original)
+++ pypy/trunk/pypy/module/_rawffi/array.py Thu Oct 14 18:35:48 2010
@@ -97,10 +97,11 @@
class W_ArrayInstance(W_DataInstance):
def __init__(self, space, shape, length, address=r_uint(0)):
- # XXX workaround for a bug in libffi on x86_64: make sure that
+ # Workaround for a strange behavior of libffi: make sure that
# we always have at least 8 bytes. For W_ArrayInstances that are
# used as the result value of a function call, ffi_call() writes
# 8 bytes into it even if the function's result type asks for less.
+ # This strange behavior is documented.
memsize = shape.size * length
if memsize < 8:
memsize = 8
From arigo at codespeak.net Thu Oct 14 18:43:40 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 14 Oct 2010 18:43:40 +0200 (CEST)
Subject: [pypy-svn] r77954 - pypy/branch/leak-finder
Message-ID: <20101014164340.5BCFD282BEA@codespeak.net>
Author: arigo
Date: Thu Oct 14 18:43:38 2010
New Revision: 77954
Added:
pypy/branch/leak-finder/
- copied from r77953, pypy/trunk/
Log:
Trying to figure out how to detect leaks of raw-flavoured mallocs.
From cfbolz at codespeak.net Thu Oct 14 18:45:54 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 14 Oct 2010 18:45:54 +0200 (CEST)
Subject: [pypy-svn] r77955 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014164554.B2C8936E076@codespeak.net>
Author: cfbolz
Date: Thu Oct 14 18:45:53 2010
New Revision: 77955
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
remove a number of "thus"s. simplify two paragraphs in the background section.
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 18:45:53 2010
@@ -162,8 +162,11 @@
A recently popular approach to implementing just-in-time compilers for dynamic
languages is that of a tracing JIT. A tracing JIT works by observing the running
-program and recording its hot spots into linear execution traces, which are then turned into
-machine code. One reason for the popularity of tracing JITs is their relative
+program and recording its hot spots into linear execution traces. Working on
+traces is the central idea of a tracing JIT. Those traces are optimized and
+turned into machine code.
+
+One reason for the popularity of tracing JITs is their relative
simplicity. They can often be added to an interpreter and a lot of the
infrastructure of the interpreter can be reused. They give some important
optimizations like inlining and constant-folding for free. A tracing JIT always
@@ -215,11 +218,10 @@
\emph{RPython} \cite{davide_ancona_rpython:_2007}. RPython ("restricted Python")
is a subset of Python chosen in such a way that type inference becomes
possible. The language interpreter can then be compiled (``translated'') with
-PyPy's tools into a VM on the C level. Because the interpreter is written at a
-relatively high level, the language implementation is kept free of low-level
-details, such as object layout, garbage collection or memory model. Those
-aspects of the final VM are woven into the generated code during the translation
-to C.
+PyPy's tools into a VM on the C level. During translation to C, many low-level
+aspects of the final VM, such as object layout, garbage collection and memory
+model, are woven into the generated code. Therefore the interpreter itself can
+remain at a relatively high level of abstraction.
A number of languages have been implemented with PyPy. The project was initiated
to get a better Python implementation, which inspired the name of the project
@@ -231,7 +233,7 @@
The feature that makes PyPy more than a compiler with a runtime system is its
support for automated JIT compiler generation \cite{bolz_tracing_2009}. During
-the translation to C, PyPy's tools can generate a just-in-time compiler for the
+the translation to C, PyPy's tools can generate a tracing just-in-time compiler for the
language that the interpreter is implementing. This process is mostly
automatic; it only needs to be guided by the language implementer using a small number of
source-code hints. Mostly-automatically generating a JIT compiler has many advantages
@@ -240,12 +242,9 @@
Many optimizations can benefit all languages implemented as an interpreter in RPython.
Moreover, thanks to the internal design of the JIT generator, it is very easy
-to add new \emph{backends} for producing the actual machine code, in addition
-to the original backend for the Intel \emph{x86} architecture. Examples of
-additional JIT backends are the one for Intel \emph{x86-64} and an
+to add new \emph{backends} for producing the actual machine code. Examples of
+JIT backends that are implemented are the one for Intel x86 and x86-64 and an
experimental one for the CLI .NET Virtual Machine \cite{cuni_high_2010}.
-PyPy's JIT generator generates a \emph{tracing JIT
-compiler}, a concept which we now explain in more details.
\subsection{Tracing JIT Compilers}
\label{sub:JIT_background}
@@ -373,9 +372,9 @@
Using these classes to implement arithmetic shows the basic problem that a
dynamic language implementation has. All the numbers are instances of either
-\lstinline{BoxedInteger} or \lstinline{BoxedFloat}, thus they consume space on the
+\lstinline{BoxedInteger} or \lstinline{BoxedFloat}, therefore they consume space on the
heap. Performing many arithmetic operations produces lots of garbage quickly,
-thus putting pressure on the garbage collector. Using double dispatching to
+which puts pressure on the garbage collector. Using double dispatching to
implement the numeric tower needs two method calls per arithmetic operation,
which is costly due to the method dispatch.
@@ -593,8 +592,8 @@
the type check the guard does is statically known.
In the example from last section, the following operations in the upper half
-of Fig.~\ref{fig:unopt-trace} would produce two
-static objects, and would thus be completely removed from the optimized trace:
+of Fig.~\ref{fig:unopt-trace} produce two
+static objects, and can be completely removed from the optimized trace:
\begin{lstlisting}[mathescape,xleftmargin=20pt]
$p_{5}$ = new(BoxedInteger)
@@ -625,7 +624,7 @@
First, the \lstinline{guard_class} operations can be removed, because the classes of $p_{5}$ and
$p_{6}$ are known to be \lstinline{BoxedInteger}. Second, the \lstinline{get} operations can be removed
-and $i_{7}$ and $i_{8}$ are just replaced by $i_{4}$ and -100. Thus the only
+and $i_{7}$ and $i_{8}$ are just replaced by $i_{4}$ and -100. The only
remaining operation in the optimized trace would be:
\begin{lstlisting}[mathescape,xleftmargin=20pt]
@@ -746,8 +745,8 @@
to locations and the heap is a partial function from locations to objects. Note
that a variable can never be null in the environment, otherwise the trace would
be malformed. The environment could not directly map variables to object,
-because several variables can contain a pointer to the \emph{same} object. Thus
-the "indirection" is needed to express sharing.
+because several variables can contain a pointer to the \emph{same} object.
+The "indirection" is needed to express sharing.
We use the following notation for updating partial functions:
$E[v\mapsto l]$ denotes the environment which is just like $E$, but maps $v$ to
@@ -830,7 +829,7 @@
themselves written with a
$\ ^*$ for clarity). The reason for introducing new variables in the optimized
trace is that several variables that appear in the unoptimized trace can turn
-into the same variables in the optimized trace. Thus the environment of the
+into the same variables in the optimized trace. The environment of the
optimizer serves a function similar to that of the environment in the semantics: sharing.
The static heap is a partial function from $V^*$ into the
From agaynor at codespeak.net Thu Oct 14 19:30:53 2010
From: agaynor at codespeak.net (agaynor at codespeak.net)
Date: Thu, 14 Oct 2010 19:30:53 +0200 (CEST)
Subject: [pypy-svn] r77956 - pypy/extradoc/talk/pepm2011
Message-ID: <20101014173053.ADC06282BEA@codespeak.net>
Author: agaynor
Date: Thu Oct 14 19:30:51 2010
New Revision: 77956
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
Small grammar fix.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 14 19:30:51 2010
@@ -184,7 +184,7 @@
The core of our trace optimization technique can be
viewed as partial evaluation: the partial evaluation
-performs a form of escape analysis \cite{bruno_blanchet_escape_2003} on the traces and make some
+performs a form of escape analysis \cite{bruno_blanchet_escape_2003} on the traces and makes some
objects that are allocated in the trace \emph{static,} which
means that they do not occur any more in the optimized trace. This technique is
informally described in Section~\ref{sec:statics}; a more formal description is
From afa at codespeak.net Thu Oct 14 21:10:43 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 14 Oct 2010 21:10:43 +0200 (CEST)
Subject: [pypy-svn] r77957 - in pypy/branch/fast-forward/pypy/module: posix
thread
Message-ID: <20101014191043.E6897282BEA@codespeak.net>
Author: afa
Date: Thu Oct 14 21:10:17 2010
New Revision: 77957
Modified:
pypy/branch/fast-forward/pypy/module/posix/interp_posix.py
pypy/branch/fast-forward/pypy/module/thread/__init__.py
pypy/branch/fast-forward/pypy/module/thread/gil.py
pypy/branch/fast-forward/pypy/module/thread/os_thread.py
Log:
Add infrastructure for hooks around os.fork():
'before', 'parent', and 'child'.
Use it to reinit the thread.bootstrap lock,
and the threadlocals structure as well which seems initialized at the
same time.
This fixes the previous test, and helps test_multiprocessing a lot.
Modified: pypy/branch/fast-forward/pypy/module/posix/interp_posix.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/posix/interp_posix.py (original)
+++ pypy/branch/fast-forward/pypy/module/posix/interp_posix.py Thu Oct 14 21:10:17 2010
@@ -473,7 +473,7 @@
return True
def get(space):
- return space.fromcache(State)
+ return space.fromcache(State)
def _convertenviron(space, w_env):
space.call_method(w_env, 'clear')
@@ -600,11 +600,48 @@
return space.wrap(result)
readlink.unwrap_spec = [ObjSpace, str]
+before_fork_hooks = []
+after_fork_child_hooks = []
+after_fork_parent_hooks = []
+
+ at specialize.memo()
+def get_fork_hooks(where):
+ if where == 'before':
+ return before_fork_hooks
+ elif where == 'child':
+ return after_fork_child_hooks
+ elif where == 'parent':
+ return after_fork_parent_hooks
+ else:
+ assert False, "Unknown fork hook"
+
+def add_fork_hook(where, hook):
+ "NOT_RPYTHON"
+ get_fork_hooks(where).append(hook)
+
+ at specialize.arg(0)
+def run_fork_hooks(where, space):
+ for hook in get_fork_hooks(where):
+ hook(space)
+
def fork(space):
+ run_fork_hooks('before', space)
+
try:
pid = os.fork()
- except OSError, e:
- raise wrap_oserror(space, e)
+ except OSError, e:
+ try:
+ run_fork_hooks('parent', space)
+ except:
+ # Don't clobber the OSError if the fork failed
+ pass
+ raise wrap_oserror(space, e)
+
+ if pid == 0:
+ run_fork_hooks('child', space)
+ else:
+ run_fork_hooks('parent', space)
+
return space.wrap(pid)
def openpty(space):
Modified: pypy/branch/fast-forward/pypy/module/thread/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/thread/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/thread/__init__.py Thu Oct 14 21:10:17 2010
@@ -29,3 +29,8 @@
space.threadlocals = gil.GILThreadLocals()
space.threadlocals.initialize(space)
space.threadlocals.setvalue(prev)
+
+ from pypy.module.posix.interp_posix import add_fork_hook
+ from pypy.module.thread.os_thread import reinit_threads
+ add_fork_hook('child', reinit_threads)
+
Modified: pypy/branch/fast-forward/pypy/module/thread/gil.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/thread/gil.py (original)
+++ pypy/branch/fast-forward/pypy/module/thread/gil.py Thu Oct 14 21:10:17 2010
@@ -48,6 +48,12 @@
invoke_around_extcall(before_external_call, after_external_call)
return result
+ def reinit_threads(self, space):
+ if self.ll_GIL:
+ self.ll_GIL = thread.allocate_ll_lock()
+ thread.acquire_NOAUTO(self.ll_GIL, True)
+ self.enter_thread(space)
+
def yield_thread(self):
thread.yield_thread() # explicitly release the gil (used by test_gil)
Modified: pypy/branch/fast-forward/pypy/module/thread/os_thread.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/thread/os_thread.py (original)
+++ pypy/branch/fast-forward/pypy/module/thread/os_thread.py Thu Oct 14 21:10:17 2010
@@ -67,13 +67,17 @@
args = None
w_callable = None
+ @staticmethod
def setup(space):
if bootstrapper.lock is None:
try:
bootstrapper.lock = thread.allocate_lock()
except thread.error:
raise wrap_thread_error(space, "can't allocate bootstrap lock")
- setup = staticmethod(setup)
+
+ @staticmethod
+ def reinit(space):
+ bootstrapper.lock = None
def bootstrap():
# Note that when this runs, we already hold the GIL. This is ensured
@@ -135,6 +139,17 @@
space.threadlocals.setup_threads(space)
bootstrapper.setup(space)
+def reinit_threads(space):
+ "Called in the child process after a fork()"
+ space.threadlocals.reinit_threads(space)
+ bootstrapper.reinit(space)
+
+ # Clean the threading module after a fork()
+ w_modules = space.sys.get('modules')
+ w_threading = space.finditem_str(w_modules, 'threading')
+ if w_threading is not None:
+ space.call_method(w_threading, "_after_fork")
+
def start_new_thread(space, w_callable, w_args, w_kwargs=NoneNotWrapped):
"""Start a new thread and return its identifier. The thread will call the
From hakanardo at codespeak.net Thu Oct 14 21:49:26 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Thu, 14 Oct 2010 21:49:26 +0200 (CEST)
Subject: [pypy-svn] r77958 - in pypy/branch/jit-unroll-loops/pypy/jit:
metainterp/optimizeopt metainterp/test tl
Message-ID: <20101014194926.ECF41282BEC@codespeak.net>
Author: hakanardo
Date: Thu Oct 14 21:49:24 2010
New Revision: 77958
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py
pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py
Log:
Clear all last_guard_index before inlining, support the reording of operations in force_lazy_setfield and allow new inputargs to be added by operations induced by forcing the boxes of the final jump.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Thu Oct 14 21:49:24 2010
@@ -39,6 +39,7 @@
argmap[loop_args[i]] = jump_args[i]
for v in self.optimizer.values.values():
+ v.last_guard_index = -1 # FIXME: Are there any more indexes stored?
if not v.is_constant() and v.box:
v.fromstart = True
@@ -72,10 +73,16 @@
args.extend(self.getvalue(arg).get_forced_boxes())
newop.initarglist(args + inputargs[len(args):])
+ #print 'P: ', newop
current = len(self.optimizer.newoperations)
self.emit_operation(newop)
+ # FIXME: force_lazy_setfield in heap.py may reorder last ops
+ if current > 0:
+ current -= 1
+
for op in self.optimizer.newoperations[current:]:
+ #print 'E: ', op, self.optimizer.newoperations.index(op)
if op.is_guard():
descr = op.getdescr()
assert isinstance(descr, ResumeGuardDescr)
@@ -83,13 +90,18 @@
args = op.getarglist()
if op.is_guard():
args = args + op.getfailargs()
+ #if op.getopnum() == rop.SETFIELD_GC:
+ # import pdb; pdb.set_trace()
for a in args:
if not isinstance(a, Const) and a in self.optimizer.values:
v = self.getvalue(a)
if v.fromstart and a not in inputargs:
inputargs.append(a)
- if op.getopnum() == rop.JUMP:
- op.initarglist(op.getarglist() + [argmap[a]])
+ jmp = self.optimizer.newoperations[-1]
+ if jmp.getopnum() == rop.JUMP:
+ newval = self.getvalue(argmap[a])
+ newarg = newval.get_forced_boxes()
+ jmp.initarglist(jmp.getarglist() + newarg)
return inputargs
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Thu Oct 14 21:49:24 2010
@@ -8,7 +8,7 @@
import pypy.jit.metainterp.optimizeopt.virtualize as virtualize
from pypy.jit.metainterp.optimizeopt import optimize_loop_1
from pypy.jit.metainterp.optimizeutil import InvalidLoop
-from pypy.jit.metainterp.history import AbstractDescr, ConstInt, BoxInt
+from pypy.jit.metainterp.history import AbstractDescr, ConstInt, BoxInt, TreeLoop
from pypy.jit.metainterp.jitprof import EmptyProfiler
from pypy.jit.metainterp import executor, compile, resume, history
from pypy.jit.metainterp.resoperation import rop, opname, ResOperation
@@ -214,6 +214,8 @@
op.setfailargs(boxes)
def __eq__(self, other):
return type(self) is type(other) # xxx obscure
+ def clone_if_mutable(self):
+ return self
def _sortboxes(boxes):
_kind2count = {history.INT: 1, history.REF: 2, history.FLOAT: 3}
@@ -254,12 +256,15 @@
loop.token.specnodes = self.unpack_specnodes(spectext)
#
self.loop = loop
+ loop.preamble = TreeLoop('preamble')
metainterp_sd = FakeMetaInterpStaticData(self.cpu)
if hasattr(self, 'vrefinfo'):
metainterp_sd.virtualref_info = self.vrefinfo
optimize_loop_1(metainterp_sd, loop)
#
expected = self.parse(optops)
+ print
+ print "Ops: "
print '\n'.join([str(o) for o in loop.operations])
self.assert_equal(loop, expected)
@@ -3559,14 +3564,10 @@
jump(p0, i22)
"""
expected = """
- [p0, i22]
- i1 = getfield_gc(p0, descr=valuedescr)
- i2 = int_gt(i1, i22)
- guard_false(i2) []
- i3 = int_add(i1, 1)
+ [p0, i22, i1]
i331 = force_token()
setfield_gc(p0, i1, descr=valuedescr)
- jump(p0, i22)
+ jump(p0, i22, i1)
"""
self.optimize_loop(ops, 'Not, Not', expected)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py Thu Oct 14 21:49:24 2010
@@ -2,7 +2,7 @@
from pypy.rpython.lltypesystem import lltype
from pypy.jit.metainterp import warmspot
from pypy.module.pypyjit.policy import PyPyJitPolicy
-from pypy.rlib.jit import OPTIMIZER_FULL
+from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_NO_PERFECTSPEC
def run_child(glob, loc):
@@ -35,5 +35,5 @@
warmspot.jittify_and_run(interp, graph, [], policy=policy,
listops=True, CPUClass=CPUClass,
backendopt=True, inline=True,
- optimizer=OPTIMIZER_FULL)
+ optimizer=OPTIMIZER_NO_PERFECTSPEC)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py Thu Oct 14 21:49:24 2010
@@ -38,25 +38,45 @@
## print t2 - t1
try:
+## from array import array
+
+## def coords(w,h):
+## y = 0
+## while y < h:
+## x = 0
+## while x < w:
+## yield x,y
+## x += 1
+## y += 1
+
+## def f(img):
+## sa=0
+## for x, y in coords(4,4):
+## sa += x * y
+## return sa
+
+## #img=array('h',(1,2,3,4))
+## print f(3)
from array import array
+ class Circular(array):
+ def __new__(cls):
+ self = array.__new__(cls, 'i', range(16))
+ return self
+ def __getitem__(self, i):
+ assert self.__len__() == 16
+ return array.__getitem__(self, i & 15)
- def coords(w,h):
- y = 0
- while y < h:
- x = 0
- while x < w:
- yield x,y
- x += 1
- y += 1
-
- def f(img):
- sa=0
- for x, y in coords(4,4):
- sa += x * y
+ def main():
+ buf = Circular()
+ i = 10
+ sa = 0
+ while i < 20:
+ sa += buf[i-2] + buf[i-1] + buf[i] + buf[i+1] + buf[i+2]
+ i += 1
return sa
- #img=array('h',(1,2,3,4))
- print f(3)
+ print main()
+
except Exception, e:
print "Exception: ", type(e)
print e
From afa at codespeak.net Thu Oct 14 21:58:36 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 14 Oct 2010 21:58:36 +0200 (CEST)
Subject: [pypy-svn] r77959 - pypy/branch/fast-forward/pypy/module/imp
Message-ID: <20101014195836.7A118282BEC@codespeak.net>
Author: afa
Date: Thu Oct 14 21:58:34 2010
New Revision: 77959
Modified:
pypy/branch/fast-forward/pypy/module/imp/__init__.py
pypy/branch/fast-forward/pypy/module/imp/importing.py
pypy/branch/fast-forward/pypy/module/imp/interp_imp.py
Log:
Add disabled code that *could* help with the import lock
being held while another thread forks the interpreter.
Let's see if a failing test in cpython tests suite needs it.
Modified: pypy/branch/fast-forward/pypy/module/imp/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/imp/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/imp/__init__.py Thu Oct 14 21:58:34 2010
@@ -34,3 +34,13 @@
appleveldefs = {
}
+
+ def __init__(self, space, *args):
+ "NOT_RPYTHON"
+ MixedModule.__init__(self, space, *args)
+ # from pypy.module.posix.interp_posix import add_fork_hook
+ # from pypy.module.imp import interp_imp
+ # add_fork_hook('before', interp_imp.acquire_lock)
+ # add_fork_hook('parent', interp_imp.release_lock)
+ # add_fork_hook('child', interp_imp.reinit_lock)
+
Modified: pypy/branch/fast-forward/pypy/module/imp/importing.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/imp/importing.py (original)
+++ pypy/branch/fast-forward/pypy/module/imp/importing.py Thu Oct 14 21:58:34 2010
@@ -580,6 +580,13 @@
self.lockowner = None
self.lock.release()
+ def reinit_lock(self):
+ # Called after fork() to ensure that newly created child
+ # processes do not share locks with the parent
+ self.lock = None
+ self.lockowner = None
+ self.lockcounter = 0
+
def getimportlock(space):
return space.fromcache(ImportRLock)
Modified: pypy/branch/fast-forward/pypy/module/imp/interp_imp.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/imp/interp_imp.py (original)
+++ pypy/branch/fast-forward/pypy/module/imp/interp_imp.py Thu Oct 14 21:58:34 2010
@@ -152,3 +152,7 @@
def release_lock(space):
if space.config.objspace.usemodules.thread:
importing.getimportlock(space).release_lock()
+
+def reinit_lock(space):
+ if space.config.objspace.usemodules.thread:
+ importing.getimportlock(space).reinit_lock()
From afa at codespeak.net Thu Oct 14 23:50:01 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 14 Oct 2010 23:50:01 +0200 (CEST)
Subject: [pypy-svn] r77960 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101014215001.79381282BEA@codespeak.net>
Author: afa
Date: Thu Oct 14 23:49:58 2010
New Revision: 77960
Added:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (contents, props changed)
Modified:
pypy/branch/fast-forward/pypy/module/_io/__init__.py
pypy/branch/fast-forward/pypy/module/_io/interp_io.py
pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
Log:
Start implementing _io.FileIO.
cheat and have io.open() use _pyio.open()...
Modified: pypy/branch/fast-forward/pypy/module/_io/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/__init__.py Thu Oct 14 23:49:58 2010
@@ -14,7 +14,7 @@
'_BufferedIOBase': 'interp_io.W_BufferedIOBase',
'_TextIOBase': 'interp_io.W_TextIOBase',
- 'FileIO': 'interp_io.W_FileIO',
+ 'FileIO': 'interp_fileio.W_FileIO',
'BytesIO': 'interp_io.W_BytesIO',
'StringIO': 'interp_stringio.W_StringIO',
'BufferedReader': 'interp_io.W_BufferedReader',
@@ -23,7 +23,7 @@
'BufferedRandom': 'interp_io.W_BufferedRandom',
'TextIOWrapper': 'interp_io.W_TextIOWrapper',
- 'open': 'space.w_None',
+ 'open': 'interp_io.open',
'UnsupportedOperation': 'space.w_None',
'IncrementalNewlineDecoder': 'space.w_None',
}
Added: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Thu Oct 14 23:49:58 2010
@@ -0,0 +1,129 @@
+from pypy.module._io.interp_io import W_RawIOBase
+from pypy.interpreter.typedef import TypeDef
+from pypy.interpreter.gateway import interp2app, unwrap_spec, Arguments
+from pypy.interpreter.baseobjspace import ObjSpace, W_Root
+from pypy.interpreter.error import OperationError, wrap_oserror2
+import os
+
+def _bad_mode(space):
+ raise OperationError(space.w_ValueError, space.wrap(
+ "Must have exactly one of read/write/append mode"))
+
+def decode_mode(spac, mode):
+ flags = 0
+ rwa = False
+ readable = False
+ writable = False
+ append = False
+ plus = False
+
+ for s in mode:
+ if s == 'r':
+ if rwa:
+ _bad_mode(space)
+ rwa = True
+ readable = True
+ elif s == 'w':
+ if rwa:
+ _bad_mode(space)
+ rwa = True
+ writable = True
+ flags |= os.O_CREAT | os.O_TRUNC
+ elif s == 'a':
+ if rwa:
+ _bad_mode(space)
+ rwa = 1
+ writable = True
+ flags |= os.O_CREAT
+ append = True
+ elif s == 'b':
+ pass
+ elif s == '+':
+ if plus:
+ _bad_mode(space)
+ readable = writable = True
+ plus = True
+ else:
+ raise OperationError(space.w_ValueError, space.wrap(
+ "invalid mode: %s" % (mode,)))
+
+ if not rwa:
+ _bad_mode(space)
+
+ if readable and writable:
+ flags |= os.O_RDWR
+ elif readable:
+ flags |= os.O_RDONLY
+ else:
+ flags |= os.O_WRONLY
+
+ if hasattr(os, 'O_BINARY'):
+ flags |= os.O_BINARY
+
+ if hasattr(os, 'O_APPEND') and append:
+ flags |= os.O_APPEND
+
+ return readable, writable, flags
+
+class W_FileIO(W_RawIOBase):
+ def __init__(self, space):
+ W_RawIOBase.__init__(self, space)
+ self.fd = -1
+ self.readable = False
+ self.writable = False
+ self.seekable = -1
+ self.closefd = True
+
+ @unwrap_spec(ObjSpace, W_Root, Arguments)
+ def descr_new(space, w_subtype, __args__):
+ self = space.allocate_instance(W_FileIO, w_subtype)
+ W_FileIO.__init__(self, space)
+ return space.wrap(self)
+
+ @unwrap_spec('self', ObjSpace, W_Root, str, int)
+ def descr_init(self, space, w_name, mode, closefd):
+ if space.isinstance_w(w_name, space.w_float):
+ raise OperationError(space.w_TypeError, space.wrap(
+ "integer argument expected, got float"))
+ try:
+ fd = space.int_w(w_name)
+ except OperationError, e:
+ pass
+ else:
+ if fd < 0:
+ raise OperationError(space.w_ValueError, space.wrap(
+ "negative file descriptor"))
+
+ self.readable, self.writable, flags = decode_mode(space, mode)
+
+ from pypy.module.posix.interp_posix import dispatch_filename, rposix
+ try:
+ self.fd = dispatch_filename(rposix.open)(
+ space, w_name, flags, 0666)
+ except OSError, e:
+ raise wrap_oserror2(space, e, w_fname)
+ self.closefd = bool(closefd)
+
+ def _check_closed(self, space):
+ if self.fd < 0:
+ raise OperationError(space.w_ValueError, space.wrap(
+ "I/O operation on closed file"))
+
+ @unwrap_spec('self', ObjSpace)
+ def readable_w(self, space):
+ self._check_closed(space)
+ return space.wrap(self.readable)
+
+ @unwrap_spec('self', ObjSpace)
+ def writable_w(self, space):
+ self._check_closed(space)
+ return space.wrap(self.writable)
+
+W_FileIO.typedef = TypeDef(
+ 'FileIO', W_RawIOBase.typedef,
+ __new__ = interp2app(W_FileIO.descr_new.im_func),
+ __init__ = interp2app(W_FileIO.descr_init),
+ readable = interp2app(W_FileIO.readable_w),
+ writable = interp2app(W_FileIO.writable_w),
+ )
+
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_io.py Thu Oct 14 23:49:58 2010
@@ -94,6 +94,22 @@
raise OperationError(space.w_StopIteration, space.w_None)
return w_line
+ @unwrap_spec('self', ObjSpace)
+ def isatty_w(self, space):
+ return space.w_False
+
+ @unwrap_spec('self', ObjSpace)
+ def readable_w(self, space):
+ return space.w_False
+
+ @unwrap_spec('self', ObjSpace)
+ def writable_w(self, space):
+ return space.w_False
+
+ @unwrap_spec('self', ObjSpace)
+ def seekable_w(self, space):
+ return space.w_False
+
W_IOBase.typedef = TypeDef(
'_IOBase',
__new__ = generic_new_descr(W_IOBase),
@@ -103,6 +119,10 @@
next = interp2app(W_IOBase.next_w),
close = interp2app(W_IOBase.close_w),
flush = interp2app(W_IOBase.flush_w),
+ isatty = interp2app(W_IOBase.isatty_w),
+ readable = interp2app(W_IOBase.readable_w),
+ writable = interp2app(W_IOBase.writable_w),
+ seekable = interp2app(W_IOBase.seekable_w),
closed = GetSetProperty(W_IOBase.closed_get_w),
)
@@ -128,12 +148,6 @@
__new__ = generic_new_descr(W_TextIOBase),
)
-class W_FileIO(W_RawIOBase):
- pass
-W_FileIO.typedef = TypeDef(
- 'FileIO', W_RawIOBase.typedef,
- )
-
class W_BytesIO(W_BufferedIOBase):
pass
W_BytesIO.typedef = TypeDef(
@@ -169,3 +183,12 @@
W_TextIOWrapper.typedef = TypeDef(
'TextIOWrapper', W_TextIOBase.typedef,
)
+
+ at unwrap_spec(ObjSpace, Arguments)
+def open(space, __args__):
+ # XXX cheat!
+ w_pyio = space.call_method(space.builtin, '__import__',
+ space.wrap("_pyio"))
+ w_func = space.getattr(w_pyio, space.wrap("open"))
+ return space.call_args(w_func, __args__)
+
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Thu Oct 14 23:49:58 2010
@@ -36,3 +36,14 @@
return ""
assert list(MyFile()) == ["line1", "line2"]
+
+class AppTestOpen:
+ def setup_class(cls):
+ from pypy.tool.udir import udir
+ tmpfile = udir.join('tmpfile').ensure()
+ cls.w_tmpfile = cls.space.wrap(str(tmpfile))
+
+ def test_open(self):
+ import io
+ f = io.open(self.tmpfile, "rb")
+ f.close()
From afa at codespeak.net Thu Oct 14 23:50:51 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 14 Oct 2010 23:50:51 +0200 (CEST)
Subject: [pypy-svn] r77961 - pypy/branch/fast-forward/lib_pypy
Message-ID: <20101014215051.4C79F282BEA@codespeak.net>
Author: afa
Date: Thu Oct 14 23:50:49 2010
New Revision: 77961
Modified:
pypy/branch/fast-forward/lib_pypy/_csv.py
Log:
csv.dialect attributes are read-only
Modified: pypy/branch/fast-forward/lib_pypy/_csv.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/_csv.py (original)
+++ pypy/branch/fast-forward/lib_pypy/_csv.py Thu Oct 14 23:50:49 2010
@@ -135,8 +135,8 @@
converter = attributes[name][1]
if converter:
value = converter(value)
-
- setattr(self, name, value)
+
+ setattr(self, '_' + name, value)
if not self.delimiter:
raise TypeError("delimiter must be set")
@@ -148,7 +148,16 @@
raise TypeError("lineterminator must be set")
return self
-
+
+ delimiter = property(lambda self: self._delimiter)
+ doublequote = property(lambda self: self._doublequote)
+ escapechar = property(lambda self: self._escapechar)
+ lineterminator = property(lambda self: self._lineterminator)
+ quotechar = property(lambda self: self._quotechar)
+ quoting = property(lambda self: self._quoting)
+ skipinitialspace = property(lambda self: self._skipinitialspace)
+ strict = property(lambda self: self._strict)
+
def _call_dialect(dialect_inst, kwargs):
return Dialect(dialect_inst, **kwargs)
From afa at codespeak.net Thu Oct 14 23:58:55 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 14 Oct 2010 23:58:55 +0200 (CEST)
Subject: [pypy-svn] r77962 - in pypy/branch/fast-forward:
lib-python/modified-2.7.0/test lib_pypy
Message-ID: <20101014215855.B381F282BEA@codespeak.net>
Author: afa
Date: Thu Oct 14 23:58:54 2010
New Revision: 77962
Added:
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_csv.py
- copied, changed from r77956, pypy/branch/fast-forward/lib-python/2.7.0/test/test_csv.py
Modified:
pypy/branch/fast-forward/lib_pypy/_csv.py
Log:
Fix, and relax the test a little
With CPython, delattr and setattr raise TypeError or AttributeError,
depending on the descriptor being a readonly member or a readonly property...
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_csv.py (from r77956, pypy/branch/fast-forward/lib-python/2.7.0/test/test_csv.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/test/test_csv.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_csv.py Thu Oct 14 23:58:54 2010
@@ -54,8 +54,10 @@
self.assertEqual(obj.dialect.skipinitialspace, False)
self.assertEqual(obj.dialect.strict, False)
# Try deleting or changing attributes (they are read-only)
- self.assertRaises(TypeError, delattr, obj.dialect, 'delimiter')
- self.assertRaises(TypeError, setattr, obj.dialect, 'delimiter', ':')
+ self.assertRaises((TypeError, AttributeError), delattr, obj.dialect,
+ 'delimiter')
+ self.assertRaises((TypeError, AttributeError), setattr, obj.dialect,
+ 'delimiter', ':')
self.assertRaises(AttributeError, delattr, obj.dialect, 'quoting')
self.assertRaises(AttributeError, setattr, obj.dialect,
'quoting', None)
Modified: pypy/branch/fast-forward/lib_pypy/_csv.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/_csv.py (original)
+++ pypy/branch/fast-forward/lib_pypy/_csv.py Thu Oct 14 23:58:54 2010
@@ -70,13 +70,14 @@
The Dialect type records CSV parsing and generation options."""
- __slots__ = ["delimiter", "doublequote", "escapechar", "lineterminator",
- "quotechar", "quoting", "skipinitialspace", "strict"]
+ __slots__ = ["_delimiter", "_doublequote", "_escapechar",
+ "_lineterminator", "_quotechar", "_quoting",
+ "_skipinitialspace", "_strict"]
def __new__(cls, dialect, **kwargs):
for name in kwargs:
- if name not in Dialect.__slots__:
+ if '_' + name not in Dialect.__slots__:
raise TypeError("unexpected keyword argument '%s'" %
(name,))
@@ -120,6 +121,7 @@
# Copy attributes
notset = object()
for name in Dialect.__slots__:
+ name = name[1:]
value = notset
if name in kwargs:
value = kwargs[name]
From afa at codespeak.net Fri Oct 15 00:12:58 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 15 Oct 2010 00:12:58 +0200 (CEST)
Subject: [pypy-svn] r77963 - pypy/branch/fast-forward/pypy/module/_io
Message-ID: <20101014221258.70FC6282BEA@codespeak.net>
Author: afa
Date: Fri Oct 15 00:12:56 2010
New Revision: 77963
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
Log:
Translation fix
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Fri Oct 15 00:12:56 2010
@@ -101,7 +101,7 @@
self.fd = dispatch_filename(rposix.open)(
space, w_name, flags, 0666)
except OSError, e:
- raise wrap_oserror2(space, e, w_fname)
+ raise wrap_oserror2(space, e, w_name)
self.closefd = bool(closefd)
def _check_closed(self, space):
From leuschel at codespeak.net Fri Oct 15 07:02:04 2010
From: leuschel at codespeak.net (leuschel at codespeak.net)
Date: Fri, 15 Oct 2010 07:02:04 +0200 (CEST)
Subject: [pypy-svn] r77964 - pypy/extradoc/talk/pepm2011
Message-ID: <20101015050204.34676282BEA@codespeak.net>
Author: leuschel
Date: Fri Oct 15 07:02:01 2010
New Revision: 77964
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
minor edit
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 15 07:02:01 2010
@@ -450,7 +450,7 @@
guard_true($i_{17}$)
jump($p_{15}$, $p_{10}$)
\end{lstlisting}
-\caption{Unoptimized Trace for the Simple Object Model}
+\caption{An Unoptimized Trace of the Example Interpreter}
\label{fig:unopt-trace}
\end{figure}
@@ -473,7 +473,7 @@
\item \lstinline{set} correspond to attribute writes.
\item \lstinline{guard_class} correspond to method calls and are followed by
the trace of the called method.
- \item \lstinline{int_add} and \lstinline{int_get} are integer addition and
+ \item \lstinline{int_add} and \lstinline{int_gt} are integer addition and
comparison (``greater than''), respectively.
\end{itemize}
@@ -1134,18 +1134,20 @@
partial evaluation:
Partially known data structures are built directly into Prolog (via unbound
-logic variables) and thus the optimization of partially static data structures
-was part of partial evaluation of Prolog programs from the beginning
+logic variables) and thus the treatment of partially static data structures
+was part of partial evaluation of Prolog programs from the early stages
\cite{lloyd_partial_1991}. One effect of unfolding in Prolog is that terms that
-are constructed and immediately matched again completely disappear in the
-residual program, which is equivalent to what our optimization does for an
+are constructed and immediately matched again, completely disappear in the
+residual program. This is similar to what our optimization does for an
imperative language.
- FP
-
+xxx:
partially static data structures: kenichi asai's thesis?
-xxx: relation to compile-time garbage collection \cite{mazur_practical_2001}; separation logic; John Hughes: type specialization
+xxx:
+Other related work is compile-time garbage collection \cite{mazur_practical_2001}, which
+tries to ....;
+ separation logic; John Hughes: type specialization
\section{Conclusion}
\label{sec:conclusion}
From hakanardo at codespeak.net Fri Oct 15 07:21:42 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Fri, 15 Oct 2010 07:21:42 +0200 (CEST)
Subject: [pypy-svn] r77965 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt
Message-ID: <20101015052142.8E16E282BEA@codespeak.net>
Author: hakanardo
Date: Fri Oct 15 07:21:40 2010
New Revision: 77965
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
Log:
translation fix
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Fri Oct 15 07:21:40 2010
@@ -78,8 +78,7 @@
self.emit_operation(newop)
# FIXME: force_lazy_setfield in heap.py may reorder last ops
- if current > 0:
- current -= 1
+ current = max(current-1, 0)
for op in self.optimizer.newoperations[current:]:
#print 'E: ', op, self.optimizer.newoperations.index(op)
From afa at codespeak.net Fri Oct 15 08:22:57 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 15 Oct 2010 08:22:57 +0200 (CEST)
Subject: [pypy-svn] r77966 - pypy/branch/fast-forward/pypy/module/_io
Message-ID: <20101015062257.1B0A9282BEA@codespeak.net>
Author: afa
Date: Fri Oct 15 08:22:55 2010
New Revision: 77966
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
Log:
Typos
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Fri Oct 15 08:22:55 2010
@@ -9,7 +9,7 @@
raise OperationError(space.w_ValueError, space.wrap(
"Must have exactly one of read/write/append mode"))
-def decode_mode(spac, mode):
+def decode_mode(space, mode):
flags = 0
rwa = False
readable = False
@@ -32,7 +32,7 @@
elif s == 'a':
if rwa:
_bad_mode(space)
- rwa = 1
+ rwa = True
writable = True
flags |= os.O_CREAT
append = True
From afa at codespeak.net Fri Oct 15 08:40:02 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 15 Oct 2010 08:40:02 +0200 (CEST)
Subject: [pypy-svn] r77967 - pypy/branch/fast-forward/pypy/module/_io
Message-ID: <20101015064002.A60F8282BEA@codespeak.net>
Author: afa
Date: Fri Oct 15 08:40:00 2010
New Revision: 77967
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
Log:
Try to fix translation again
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Fri Oct 15 08:40:00 2010
@@ -3,8 +3,12 @@
from pypy.interpreter.gateway import interp2app, unwrap_spec, Arguments
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
from pypy.interpreter.error import OperationError, wrap_oserror2
+from os import O_RDONLY, O_WRONLY, O_RDWR, O_CREAT, O_TRUNC
import os
+O_BINARY = getattr(os, "O_BINARY", 0)
+O_APPEND = getattr(os, "O_APPEND", 0)
+
def _bad_mode(space):
raise OperationError(space.w_ValueError, space.wrap(
"Must have exactly one of read/write/append mode"))
@@ -28,13 +32,13 @@
_bad_mode(space)
rwa = True
writable = True
- flags |= os.O_CREAT | os.O_TRUNC
+ flags |= O_CREAT | O_TRUNC
elif s == 'a':
if rwa:
_bad_mode(space)
rwa = True
writable = True
- flags |= os.O_CREAT
+ flags |= O_CREAT
append = True
elif s == 'b':
pass
@@ -51,17 +55,16 @@
_bad_mode(space)
if readable and writable:
- flags |= os.O_RDWR
+ flags |= O_RDWR
elif readable:
- flags |= os.O_RDONLY
+ flags |= O_RDONLY
else:
- flags |= os.O_WRONLY
+ flags |= O_WRONLY
- if hasattr(os, 'O_BINARY'):
- flags |= os.O_BINARY
+ flags |= O_BINARY
- if hasattr(os, 'O_APPEND') and append:
- flags |= os.O_APPEND
+ if append:
+ flags |= O_APPEND
return readable, writable, flags
From afa at codespeak.net Fri Oct 15 10:45:33 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 15 Oct 2010 10:45:33 +0200 (CEST)
Subject: [pypy-svn] r77968 -
pypy/branch/fast-forward/pypy/module/_hashlib/test
Message-ID: <20101015084533.EBA71282BEA@codespeak.net>
Author: afa
Date: Fri Oct 15 10:45:31 2010
New Revision: 77968
Modified:
pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py
Log:
Avoid __name mangling when running tests with -A
Modified: pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py (original)
+++ pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py Fri Oct 15 10:45:31 2010
@@ -38,7 +38,8 @@
c_hexdigest = hexdigest
# also test the pure Python implementation
- h = hashlib.__get_builtin_constructor(name)('')
+ py_new = getattr(hashlib, '__get_builtin_constructor')
+ h = py_new(name)('')
assert h.digest_size == expected_size
assert h.digestsize == expected_size
#
From afa at codespeak.net Fri Oct 15 11:08:43 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 15 Oct 2010 11:08:43 +0200 (CEST)
Subject: [pypy-svn] r77969 - pypy/branch/fast-forward/lib_pypy/_ctypes
Message-ID: <20101015090843.3608B282BEA@codespeak.net>
Author: afa
Date: Fri Oct 15 11:08:41 2010
New Revision: 77969
Modified:
pypy/branch/fast-forward/lib_pypy/_ctypes/array.py
Log:
Fix ctypes tests
Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/array.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/_ctypes/array.py (original)
+++ pypy/branch/fast-forward/lib_pypy/_ctypes/array.py Fri Oct 15 11:08:41 2010
@@ -109,15 +109,16 @@
def array_get_slice_params(self, index):
if index.step is not None:
raise TypeError("3 arg slices not supported (for no reason)")
+ check_bounds = hasattr(self, '_length_')
if index.start is not None:
start = index.start
- if start < 0:
+ if check_bounds and start < 0:
start = 0
else:
start = 0
if index.stop is not None:
stop = index.stop
- if stop > self._length_:
+ if check_bounds and stop > self._length_:
stop = self._length_
else:
stop = self._length_
From antocuni at codespeak.net Fri Oct 15 11:10:07 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri, 15 Oct 2010 11:10:07 +0200 (CEST)
Subject: [pypy-svn] r77970 - in pypy/branch/jitffi/pypy/jit/metainterp:
optimizeopt test
Message-ID: <20101015091007.804A4282BEA@codespeak.net>
Author: antocuni
Date: Fri Oct 15 11:10:06 2010
New Revision: 77970
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
Log:
- simplify even more the logic of the optimization: now I think it's really
easy to understand, and moreover it optimize nicely one more case, as the
new test proves
- reformat the tests to be visually nicer
Modified: pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py Fri Oct 15 11:10:06 2010
@@ -13,6 +13,7 @@
descr = None
def __init__(self, funcval, cpu, prepare_op):
+ self.funcval = funcval
self.opargs = []
argtypes, restype = self._get_signature(funcval)
self.descr = cpu.calldescrof_dynamic(argtypes, restype)
@@ -62,26 +63,31 @@
class OptFfiCall(Optimization):
def __init__(self):
- self.funcval = None
self.funcinfo = None
- def _get_oopspec(self, op):
- effectinfo = op.getdescr().get_extra_info()
- if effectinfo is not None:
- return effectinfo.oopspecindex
- return EffectInfo.OS_NONE
+ def begin_optimization(self, funcval, op):
+ self.rollback_maybe()
+ self.funcinfo = FuncInfo(funcval, self.optimizer.cpu, op)
- def rollback(self):
- self.emit_operation(self.funcinfo.prepare_op)
- for op in self.funcinfo.opargs:
- self.emit_operation(op)
- self.funcval = None
+ def commit_optimization(self):
self.funcinfo = None
- def optimize_default(self, op):
- if self.funcval:
- self.rollback()
- self.emit_operation(op)
+ def rollback_maybe(self):
+ if self.funcinfo is None:
+ return # nothing to rollback
+ #
+ # we immediately set funcinfo to None to prevent recursion when
+ # calling emit_op
+ funcinfo = self.funcinfo
+ self.funcinfo = None
+ self.emit_operation(funcinfo.prepare_op)
+ for op in funcinfo.opargs:
+ self.emit_operation(op)
+
+ def emit_operation(self, op):
+ # we cannot emit any operation during the optimization
+ self.rollback_maybe()
+ Optimization.emit_operation(self, op)
def optimize_CALL(self, op):
oopspec = self._get_oopspec(op)
@@ -97,43 +103,33 @@
optimize_CALL_MAY_FORCE = optimize_CALL
- def _get_funcval(self, op):
- funcval = self.getvalue(op.getarg(1))
- if self.funcval:
- if self.funcval is not funcval:
- self.rollback()
- return None
- if not funcval.is_constant():
- return None
- return funcval
-
def do_prepare_call(self, op):
+ self.rollback_maybe()
funcval = self._get_funcval(op)
- if not funcval:
- return op
- assert self.funcval is None # XXX: do something nice etc. etc.
- self.funcval = funcval
- self.funcinfo = FuncInfo(funcval, self.optimizer.cpu, op)
+ if not funcval.is_constant():
+ return op # cannot optimize
+ self.begin_optimization(funcval, op)
+ return None
def do_push_arg(self, op):
- if self.funcval is None:
- return op
funcval = self._get_funcval(op)
+ if not self.funcinfo or self.funcinfo.funcval is not funcval:
+ return op # cannot optimize
self.funcinfo.opargs.append(op)
+ return None
def do_call(self, op):
- if self.funcval is None:
- return op
funcval = self._get_funcval(op)
- info = self.funcinfo
+ if not self.funcinfo or self.funcinfo.funcval is not funcval:
+ return op # cannot optimize
funcsymval = self.getvalue(op.getarg(2))
arglist = [funcsymval.force_box()]
- for push_op in info.opargs:
+ for push_op in self.funcinfo.opargs:
argval = self.getvalue(push_op.getarg(2))
arglist.append(argval.force_box())
- newop = ResOperation(rop.CALL_MAY_FORCE, arglist, op.result, descr=info.descr)
- self.funcval = None
- self.funcinfo = None
+ newop = ResOperation(rop.CALL_MAY_FORCE, arglist, op.result,
+ descr=self.funcinfo.descr)
+ self.commit_optimization()
return newop
def propagate_forward(self, op):
@@ -143,6 +139,15 @@
func(self, op)
break
else:
- self.optimize_default(op)
+ self.emit_operation(op)
+
+ def _get_oopspec(self, op):
+ effectinfo = op.getdescr().get_extra_info()
+ if effectinfo is not None:
+ return effectinfo.oopspecindex
+ return EffectInfo.OS_NONE
+
+ def _get_funcval(self, op):
+ return self.getvalue(op.getarg(1))
optimize_ops = _findall(OptFfiCall, 'optimize_')
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py Fri Oct 15 11:10:06 2010
@@ -54,12 +54,16 @@
namespace = namespace.__dict__
+ # ----------------------------------------------------------------------
+ # this group of tests is the most important, as they represent the "real"
+ # cases you actually get when using rlib.libffi
+
def test_ffi_call_opt(self):
ops = """
[i0, f1]
- call(0, ConstPtr(func), descr=libffi_prepare)
- call(0, ConstPtr(func), i0, descr=libffi_push_arg)
- call(0, ConstPtr(func), f1, descr=libffi_push_arg)
+ call(0, ConstPtr(func), descr=libffi_prepare)
+ call(0, ConstPtr(func), i0, descr=libffi_push_arg)
+ call(0, ConstPtr(func), f1, descr=libffi_push_arg)
i3 = call_may_force(0, ConstPtr(func), 12345, descr=libffi_call)
guard_not_forced() []
guard_no_exception() []
@@ -77,9 +81,9 @@
def test_ffi_call_nonconst(self):
ops = """
[i0, f1, p2]
- call(0, p2, descr=libffi_prepare)
- call(0, p2, i0, descr=libffi_push_arg)
- call(0, p2, f1, descr=libffi_push_arg)
+ call(0, p2, descr=libffi_prepare)
+ call(0, p2, i0, descr=libffi_push_arg)
+ call(0, p2, f1, descr=libffi_push_arg)
i3 = call_may_force(0, p2, 12345, descr=libffi_call)
guard_not_forced() []
guard_no_exception() []
@@ -88,13 +92,17 @@
expected = ops
loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
+ # ----------------------------------------------------------------------
+ # in pratice, the situations described in these tests should never happen,
+ # but we still want to ensure correctness
+
def test_rollback_if_op_in_between(self):
ops = """
[i0, f1]
- call(0, ConstPtr(func), descr=libffi_prepare)
- call(0, ConstPtr(func), i0, descr=libffi_push_arg)
+ call(0, ConstPtr(func), descr=libffi_prepare)
+ call(0, ConstPtr(func), i0, descr=libffi_push_arg)
i1 = int_add(i0, 1)
- call(0, ConstPtr(func), f1, descr=libffi_push_arg)
+ call(0, ConstPtr(func), f1, descr=libffi_push_arg)
i3 = call_may_force(0, ConstPtr(func), 12345, descr=libffi_call)
guard_not_forced() []
guard_no_exception() []
@@ -106,17 +114,84 @@
def test_rollback_multiple_calls(self):
ops = """
[i0, i2, f1]
- call(0, ConstPtr(func), descr=libffi_prepare)
- call(0, ConstPtr(func), i0, descr=libffi_push_arg)
- call(0, ConstPtr(func2), descr=libffi_prepare) # culprit!
- call(0, ConstPtr(func), f1, descr=libffi_push_arg)
- i3 = call_may_force(0, ConstPtr(func), 12345, descr=libffi_call)
- call(0, ConstPtr(func2), i0, descr=libffi_push_arg)
- call(0, ConstPtr(func2), f1, descr=libffi_push_arg)
- i4 = call_may_force(0, ConstPtr(func), 12345, descr=libffi_call)
+ call(0, ConstPtr(func), descr=libffi_prepare)
+ call(0, ConstPtr(func), i0, descr=libffi_push_arg)
+ #
+ # this is the culprit!
+ call(0, ConstPtr(func2), descr=libffi_prepare)
+ #
+ call(0, ConstPtr(func), f1, descr=libffi_push_arg)
+ i3 = call_may_force(0, ConstPtr(func), 12345, descr=libffi_call)
+ guard_not_forced() []
+ guard_no_exception() []
+ call(0, ConstPtr(func2), i0, descr=libffi_push_arg)
+ call(0, ConstPtr(func2), f1, descr=libffi_push_arg)
+ i4 = call_may_force(0, ConstPtr(func2), 67890, descr=libffi_call)
+ guard_not_forced() []
+ guard_no_exception() []
+ jump(i3, i4, f1)
+ """
+ expected = ops
+ loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
+
+ def test_rollback_multiple_prepare(self):
+ ops = """
+ [i0, i2, f1]
+ call(0, ConstPtr(func), descr=libffi_prepare)
+ #
+ # this is the culprit!
+ call(0, ConstPtr(func2), descr=libffi_prepare)
+ #
+ call(0, ConstPtr(func), i0, descr=libffi_push_arg)
+ call(0, ConstPtr(func), f1, descr=libffi_push_arg)
+ i3 = call_may_force(0, ConstPtr(func), 12345, descr=libffi_call)
+ guard_not_forced() []
+ guard_no_exception() []
+ call(0, ConstPtr(func2), i0, descr=libffi_push_arg)
+ call(0, ConstPtr(func2), f1, descr=libffi_push_arg)
+ i4 = call_may_force(0, ConstPtr(func2), 67890, descr=libffi_call)
guard_not_forced() []
guard_no_exception() []
jump(i3, i4, f1)
"""
expected = ops
loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
+
+ def test_optimize_nested_call(self):
+ ops = """
+ [i0, i2, f1]
+ call(0, ConstPtr(func), descr=libffi_prepare)
+ #
+ # this "nested" call is nicely optimized
+ call(0, ConstPtr(func2), descr=libffi_prepare)
+ call(0, ConstPtr(func2), i0, descr=libffi_push_arg)
+ call(0, ConstPtr(func2), f1, descr=libffi_push_arg)
+ i4 = call_may_force(0, ConstPtr(func2), 67890, descr=libffi_call)
+ guard_not_forced() []
+ guard_no_exception() []
+ #
+ call(0, ConstPtr(func), i0, descr=libffi_push_arg)
+ call(0, ConstPtr(func), f1, descr=libffi_push_arg)
+ i3 = call_may_force(0, ConstPtr(func), 12345, descr=libffi_call)
+ guard_not_forced() []
+ guard_no_exception() []
+ jump(i3, i4, f1)
+ """
+ expected = """
+ [i0, i2, f1]
+ call(0, ConstPtr(func), descr=libffi_prepare)
+ #
+ # this "nested" call is nicely optimized
+ i4 = call_may_force(67890, i0, f1, descr=int_float__int)
+ guard_not_forced() []
+ guard_no_exception() []
+ #
+ call(0, ConstPtr(func), i0, descr=libffi_push_arg)
+ call(0, ConstPtr(func), f1, descr=libffi_push_arg)
+ i3 = call_may_force(0, ConstPtr(func), 12345, descr=libffi_call)
+ guard_not_forced() []
+ guard_no_exception() []
+ jump(i3, i4, f1)
+ """
+ loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
+
From afa at codespeak.net Fri Oct 15 11:40:36 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 15 Oct 2010 11:40:36 +0200 (CEST)
Subject: [pypy-svn] r77971 - in pypy/branch/fast-forward/lib_pypy: .
pypy_test
Message-ID: <20101015094036.C9E2B282BEA@codespeak.net>
Author: afa
Date: Fri Oct 15 11:40:35 2010
New Revision: 77971
Modified:
pypy/branch/fast-forward/lib_pypy/_md5.py
pypy/branch/fast-forward/lib_pypy/pypy_test/test_md5_extra.py
pypy/branch/fast-forward/lib_pypy/pypy_test/test_sha_extra.py
Log:
Try to fix tests
Modified: pypy/branch/fast-forward/lib_pypy/_md5.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/_md5.py (original)
+++ pypy/branch/fast-forward/lib_pypy/_md5.py Fri Oct 15 11:40:35 2010
@@ -375,11 +375,14 @@
digest_size = 16
-def md5(arg=None):
- """Same as new().
-
- For backward compatibility reasons, this is an alternative
- name for the new() function.
+def new(arg=None):
+ """Return a new md5 crypto object.
+ If arg is present, the method call update(arg) is made.
"""
- return new(arg)
+ crypto = MD5Type()
+ if arg:
+ crypto.update(arg)
+
+ return crypto
+
Modified: pypy/branch/fast-forward/lib_pypy/pypy_test/test_md5_extra.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/pypy_test/test_md5_extra.py (original)
+++ pypy/branch/fast-forward/lib_pypy/pypy_test/test_md5_extra.py Fri Oct 15 11:40:35 2010
@@ -57,7 +57,7 @@
d1h = m1.hexdigest()
# Use MD5 module in pure Python.
- m2 = pymd5.md5()
+ m2 = pymd5.new()
m2.update(message)
d2 = m2.digest()
d2h = m2.hexdigest()
@@ -203,7 +203,7 @@
m1.update(prefix1)
m1c = m1.copy()
- m2 = pymd5.md5()
+ m2 = pymd5.new()
m2.update(prefix1)
m2c = m2.copy()
@@ -222,7 +222,6 @@
def test_attributes():
assert pymd5.digest_size == 16
- assert pymd5.digestsize == 16
- assert pymd5.blocksize == 1
- assert pymd5.md5().digest_size == 16
- assert pymd5.md5().digestsize == 16
+ assert pymd5.new().digest_size == 16
+ assert pymd5.new().digestsize == 16
+ assert pymd5.new().block_size == 64
Modified: pypy/branch/fast-forward/lib_pypy/pypy_test/test_sha_extra.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/pypy_test/test_sha_extra.py (original)
+++ pypy/branch/fast-forward/lib_pypy/pypy_test/test_sha_extra.py Fri Oct 15 11:40:35 2010
@@ -4,11 +4,11 @@
# Publication 180-1, Secure Hash Standard, 1995 April 17
# http://www.itl.nist.gov/div897/pubs/fip180-1.htm
from __future__ import absolute_import
-from .. import _sha
+from .. import _sha as pysha
class TestSHA:
def check(self, data, digest):
- computed = sha.new(data).hexdigest()
+ computed = pysha.new(data).hexdigest()
assert computed == digest
def test_case_1(self):
@@ -25,8 +25,8 @@
def test_attributes():
- assert sha.digest_size == 20
- assert sha.digestsize == 20
- assert sha.blocksize == 1
- assert sha.sha().digest_size == 20
- assert sha.sha().digestsize == 20
+ assert pysha.digest_size == 20
+ assert pysha.digestsize == 20
+ assert pysha.blocksize == 1
+ assert pysha.new().digest_size == 20
+ assert pysha.new().digestsize == 20
From david at codespeak.net Fri Oct 15 11:51:26 2010
From: david at codespeak.net (david at codespeak.net)
Date: Fri, 15 Oct 2010 11:51:26 +0200 (CEST)
Subject: [pypy-svn] r77972 - in
pypy/branch/arm-backend/pypy/jit/backend/arm: . test
Message-ID: <20101015095126.23C4F282BEA@codespeak.net>
Author: david
Date: Fri Oct 15 11:51:24 2010
New Revision: 77972
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
Log:
Encode ARM imm data instructions
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Fri Oct 15 11:51:24 2010
@@ -10,33 +10,6 @@
self._data = alloc(1024)
self._pos = 0
- def ADD_ri(self, rt, rn, imm, cond=cond.AL):
- # XXX S bit
- self.write32(cond << 28
- | 2 << 24
- | 8 << 20
- | (rn & 0xF) << 16
- | (rt & 0xF) << 12
- | (imm & 0xFFF))
-
- def SUB_ri(self, rd, rn, imm=0, cond=cond.AL, s=0):
- self.write32(cond << 28
- | 9 << 22
- | (s & 0x1) << 20
- | (rn & 0xF) << 16
- | (rd & 0xF) << 12
- | (imm & 0xFFF))
-
- def MOV_ri(self, rt, imm=0, cond=cond.AL):
- # XXX Check the actual allowed size for imm
- # XXX S bit
- self.write32(cond << 28
- | 0x3 << 24
- | 0xA << 20
- #| 0x0 << 16
- | (rt & 0xF) << 12
- | (imm & 0xFFF))
-
def PUSH(self, regs, cond=cond.AL):
assert reg.sp not in regs
instr = self._encode_reg_list(cond << 28 | 0x92D << 16, regs)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py Fri Oct 15 11:51:24 2010
@@ -17,9 +17,7 @@
| (p & 0x1) << 24
| (u & 0x1) << 23
| (w & 0x1) << 21
- | (rn & 0xFF) << 16
- | (rt & 0xFF) << 12
- | (imm & 0xFFF))
+ | imm_operation(rt, rn, imm))
else:
def f(self, rt, rn, rm, imm=0, cond=cond.AL, s=0, shifttype=0):
p = 1
@@ -32,6 +30,31 @@
| (w & 0x1) << 21
| reg_operation(rt, rn, rm, imm, s, shifttype))
return f
+def define_data_proc_imm(name, table):
+ n = (0x1 << 25
+ | (table['op'] & 0x1F) << 20)
+ if table['result'] and table['base']:
+ def imm_func(self, rd, rn, imm=0, cond=cond.AL, s=0):
+ # XXX check condition on rn
+ self.write32(n
+ | cond << 28
+ | s << 20
+ | imm_operation(rd, rn, imm))
+ elif not table['base']:
+ def imm_func(self, rd, imm=0, cond=cond.AL, s=0):
+ # XXX check condition on rn
+ self.write32(n
+ | cond << 28
+ | s << 20
+ | imm_operation(rd, 0, imm))
+ else:
+ def imm_func(self, rn, imm=0, cond=cond.AL, s=0):
+ # XXX check condition on rn
+ self.write32(n
+ | cond << 28
+ | s << 20
+ | imm_operation(0, rn, imm))
+ return imm_func
def define_data_proc(name, table):
n = ((table['op1'] & 0x1F) << 20
@@ -69,6 +92,10 @@
| reg_operation(rd, rn, rm, imm, s, shifttype))
return f
+def imm_operation(rt, rn, imm):
+ return ((rn & 0xFF) << 16
+ | (rt & 0xFF) << 12
+ | (imm & 0xFFF))
def reg_operation(rt, rn, rm, imm, s, shifttype):
# XXX encode shiftype correctly
@@ -79,11 +106,16 @@
| (shifttype & 0x3) << 5
| (rm & 0xFF))
+def define_instruction(builder, key, val, target):
+ f = builder(key, val)
+ setattr(target, key, f)
+
def define_instructions(target):
for key, val in instructions.load_store.iteritems():
- f = define_load_store_func(key, val)
- setattr(target, key, f)
+ define_instruction(define_load_store_func, key, val, target)
for key, val in instructions.data_proc.iteritems():
- f = define_data_proc(key, val)
- setattr(target, key, f)
+ define_instruction(define_data_proc, key, val, target)
+
+ for key, val in instructions.data_proc_imm.iteritems():
+ define_instruction(define_data_proc_imm, key, val, target)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py Fri Oct 15 11:51:24 2010
@@ -32,3 +32,23 @@
#'RRX_ri': {'op1':0x1A, 'op2':0, 'op3':0x3, 'op2cond':'0', 'result':False, 'base':True},
'ROR_ri': {'op1':0x1A, 'op2':0x0, 'op3':0x3, 'op2cond':'!0', 'result':True, 'base':False},
}
+
+data_proc_imm = {
+ 'ADD_ri': {'op': 0, 'rncond':'', 'result':True, 'base':True},
+ 'EOR_ri': {'op': 0x2, 'rncond':'', 'result':True, 'base':True},
+ 'SUB_ri': {'op': 0x4, 'rncond':'!0xF', 'result':True, 'base':True},
+ #'ADR_ri': {'op': 0x4, 'rncond':'0xF', 'result':True, 'base':True},
+ 'RSB_ri': {'op': 0x6, 'rncond':'', 'result':True, 'base':True},
+ 'ADD_ri': {'op': 0x8, 'rncond':'!0xF', 'result':True, 'base':True},
+ 'ADC_ri': {'op': 0xA, 'rncond':'', 'result':True, 'base':True},
+ 'SBC_ri': {'op': 0xC, 'rncond':'', 'result':True, 'base':True},
+ 'RSC_ri': {'op': 0xE, 'rncond':'', 'result':True, 'base':True},
+ 'TST_ri': {'op': 0x11, 'rncond':'', 'result':False, 'base':True},
+ 'TEQ_ri': {'op': 0x13, 'rncond':'', 'result':False, 'base':True},
+ 'CMP_ri': {'op': 0x15, 'rncond':'', 'result':False, 'base':True},
+ 'CMN_ri': {'op': 0x17, 'rncond':'', 'result':False, 'base':True},
+ 'ORR_ri': {'op': 0x18, 'rncond':'', 'result':True, 'base':True},
+ 'MOV_ri': {'op': 0x1A, 'rncond':'', 'result':True, 'base':False},
+ 'BIC_ri': {'op': 0x1C, 'rncond':'', 'result':True, 'base':True},
+ 'MVN_ri': {'op': 0x1E, 'rncond':'', 'result':True, 'base':False},
+}
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Fri Oct 15 11:51:24 2010
@@ -120,17 +120,43 @@
self.cb = CodeBuilder()
def build_tests():
+ test_name = 'test_generated_%s'
for key, value in instructions.load_store.iteritems():
if value['imm']:
f = gen_test_imm_func
else:
f = gen_test_reg_func
- test = f(key, value)
- setattr(TestInstrCodeBuilderForGeneratedInstr, 'test_%s' % key, test)
+ build_test(f, key, value, test_name)
for key, value, in instructions.data_proc.iteritems():
- test = gen_test_data_reg_func(key, value)
- setattr(TestInstrCodeBuilderForGeneratedInstr, 'test_%s' % key, test)
+ build_test(gen_test_data_reg_func, key, value, test_name)
+
+ for key, value, in instructions.data_proc_imm.iteritems():
+ build_test(gen_test_data_proc_imm_func, key, value, test_name)
+
+# XXX refactor this functions
+
+def build_test(builder, key, value, test_name):
+ test = builder(key, value)
+ setattr(TestInstrCodeBuilderForGeneratedInstr, test_name % key, test)
+
+def gen_test_data_proc_imm_func(name, table):
+ if table['result'] and table['base']:
+ def f(self):
+ func = getattr(self.cb, name)
+ func(r.r3, r.r7, 23)
+ self.assert_equal('%s r3, r7, #23' % name[:name.index('_')])
+ elif not table['base']:
+ def f(self):
+ func = getattr(self.cb, name)
+ func(r.r3, 23)
+ self.assert_equal('%s r3, #23' % name[:name.index('_')])
+ else:
+ def f(self):
+ func = getattr(self.cb, name)
+ func(r.r3, 23)
+ self.assert_equal('%s r3, #23' % name[:name.index('_')])
+ return f
def gen_test_imm_func(name, table):
def f(self):
From leuschel at codespeak.net Fri Oct 15 11:54:20 2010
From: leuschel at codespeak.net (leuschel at codespeak.net)
Date: Fri, 15 Oct 2010 11:54:20 +0200 (CEST)
Subject: [pypy-svn] r77973 - pypy/extradoc/talk/pepm2011
Message-ID: <20101015095420.DDF66282BEA@codespeak.net>
Author: leuschel
Date: Fri Oct 15 11:54:18 2010
New Revision: 77973
Modified:
pypy/extradoc/talk/pepm2011/paper.bib
pypy/extradoc/talk/pepm2011/paper.tex
Log:
added reference to constructor specialisation
Modified: pypy/extradoc/talk/pepm2011/paper.bib
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.bib (original)
+++ pypy/extradoc/talk/pepm2011/paper.bib Fri Oct 15 11:54:18 2010
@@ -342,3 +342,13 @@
year = {2008},
pages = {123--139}
}
+ at inproceedings{Mogensen:PEPM93,
+ author = {Torben Mogensen},
+ title = {Constructor Specialization},
+ booktitle = {Proceedings of PEPM'93, the ACM Sigplan Symposium on Partial
+ Evaluation and Semantics-Based Program Manipulation},
+ year = 1993,
+ month = {June},
+ publisher = {ACM Press},
+ pages = {22--32},
+}
\ No newline at end of file
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 15 11:54:18 2010
@@ -1140,6 +1140,8 @@
are constructed and immediately matched again, completely disappear in the
residual program. This is similar to what our optimization does for an
imperative language.
+In functional programming this idea was introduced as
+ constructor specialisation by Mogensen \cite{Mogensen:PEPM93}.)
xxx:
partially static data structures: kenichi asai's thesis?
From leuschel at codespeak.net Fri Oct 15 13:23:46 2010
From: leuschel at codespeak.net (leuschel at codespeak.net)
Date: Fri, 15 Oct 2010 13:23:46 +0200 (CEST)
Subject: [pypy-svn] r77974 - pypy/extradoc/talk/pepm2011
Message-ID: <20101015112346.E1C73282BEA@codespeak.net>
Author: leuschel
Date: Fri Oct 15 13:23:43 2010
New Revision: 77974
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
some minor changes + requeusts
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 15 13:23:43 2010
@@ -876,6 +876,8 @@
recursively lifting its fields.
+XXX Apply the rules of the algorithm to parts of the sample trace
+
% subsection Optimizing Traces (end)
\subsection{Analysis of the Algorithm}
@@ -887,9 +889,10 @@
single pass over the list of operations. We can check that altough
recursively lifting a static object is not a constant-time operation,
the algorithm only takes a total time linear in the length of the trace.
-Moreover, it gives the ``best'' possible result within its constrains,
+Moreover, it gives the ``best'' possible result within its constraints,
+\marginpar{Why? What are the constraints ? Justify}
\eg in term of the number of residual operations. The
-algorithm itself is not particularly complex or innovative; our focus is
+algorithm itself is not particularly complex; our focus is
rather that \emph{in the context of tracing JITs} it is possible to find a
simple enough algorithm that still gives the best results.
@@ -921,6 +924,9 @@
\cfbolz{probably can be cut in case of space problems}
+XXX unclear what the relationship is to the technique; has it been used in the experiments??
+XXX Maybe move to discussion and related work section?
+
One problem to the successful application of the allocation removal
techniques described in the previous sections is the presence of
frame-introspection features in many dynamic languages and the fact
@@ -928,7 +934,7 @@
allow the programmer to get access to the frames object that the
interpreter uses to store local variables. This is a useful feature,
as it makes the implementation of a debugger possible in Python
-without needing much explicit support from the VM level. On the other
+without extra support from the VM. On the other
hand, it severely hinders the effectiveness of allocation removal,
because every time an object is stored into a local variable, it is
stored into the frame-object, which makes it escape.
@@ -1143,13 +1149,14 @@
In functional programming this idea was introduced as
constructor specialisation by Mogensen \cite{Mogensen:PEPM93}.)
-xxx:
-partially static data structures: kenichi asai's thesis?
+%xxx:
+%partially static data structures: kenichi asai's thesis?
xxx:
Other related work is compile-time garbage collection \cite{mazur_practical_2001}, which
tries to ....;
- separation logic; John Hughes: type specialization
+ separation logic
+ %; John Hughes: type specialization
\section{Conclusion}
\label{sec:conclusion}
From afa at codespeak.net Fri Oct 15 13:47:48 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 15 Oct 2010 13:47:48 +0200 (CEST)
Subject: [pypy-svn] r77975 -
pypy/branch/fast-forward/lib-python/modified-2.7.0
Message-ID: <20101015114748.D6A81282BEA@codespeak.net>
Author: afa
Date: Fri Oct 15 13:47:47 2010
New Revision: 77975
Added:
pypy/branch/fast-forward/lib-python/modified-2.7.0/subprocess.py
- copied, changed from r77967, pypy/branch/fast-forward/lib-python/2.7.0/subprocess.py
Log:
Don't rely on reference counting to close the non-heritable handle.
This avoids a deadlock on Windows, when stdout=PIPE.
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/subprocess.py (from r77967, pypy/branch/fast-forward/lib-python/2.7.0/subprocess.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/subprocess.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/subprocess.py Fri Oct 15 13:47:47 2010
@@ -792,7 +792,7 @@
elif stderr == PIPE:
errread, errwrite = _subprocess.CreatePipe(None, 0)
elif stderr == STDOUT:
- errwrite = c2pwrite
+ errwrite = c2pwrite.handle # pass id to not close it
elif isinstance(stderr, int):
errwrite = msvcrt.get_osfhandle(stderr)
else:
@@ -807,9 +807,13 @@
def _make_inheritable(self, handle):
"""Return a duplicate of handle, which is inheritable"""
- return _subprocess.DuplicateHandle(_subprocess.GetCurrentProcess(),
+ dupl = _subprocess.DuplicateHandle(_subprocess.GetCurrentProcess(),
handle, _subprocess.GetCurrentProcess(), 0, 1,
_subprocess.DUPLICATE_SAME_ACCESS)
+ # If the initial handle was obtained with CreatePipe, close it.
+ if not isinstance(handle, int):
+ handle.Close()
+ return dupl
def _find_w9xpopen(self):
From cfbolz at codespeak.net Fri Oct 15 14:06:00 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 15 Oct 2010 14:06:00 +0200 (CEST)
Subject: [pypy-svn] r77976 - pypy/extradoc/talk/pepm2011
Message-ID: <20101015120600.B22B1282BEA@codespeak.net>
Author: cfbolz
Date: Fri Oct 15 14:05:59 2010
New Revision: 77976
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
tone down claims a bit
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 15 14:05:59 2010
@@ -886,20 +886,17 @@
While we do not offer a formal proof of it, it should be relatively clear
that the algorithm presented above is sound: it works by delaying (and
often completely removing) some operations. The algorithm runs in a
-single pass over the list of operations. We can check that altough
+single pass over the list of operations. We can check that although
recursively lifting a static object is not a constant-time operation,
the algorithm only takes a total time linear in the length of the trace.
-Moreover, it gives the ``best'' possible result within its constraints,
-\marginpar{Why? What are the constraints ? Justify}
-\eg in term of the number of residual operations. The
-algorithm itself is not particularly complex; our focus is
+The algorithm itself is not particularly complex; our focus is
rather that \emph{in the context of tracing JITs} it is possible to find a
-simple enough algorithm that still gives the best results.
+simple enough algorithm that still gives very good results.
Note in particular that objects in category 1 (\ie the ones that do
not escape) are completely removed; moreover, objects in category 2
(\ie escaping) are still partially dealt with: if such an object
-escapes later than its creation point, all the operations inbetween that
+escapes later than its creation point, all the operations in between that
involve the object are removed.
The optimization is particularly effective for chains of operations.
From cfbolz at codespeak.net Fri Oct 15 14:21:38 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 15 Oct 2010 14:21:38 +0200 (CEST)
Subject: [pypy-svn] r77977 - pypy/extradoc/talk/pepm2011
Message-ID: <20101015122138.829F1282BEA@codespeak.net>
Author: cfbolz
Date: Fri Oct 15 14:21:37 2010
New Revision: 77977
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
kill the section about frames, as michael thinks its too confusing. Instead
mention frames minimally in the evaluation section.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 15 14:21:37 2010
@@ -188,10 +188,7 @@
objects that are allocated in the trace \emph{static,} which
means that they do not occur any more in the optimized trace. This technique is
informally described in Section~\ref{sec:statics}; a more formal description is
-given in Section~\ref{sec:formal}.
-
-In Section~\ref{sec:frames} we describe some supporting techniques that are not
-central to the approach, but are essential to improve the results. The introduced
+given in Section~\ref{sec:formal}. The introduced
techniques are evaluated in Section~\ref{sec:Evaluation} using PyPy's Python
interpreter as a case study.
@@ -903,10 +900,8 @@
For example, it is typical for an interpreter to generate sequences of
writes-followed-by-reads, where one interpreted opcode writes to some
object's field and the next interpreted opcode reads it back, possibly
-dispatching on the type of the object created just before. In the case
-of PyPy's Python interpreter, this optimization can even remove the
-allocation of all intermediate frames that occur in the interpreter,
-corresponding to all calls that have been inlined in the trace.
+dispatching on the type of the object created just before. A typical example
+would be chains of arithmetic operations.
% subsection Analysis of the Algorithm (end)
@@ -916,44 +911,28 @@
% section Escape Analysis in a Tracing JIT (end)
-\section{Taming Frame Objects}
-\label{sec:frames}
-
-\cfbolz{probably can be cut in case of space problems}
-
-XXX unclear what the relationship is to the technique; has it been used in the experiments??
-XXX Maybe move to discussion and related work section?
-
-One problem to the successful application of the allocation removal
-techniques described in the previous sections is the presence of
-frame-introspection features in many dynamic languages and the fact
-that frames are heap allocated. Languages such as Python and Smalltalk
-allow the programmer to get access to the frames object that the
-interpreter uses to store local variables. This is a useful feature,
-as it makes the implementation of a debugger possible in Python
-without extra support from the VM. On the other
-hand, it severely hinders the effectiveness of allocation removal,
-because every time an object is stored into a local variable, it is
-stored into the frame-object, which makes it escape.
-
-This problem is solved by making it possible to the interpreter author
-to add some hints into the source code to declare instances of one
-class as frame objects needing special treatment. The JIT will then
-fill these objects only lazily when they are actually accessed (\eg
-because a debugger is used) using values from JIT-emitted code runtime
-locations (typically the CPU stack). Therefore in the common case,
-nothing is stored into the frame objects, making the problem of too
-much escaping go away. This special handling of frame objects is a common approach in VM implementations
-\cite{miranda_context_1999,andreas_gal_trace-based_2009}. The only novelty in our approach lays in
-its generality: all the delicate support code for this is generated, as opposed to
-most other JITs, which are just specifically written for one particular
-language.
-
-% section Taming Frame Objects (end)
-
-\section{Evaluation}
+\section{Implementation and Evaluation}
\label{sec:Evaluation}
+The allocation removal techniques described in this paper were implemented in
+the optimizer of PyPy's tracing JIT. The optimization is independent of which
+interpreter a JIT is generated for. There are some practical issues beyond the
+techniques described in this paper. The actual implementation needs to deal with
+more operations than described in Section~\ref{sec:formal}, for example to
+also support static arrays in addition to static objects. The implementation of
+this optimization is about 400 lines of RPython code.
+
+A further complication is that most interpreters written with PyPy use
+heap-allocated frame objects to store local variables. Those severely hinders
+the effectiveness of allocation removal, because every time an object is stored
+into a local variable, it is stored into the frame-object, which makes it
+escape. We implemented a general technique to treat such frames objects in a
+special way to solve this problem. This is a common approach in VM
+implementations \cite{miranda_context_1999,andreas_gal_trace-based_2009}. The
+only novelty in our approach lays in its generality: most other JITs, are just
+specifically written for one particular language instead of being generated from
+an interpreter.
+
To evaluate the effectiveness of our allocation removal algorithm, we look at
the effectiveness when used in the tracing JIT of PyPy's Python interpreter. The
benchmarks we used are small-to-medium Python programs, some synthetic
From cfbolz at codespeak.net Fri Oct 15 14:28:37 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 15 Oct 2010 14:28:37 +0200 (CEST)
Subject: [pypy-svn] r77978 - pypy/extradoc/talk/pepm2011
Message-ID: <20101015122837.5ADFE282BEA@codespeak.net>
Author: cfbolz
Date: Fri Oct 15 14:28:35 2010
New Revision: 77978
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
- shorten a bit
- mention size of python interpreter
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 15 14:28:35 2010
@@ -923,18 +923,21 @@
this optimization is about 400 lines of RPython code.
A further complication is that most interpreters written with PyPy use
-heap-allocated frame objects to store local variables. Those severely hinders
+heap-allocated frame objects to store local variables. Those severely hinder
the effectiveness of allocation removal, because every time an object is stored
into a local variable, it is stored into the frame-object, which makes it
-escape. We implemented a general technique to treat such frames objects in a
+escape. We implemented a technique to treat such frames objects in a
special way to solve this problem. This is a common approach in VM
-implementations \cite{miranda_context_1999,andreas_gal_trace-based_2009}. The
-only novelty in our approach lays in its generality: most other JITs, are just
-specifically written for one particular language instead of being generated from
-an interpreter.
+implementations \cite{miranda_context_1999,andreas_gal_trace-based_2009}, the
+novelty of our approach is that we generalized it enough to be usable for
+different interpreter.
+
+To evaluate our allocation removal algorithm, we look at the effectiveness when
+used in the generated tracing JIT of PyPy's Python interpreter. This interpreter
+is a full implementation of Python 2.5 language semantics and is about 30'000
+lines of code.
-To evaluate the effectiveness of our allocation removal algorithm, we look at
-the effectiveness when used in the tracing JIT of PyPy's Python interpreter. The
+The
benchmarks we used are small-to-medium Python programs, some synthetic
benchmarks, some real applications.\footnote{All the source code of the
benchmarks can be found at \texttt{http://codespeak.net/svn/pypy/benchmarks/}.
@@ -947,7 +950,7 @@
Furthermore there are the following benchmarks:
\begin{itemize}
- \item \textbf{crypto\_pyaes}: AES implementation.
+ \item \textbf{crypto\_pyaes}: An AES implementation.
\item \textbf{django}: The templating engine of the Django web
framework\footnote{\texttt{http://www.djangoproject.com/}}.
\item \textbf{go}: A Monte-Carlo Go
From arigo at codespeak.net Fri Oct 15 14:37:19 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 14:37:19 +0200 (CEST)
Subject: [pypy-svn] r77979 - in pypy/branch/leak-finder/pypy: . annotation
jit/backend/llsupport jit/backend/test jit/backend/x86
jit/backend/x86/test jit/metainterp jit/metainterp/test
module/cpyext/test rlib/test rpython rpython/lltypesystem
rpython/lltypesystem/test rpython/memory
rpython/memory/gctransform rpython/test tool tool/test
translator/c/test
Message-ID: <20101015123719.3F83E36E0C3@codespeak.net>
Author: arigo
Date: Fri Oct 15 14:37:16 2010
New Revision: 77979
Added:
pypy/branch/leak-finder/pypy/tool/leakfinder.py
pypy/branch/leak-finder/pypy/tool/test/test_leakfinder.py
Modified:
pypy/branch/leak-finder/pypy/annotation/builtin.py
pypy/branch/leak-finder/pypy/conftest.py
pypy/branch/leak-finder/pypy/jit/backend/llsupport/gc.py
pypy/branch/leak-finder/pypy/jit/backend/llsupport/llmodel.py
pypy/branch/leak-finder/pypy/jit/backend/test/runner_test.py
pypy/branch/leak-finder/pypy/jit/backend/x86/assembler.py
pypy/branch/leak-finder/pypy/jit/backend/x86/regalloc.py
pypy/branch/leak-finder/pypy/jit/backend/x86/runner.py
pypy/branch/leak-finder/pypy/jit/backend/x86/test/test_assembler.py
pypy/branch/leak-finder/pypy/jit/metainterp/test/test_basic.py
pypy/branch/leak-finder/pypy/jit/metainterp/virtualref.py
pypy/branch/leak-finder/pypy/module/cpyext/test/test_cpyext.py
pypy/branch/leak-finder/pypy/rlib/test/test_rdynload.py
pypy/branch/leak-finder/pypy/rlib/test/test_rzlib.py
pypy/branch/leak-finder/pypy/rpython/llinterp.py
pypy/branch/leak-finder/pypy/rpython/lltypesystem/llmemory.py
pypy/branch/leak-finder/pypy/rpython/lltypesystem/lltype.py
pypy/branch/leak-finder/pypy/rpython/lltypesystem/rclass.py
pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_ll2ctypes.py
pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_llmemory.py
pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_lltype.py
pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_rffi.py
pypy/branch/leak-finder/pypy/rpython/memory/gctransform/transform.py
pypy/branch/leak-finder/pypy/rpython/memory/gcwrapper.py
pypy/branch/leak-finder/pypy/rpython/memory/support.py
pypy/branch/leak-finder/pypy/rpython/rbuiltin.py
pypy/branch/leak-finder/pypy/rpython/test/test_llinterp.py
pypy/branch/leak-finder/pypy/rpython/test/test_nongc.py
pypy/branch/leak-finder/pypy/rpython/test/test_rptr.py
pypy/branch/leak-finder/pypy/translator/c/test/test_lltyped.py
Log:
In-progress: remove the 'self.mallocs' from llinterp, and instead
use systematically the one from lltype.py introduced around r73335.
Modified: pypy/branch/leak-finder/pypy/annotation/builtin.py
==============================================================================
--- pypy/branch/leak-finder/pypy/annotation/builtin.py (original)
+++ pypy/branch/leak-finder/pypy/annotation/builtin.py Fri Oct 15 14:37:16 2010
@@ -423,7 +423,7 @@
from pypy.annotation.model import SomePtr
from pypy.rpython.lltypesystem import lltype
-def malloc(s_T, s_n=None, s_flavor=None, s_zero=None):
+def malloc(s_T, s_n=None, s_flavor=None, s_zero=None, s_track_allocation=None):
assert (s_n is None or s_n.knowntype == int
or issubclass(s_n.knowntype, pypy.rlib.rarithmetic.base_int))
assert s_T.is_constant()
@@ -438,13 +438,15 @@
r = SomePtr(lltype.typeOf(p))
else:
assert s_flavor.is_constant()
+ assert s_track_allocation is None or s_track_allocation.is_constant()
# not sure how to call malloc() for the example 'p' in the
# presence of s_extraargs
r = SomePtr(lltype.Ptr(s_T.const))
return r
-def free(s_p, s_flavor):
+def free(s_p, s_flavor, s_track_allocation=None):
assert s_flavor.is_constant()
+ assert s_track_allocation is None or s_track_allocation.is_constant()
# same problem as in malloc(): some flavors are not easy to
# malloc-by-example
#T = s_p.ll_ptrtype.TO
Modified: pypy/branch/leak-finder/pypy/conftest.py
==============================================================================
--- pypy/branch/leak-finder/pypy/conftest.py (original)
+++ pypy/branch/leak-finder/pypy/conftest.py Fri Oct 15 14:37:16 2010
@@ -7,6 +7,7 @@
from inspect import isclass, getmro
from pypy.tool.udir import udir
from pypy.tool.autopath import pypydir
+from pypy.tool import leakfinder
# pytest settings
pytest_plugins = "resultlog",
@@ -354,7 +355,14 @@
def runtest(self):
try:
- super(IntTestFunction, self).runtest()
+ leakfinder.start_tracking_allocations()
+ try:
+ super(IntTestFunction, self).runtest()
+ finally:
+ if leakfinder.TRACK_ALLOCATIONS:
+ leaks = leakfinder.stop_tracking_allocations(False)
+ else:
+ leaks = None # stop_tracking_allocations() already called
except OperationError, e:
check_keyboard_interrupt(e)
raise
@@ -373,6 +381,8 @@
_pygame_imported = True
assert option.view, ("should not invoke Pygame "
"if conftest.option.view is False")
+ if leaks: # check for leaks, but only if the test passed so far
+ raise leakfinder.MallocMismatch(leaks)
class AppTestFunction(PyPyTestFunction):
def _prunetraceback(self, traceback):
Modified: pypy/branch/leak-finder/pypy/jit/backend/llsupport/gc.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/backend/llsupport/gc.py (original)
+++ pypy/branch/leak-finder/pypy/jit/backend/llsupport/gc.py Fri Oct 15 14:37:16 2010
@@ -158,7 +158,7 @@
# used to avoid too many duplications in the GCREF_LISTs.
self.hashtable = lltype.malloc(self.HASHTABLE,
self.HASHTABLE_SIZE+1,
- flavor='raw')
+ flavor='raw', track_allocation=False)
dummy = lltype.direct_ptradd(lltype.direct_arrayitems(self.hashtable),
self.HASHTABLE_SIZE)
dummy = llmemory.cast_ptr_to_adr(dummy)
@@ -252,14 +252,15 @@
def _enlarge_gcmap(self):
newlength = 250 + self._gcmap_maxlength * 2
- newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw')
+ newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw',
+ track_allocation=False)
oldgcmap = self._gcmap
for i in range(self._gcmap_curlength):
newgcmap[i] = oldgcmap[i]
self._gcmap = newgcmap
self._gcmap_maxlength = newlength
if oldgcmap:
- lltype.free(oldgcmap, flavor='raw')
+ lltype.free(oldgcmap, flavor='raw', track_allocation=False)
def get_basic_shape(self, is_64_bit=False):
# XXX: Should this code even really know about stack frame layout of
@@ -308,7 +309,8 @@
# them inside bigger arrays) and we never try to share them.
length = len(shape)
compressed = lltype.malloc(self.CALLSHAPE_ARRAY, length,
- flavor='raw')
+ flavor='raw',
+ track_allocation=False) # memory leak
for i in range(length):
compressed[length-1-i] = rffi.cast(rffi.UCHAR, shape[i])
return llmemory.cast_ptr_to_adr(compressed)
Modified: pypy/branch/leak-finder/pypy/jit/backend/llsupport/llmodel.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/backend/llsupport/llmodel.py (original)
+++ pypy/branch/leak-finder/pypy/jit/backend/llsupport/llmodel.py Fri Oct 15 14:37:16 2010
@@ -82,7 +82,8 @@
# read back by the machine code reading at the address given by
# pos_exception() and pos_exc_value().
_exception_emulator = lltype.malloc(rffi.CArray(lltype.Signed), 2,
- zero=True, flavor='raw')
+ zero=True, flavor='raw',
+ immortal=True)
self._exception_emulator = _exception_emulator
def _store_exception(lle):
Modified: pypy/branch/leak-finder/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/branch/leak-finder/pypy/jit/backend/test/runner_test.py Fri Oct 15 14:37:16 2010
@@ -1304,6 +1304,7 @@
descr=fd)
res = self.execute_operation(get_op, [s_box], 'int', descr=fd)
assert res.getint() == 32
+ lltype.free(s, flavor='raw')
def test_new_with_vtable(self):
cpu = self.cpu
Modified: pypy/branch/leak-finder/pypy/jit/backend/x86/assembler.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/backend/x86/assembler.py (original)
+++ pypy/branch/leak-finder/pypy/jit/backend/x86/assembler.py Fri Oct 15 14:37:16 2010
@@ -249,7 +249,8 @@
def _build_float_constants(self):
# 44 bytes: 32 bytes for the data, and up to 12 bytes for alignment
- addr = lltype.malloc(rffi.CArray(lltype.Char), 44, flavor='raw')
+ addr = lltype.malloc(rffi.CArray(lltype.Char), 44, flavor='raw',
+ track_allocation=False)
if not we_are_translated():
self._keepalive_malloced_float_consts = addr
float_constants = rffi.cast(lltype.Signed, addr)
@@ -399,7 +400,8 @@
funcname = "" % len(self.loop_run_counters)
# invent the counter, so we don't get too confused
if self._debug:
- struct = lltype.malloc(DEBUG_COUNTER, flavor='raw')
+ struct = lltype.malloc(DEBUG_COUNTER, flavor='raw',
+ track_allocation=False) # known to leak
struct.i = 0
self.loop_run_counters.append((len(self.loop_run_counters), struct))
return funcname
Modified: pypy/branch/leak-finder/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/branch/leak-finder/pypy/jit/backend/x86/regalloc.py Fri Oct 15 14:37:16 2010
@@ -70,8 +70,9 @@
def _get_new_array(self):
n = self.BASE_CONSTANT_SIZE
+ # known to leak
self.cur_array = lltype.malloc(rffi.CArray(lltype.Float), n,
- flavor='raw')
+ flavor='raw', track_allocation=False)
self.cur_array_free = n
_get_new_array._dont_inline_ = True
Modified: pypy/branch/leak-finder/pypy/jit/backend/x86/runner.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/backend/x86/runner.py (original)
+++ pypy/branch/leak-finder/pypy/jit/backend/x86/runner.py Fri Oct 15 14:37:16 2010
@@ -113,7 +113,8 @@
return CPU386.cast_adr_to_int(adr)
all_null_registers = lltype.malloc(rffi.LONGP.TO, 24,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
def force(self, addr_of_force_index):
TP = rffi.CArrayPtr(lltype.Signed)
Modified: pypy/branch/leak-finder/pypy/jit/backend/x86/test/test_assembler.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/backend/x86/test/test_assembler.py (original)
+++ pypy/branch/leak-finder/pypy/jit/backend/x86/test/test_assembler.py Fri Oct 15 14:37:16 2010
@@ -81,7 +81,8 @@
# also test rebuild_faillocs_from_descr(), which should not
# reproduce the holes at all
- bytecode = lltype.malloc(rffi.UCHARP.TO, len(mc.content), flavor='raw')
+ bytecode = lltype.malloc(rffi.UCHARP.TO, len(mc.content), flavor='raw',
+ immortal=True)
for i in range(len(mc.content)):
assert 0 <= mc.content[i] <= 255
bytecode[i] = rffi.cast(rffi.UCHAR, mc.content[i])
@@ -115,7 +116,8 @@
assert withfloats
value = random.random() - 0.5
# make sure it fits into 64 bits
- tmp = lltype.malloc(rffi.LONGP.TO, 2, flavor='raw')
+ tmp = lltype.malloc(rffi.LONGP.TO, 2, flavor='raw',
+ track_allocation=False)
rffi.cast(rffi.DOUBLEP, tmp)[0] = value
return rffi.cast(rffi.DOUBLEP, tmp)[0], tmp[0], tmp[1]
@@ -152,10 +154,12 @@
# prepare the expected target arrays, the descr_bytecode,
# the 'registers' and the 'stack' arrays according to 'content'
- xmmregisters = lltype.malloc(rffi.LONGP.TO, 16+ACTUAL_CPU.NUM_REGS+1, flavor='raw')
+ xmmregisters = lltype.malloc(rffi.LONGP.TO, 16+ACTUAL_CPU.NUM_REGS+1,
+ flavor='raw', immortal=True)
registers = rffi.ptradd(xmmregisters, 16)
stacklen = baseloc + 10
- stack = lltype.malloc(rffi.LONGP.TO, stacklen, flavor='raw')
+ stack = lltype.malloc(rffi.LONGP.TO, stacklen, flavor='raw',
+ immortal=True)
expected_ints = [0] * len(content)
expected_ptrs = [lltype.nullptr(llmemory.GCREF.TO)] * len(content)
expected_floats = [0.0] * len(content)
@@ -221,7 +225,7 @@
descr_bytecode.append(0x00)
descr_bytecode.append(0xCC) # end marker
descr_bytes = lltype.malloc(rffi.UCHARP.TO, len(descr_bytecode),
- flavor='raw')
+ flavor='raw', immortal=True)
for i in range(len(descr_bytecode)):
assert 0 <= descr_bytecode[i] <= 255
descr_bytes[i] = rffi.cast(rffi.UCHAR, descr_bytecode[i])
Modified: pypy/branch/leak-finder/pypy/jit/metainterp/test/test_basic.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/metainterp/test/test_basic.py (original)
+++ pypy/branch/leak-finder/pypy/jit/metainterp/test/test_basic.py Fri Oct 15 14:37:16 2010
@@ -1751,7 +1751,7 @@
c = bool(p1)
d = not bool(p2)
return 1000*a + 100*b + 10*c + d
- prebuilt = [lltype.malloc(TP, flavor='raw')] * 2
+ prebuilt = [lltype.malloc(TP, flavor='raw', immortal=True)] * 2
expected = f(0, 1)
assert self.interp_operations(f, [0, 1]) == expected
Modified: pypy/branch/leak-finder/pypy/jit/metainterp/virtualref.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/metainterp/virtualref.py (original)
+++ pypy/branch/leak-finder/pypy/jit/metainterp/virtualref.py Fri Oct 15 14:37:16 2010
@@ -16,7 +16,8 @@
('virtualref_index', lltype.Signed),
('forced', rclass.OBJECTPTR))
self.jit_virtual_ref_vtable = lltype.malloc(rclass.OBJECT_VTABLE,
- zero=True, flavor='raw')
+ zero=True, flavor='raw',
+ immortal=True)
self.jit_virtual_ref_vtable.name = rclass.alloc_array_name(
'jit_virtual_ref')
# build some constants
Modified: pypy/branch/leak-finder/pypy/module/cpyext/test/test_cpyext.py
==============================================================================
--- pypy/branch/leak-finder/pypy/module/cpyext/test/test_cpyext.py (original)
+++ pypy/branch/leak-finder/pypy/module/cpyext/test/test_cpyext.py Fri Oct 15 14:37:16 2010
@@ -126,17 +126,17 @@
for w_obj in lost_objects_w:
print >>sys.stderr, "Lost object %r" % (w_obj, )
leaking = True
- for llvalue in set(ll2ctypes.ALLOCATED.values()) - self.frozen_ll2callocations:
- if getattr(llvalue, "_traceback", None): # this means that the allocation should be tracked
- leaking = True
- print >>sys.stderr, "Did not deallocate %r (ll2ctypes)" % (llvalue, )
- print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines())
- for llvalue in lltype.ALLOCATED.keys():
+## for llvalue in set(ll2ctypes.ALLOCATED.values()) - self.frozen_ll2callocations:
+## if llvalue in lltype.ALLOCATED:
+## leaking = True
+## print >>sys.stderr, "Did not deallocate %r (ll2ctypes)" % (llvalue, )
+## print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines())
+ for llvalue, traceback in lltype.ALLOCATED.items():
leaking = True
print >>sys.stderr, "Did not deallocate %r (llvalue)" % (llvalue, )
- print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines())
+ print >>sys.stderr, "\t" + "\n\t".join(traceback.splitlines())
- lltype.stop_tracking_allocations()
+ lltype.stop_tracking_allocations(check=False)
return leaking
class AppTestCpythonExtensionBase(LeakCheckingTest):
Modified: pypy/branch/leak-finder/pypy/rlib/test/test_rdynload.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rlib/test/test_rdynload.py (original)
+++ pypy/branch/leak-finder/pypy/rlib/test/test_rdynload.py Fri Oct 15 14:37:16 2010
@@ -5,11 +5,18 @@
class TestDLOperations:
def test_dlopen(self):
- py.test.raises(DLOpenError, "dlopen(rffi.str2charp('xxxxxxxxxxxx'))")
- assert dlopen(rffi.str2charp(get_libc_name()))
+ s = rffi.str2charp('xxxxxxxxxxxx')
+ py.test.raises(DLOpenError, "dlopen(s)")
+ rffi.free_charp(s)
+ #
+ s = rffi.str2charp(get_libc_name())
+ assert dlopen(s)
+ rffi.free_charp(s)
def test_dlsym(self):
- lib = dlopen(rffi.str2charp(get_libc_name()))
+ s = rffi.str2charp(get_libc_name())
+ lib = dlopen(s)
+ rffi.free_charp(s)
handle = rffi.cast(lltype.Ptr(lltype.FuncType([lltype.Signed],
lltype.Signed)), dlsym(lib, 'abs'))
assert 1 == handle(1)
Modified: pypy/branch/leak-finder/pypy/rlib/test/test_rzlib.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rlib/test/test_rzlib.py (original)
+++ pypy/branch/leak-finder/pypy/rlib/test/test_rzlib.py Fri Oct 15 14:37:16 2010
@@ -189,6 +189,8 @@
assert unused3 == len('more_garbage')
assert data3 == ''
+ rzlib.deflateEnd(stream)
+
def test_decompress_max_length():
"""
@@ -205,6 +207,8 @@
assert finished2 is True
assert unused2 == 0
+ rzlib.deflateEnd(stream)
+
def test_cornercases():
"""
@@ -215,6 +219,7 @@
bytes += rzlib.compress(stream, "")
bytes += rzlib.compress(stream, "", rzlib.Z_FINISH)
assert zlib.decompress(bytes) == ""
+ rzlib.deflateEnd(stream)
stream = rzlib.inflateInit()
data, finished, unused = rzlib.decompress(stream, "")
@@ -228,3 +233,4 @@
assert finished is False
assert unused > 0
buf = buf[-unused:]
+ rzlib.deflateEnd(stream)
Modified: pypy/branch/leak-finder/pypy/rpython/llinterp.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/llinterp.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/llinterp.py Fri Oct 15 14:37:16 2010
@@ -48,8 +48,7 @@
current_interpreter = None
- def __init__(self, typer, tracing=True, exc_data_ptr=None,
- malloc_check=True):
+ def __init__(self, typer, tracing=True, exc_data_ptr=None):
self.bindings = {}
self.typer = typer
# 'heap' is module or object that provides malloc, etc for lltype ops
@@ -57,9 +56,7 @@
self.exc_data_ptr = exc_data_ptr
self.frame_stack = []
self.tracer = None
- self.malloc_check = malloc_check
self.frame_class = LLFrame
- self.mallocs = {}
if tracing:
self.tracer = Tracer()
@@ -163,24 +160,6 @@
return self.exc_data_ptr
return None
- def remember_malloc(self, ptr, llframe):
- # err....
- self.mallocs[ptr._obj] = llframe
-
- def remember_free(self, ptr):
- try:
- del self.mallocs[ptr._obj]
- except KeyError:
- self._rehash_mallocs()
- del self.mallocs[ptr._obj]
-
- def _rehash_mallocs(self):
- # rehashing is needed because some objects' hash may change
- # when being turned to
- items = self.mallocs.items()
- self.mallocs = {}
- self.mallocs.update(items)
-
def _store_exception(self, exc):
raise PleaseOverwriteStoreException("You just invoked ll2ctypes callback without overwriting _store_exception on llinterpreter")
@@ -726,13 +705,13 @@
def op_malloc(self, obj, flags):
flavor = flags['flavor']
zero = flags.get('zero', False)
+ track_allocation = flags.get('track_allocation', True)
if flavor == "stack":
result = self.heap.malloc(obj, zero=zero, flavor='raw')
self.alloca_objects.append(result)
return result
- ptr = self.heap.malloc(obj, zero=zero, flavor=flavor)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_malloc(ptr, self)
+ ptr = self.heap.malloc(obj, zero=zero, flavor=flavor,
+ track_allocation=track_allocation)
return ptr
def op_malloc_varsize(self, obj, flags, size):
@@ -741,8 +720,6 @@
assert flavor in ('gc', 'raw')
try:
ptr = self.heap.malloc(obj, size, zero=zero, flavor=flavor)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_malloc(ptr, self)
return ptr
except MemoryError:
self.make_llexception()
@@ -759,11 +736,10 @@
zero = flags.get('zero', False)
return self.heap.malloc_nonmovable(TYPE, size, zero=zero)
- def op_free(self, obj, flavor):
- assert isinstance(flavor, str)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_free(obj)
- self.heap.free(obj, flavor=flavor)
+ def op_free(self, obj, flags):
+ assert flags['flavor'] == 'raw'
+ track_allocation = flags.get('track_allocation', True)
+ self.heap.free(obj, flavor='raw', track_allocation=track_allocation)
def op_shrink_array(self, obj, smallersize):
return self.heap.shrink_array(obj, smallersize)
Modified: pypy/branch/leak-finder/pypy/rpython/lltypesystem/llmemory.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/lltypesystem/llmemory.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/lltypesystem/llmemory.py Fri Oct 15 14:37:16 2010
@@ -105,11 +105,13 @@
if (isinstance(self.TYPE, lltype.ContainerType)
and self.TYPE._gckind == 'gc'):
assert self.repeat == 1
- p = lltype.malloc(self.TYPE, flavor='raw', zero=zero)
+ p = lltype.malloc(self.TYPE, flavor='raw', zero=zero,
+ track_allocation=False)
return cast_ptr_to_adr(p)
else:
T = lltype.FixedSizeArray(self.TYPE, self.repeat)
- p = lltype.malloc(T, flavor='raw', zero=zero)
+ p = lltype.malloc(T, flavor='raw', zero=zero,
+ track_allocation=False)
array_adr = cast_ptr_to_adr(p)
return array_adr + ArrayItemsOffset(T)
@@ -288,7 +290,8 @@
count = 0
p = lltype.malloc(parenttype or self.TYPE, count,
immortal = self.TYPE._gckind == 'raw',
- zero = zero)
+ zero = zero,
+ track_allocation = False)
return cast_ptr_to_adr(p)
def raw_memcopy(self, srcadr, dstadr):
Modified: pypy/branch/leak-finder/pypy/rpython/lltypesystem/lltype.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/lltypesystem/lltype.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/lltypesystem/lltype.py Fri Oct 15 14:37:16 2010
@@ -1,7 +1,3 @@
-import StringIO
-import traceback
-import sys
-
import py
from pypy.rlib.rarithmetic import (r_int, r_uint, intmask, r_singlefloat,
r_ulonglong, r_longlong, base_int,
@@ -10,25 +6,13 @@
from pypy.tool.uid import Hashable
from pypy.tool.tls import tlsobject
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
from types import NoneType
from sys import maxint
import weakref
TLS = tlsobject()
-# Track allocations to detect memory leaks
-# Don't track 'gc' and immortal mallocs
-TRACK_ALLOCATIONS = False
-ALLOCATED = identity_dict()
-
-def start_tracking_allocations():
- global TRACK_ALLOCATIONS
- TRACK_ALLOCATIONS = True
- ALLOCATED.clear()
-
-def stop_tracking_allocations():
- global TRACK_ALLOCATIONS
- TRACK_ALLOCATIONS = False
class _uninitialized(object):
def __init__(self, TYPE):
@@ -1380,41 +1364,21 @@
__slots__ = ('_TYPE',
'_parent_type', '_parent_index', '_keepparent',
'_wrparent',
- '__weakref__', '_traceback',
- '__storage')
+ '__weakref__',
+ '_storage')
- def __init__(self, TYPE, track_allocation=None):
+ def __init__(self, TYPE):
self._wrparent = None
self._TYPE = TYPE
self._storage = True # means "use default storage", as opposed to:
# None - container was freed
# - using ctypes
# (see ll2ctypes.py)
- if track_allocation is not False and TRACK_ALLOCATIONS:
- self._traceback = self._get_traceback()
- ALLOCATED[self] = None
- else:
- self._traceback = None
-
- def _get_traceback(self):
- frame = sys._getframe().f_back.f_back.f_back.f_back
- sio = StringIO.StringIO()
- traceback.print_stack(frame, file=sio)
- return sio.getvalue()
def _free(self):
self._check() # no double-frees
self._storage = None
- def _storage_get(self):
- return self.__storage
-
- def _storage_set(self, value):
- self.__storage = value
- if value is not True and self in ALLOCATED:
- del ALLOCATED[self]
- _storage = property(_storage_get, _storage_set)
-
def _was_freed(self):
if self._storage is None:
return True
@@ -1493,12 +1457,12 @@
__slots__ = ('_hash_cache_', '_compilation_info')
- def __new__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None, track_allocation=None):
+ def __new__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None):
my_variety = _struct_variety(TYPE._names)
return object.__new__(my_variety)
- def __init__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None, track_allocation=None):
- _parentable.__init__(self, TYPE, track_allocation)
+ def __init__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None):
+ _parentable.__init__(self, TYPE)
if n is not None and TYPE._arrayfld is None:
raise TypeError("%r is not variable-sized" % (TYPE,))
if n is None and TYPE._arrayfld is not None:
@@ -1506,8 +1470,7 @@
first, FIRSTTYPE = TYPE._first_struct()
for fld, typ in TYPE._flds.items():
if fld == TYPE._arrayfld:
- value = _array(typ, n, initialization=initialization, parent=self, parentindex=fld,
- track_allocation=track_allocation)
+ value = _array(typ, n, initialization=initialization, parent=self, parentindex=fld)
else:
value = typ._allocate(initialization=initialization, parent=self, parentindex=fld)
setattr(self, fld, value)
@@ -1568,12 +1531,12 @@
__slots__ = ('items',)
- def __init__(self, TYPE, n, initialization=None, parent=None, parentindex=None, track_allocation=None):
+ def __init__(self, TYPE, n, initialization=None, parent=None, parentindex=None):
if not isinstance(n, int):
raise TypeError, "array length must be an int"
if n < 0:
raise ValueError, "negative array length"
- _parentable.__init__(self, TYPE, track_allocation)
+ _parentable.__init__(self, TYPE)
try:
myrange = range(n)
except OverflowError:
@@ -1640,7 +1603,7 @@
_cache = weakref.WeakKeyDictionary() # parentarray -> {subarrays}
def __init__(self, TYPE, parent, baseoffset_or_fieldname):
- _parentable.__init__(self, TYPE, track_allocation=False)
+ _parentable.__init__(self, TYPE)
self._setparentstructure(parent, baseoffset_or_fieldname)
# Keep the parent array alive, we share the same allocation.
# Don't do it if we are inside a GC object, though -- it's someone
@@ -1648,6 +1611,13 @@
if typeOf(top_container(parent))._gckind == 'raw':
self._keepparent = parent
+ def __str__(self):
+ parent = self._wrparent()
+ if parent is None:
+ return '_subarray at %s in already freed' % (self._parent_index,)
+ return '_subarray at %r in %s' % (self._parent_index,
+ parent._TYPE)
+
def __repr__(self):
parent = self._wrparent()
if parent is None:
@@ -1861,8 +1831,9 @@
return id(self.value)
-def malloc(T, n=None, flavor='gc', immortal=False, zero=False):
- assert flavor != 'cpy'
+def malloc(T, n=None, flavor='gc', immortal=False, zero=False,
+ track_allocation=True):
+ assert flavor in ('gc', 'raw')
if zero or immortal:
initialization = 'example'
elif flavor == 'raw':
@@ -1870,9 +1841,9 @@
else:
initialization = 'malloc'
if isinstance(T, Struct):
- o = _struct(T, n, initialization=initialization, track_allocation=flavor == "raw" and not immortal)
+ o = _struct(T, n, initialization=initialization)
elif isinstance(T, Array):
- o = _array(T, n, initialization=initialization, track_allocation=flavor == "raw" and not immortal)
+ o = _array(T, n, initialization=initialization)
elif isinstance(T, OpaqueType):
assert n is None
o = _opaque(T, initialization=initialization)
@@ -1880,15 +1851,19 @@
raise TypeError, "malloc for Structs and Arrays only"
if T._gckind != 'gc' and not immortal and flavor.startswith('gc'):
raise TypeError, "gc flavor malloc of a non-GC non-immortal structure"
+ if flavor == "raw" and not immortal and track_allocation:
+ leakfinder.remember_malloc(o, framedepth=2)
solid = immortal or not flavor.startswith('gc') # immortal or non-gc case
return _ptr(Ptr(T), o, solid)
-def free(p, flavor):
+def free(p, flavor, track_allocation=True):
if flavor.startswith('gc'):
raise TypeError, "gc flavor free"
T = typeOf(p)
if not isinstance(T, Ptr) or p._togckind() != 'raw':
raise TypeError, "free(): only for pointers to non-gc containers"
+ if track_allocation:
+ leakfinder.remember_free(p._obj0)
p._obj0._free()
def functionptr(TYPE, name, **attrs):
Modified: pypy/branch/leak-finder/pypy/rpython/lltypesystem/rclass.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/lltypesystem/rclass.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/lltypesystem/rclass.py Fri Oct 15 14:37:16 2010
@@ -420,7 +420,7 @@
return cast_pointer(self.lowleveltype, result)
def create_instance(self):
- return malloc(self.object_type, flavor=self.gcflavor)
+ return malloc(self.object_type, flavor=self.gcflavor, immortal=True)
def initialize_prebuilt_data(self, value, classdef, result):
if self.classdef is not None:
Modified: pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_ll2ctypes.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_ll2ctypes.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_ll2ctypes.py Fri Oct 15 14:37:16 2010
@@ -765,6 +765,7 @@
assert abs(float(b[1]) - 1.1) < 1E-6
assert isinstance(b[2], rffi.r_singlefloat)
assert abs(float(b[2]) - 2.2) < 1E-6
+ lltype.free(a, flavor='raw')
def test_different_signatures(self):
if sys.platform=='win32':
@@ -879,6 +880,7 @@
qsort(rffi.cast(rffi.VOIDP, a), 5, rffi.sizeof(rffi.INT), compare)
for i in range(5):
assert a[i] == i + 1
+ lltype.free(a, flavor='raw')
def test_array_type_bug(self):
A = lltype.Array(lltype.Signed)
Modified: pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_llmemory.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_llmemory.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_llmemory.py Fri Oct 15 14:37:16 2010
@@ -324,12 +324,14 @@
p_t = lltype.malloc(T)
assert p_t.s == lltype.nullptr(S)
# raw malloc does not
- p_raw_t = lltype.malloc(T, flavor="raw")
- py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.s")
+ U = lltype.Struct("U", ('x', lltype.Signed))
+ p_raw_t = lltype.malloc(U, flavor="raw")
+ py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.x")
+ lltype.free(p_raw_t, flavor="raw")
# this sort of raw_malloc too
- p_raw_t = cast_adr_to_ptr(raw_malloc(sizeof(T)), lltype.Ptr(T))
- py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.s")
-
+ p_raw_t = cast_adr_to_ptr(raw_malloc(sizeof(U)), lltype.Ptr(U))
+ py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.x")
+
def test_raw_malloc_signed_bunch():
adr = raw_malloc(sizeof(lltype.Signed) * 50)
@@ -601,7 +603,8 @@
a = lltype.malloc(A, flavor='raw')
src = cast_ptr_to_adr(a) + itemoffsetof(A, 0)
raw_memclear(src, sizeof(lltype.Signed) * 0)
-
+ lltype.free(a, flavor="raw")
+
def test_nonneg():
S1 = lltype.GcStruct('S1', ('x', lltype.Float))
A1 = lltype.GcArray(lltype.Float)
Modified: pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_lltype.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_lltype.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_lltype.py Fri Oct 15 14:37:16 2010
@@ -2,6 +2,7 @@
from pypy.rpython.lltypesystem.lltype import *
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
def isweak(p, T):
try:
@@ -804,22 +805,20 @@
class TestTrackAllocation:
- def setup_method(self, func):
- start_tracking_allocations()
-
- def teardown_method(self, func):
- assert not lltype.ALLOCATED, "Memory was not correctly freed"
- stop_tracking_allocations()
+ def test_automatic_tracking(self):
+ # calls to start_tracking_allocations/stop_tracking_allocations
+ # should occur automatically from pypy/conftest.py. Check that.
+ assert leakfinder.TRACK_ALLOCATIONS
def test_track_allocation(self):
"""A malloc'd buffer fills the ALLOCATED dictionary"""
- assert lltype.TRACK_ALLOCATIONS
- assert not lltype.ALLOCATED
+ assert leakfinder.TRACK_ALLOCATIONS
+ assert not leakfinder.ALLOCATED
buf = malloc(Array(Signed), 1, flavor="raw")
- assert len(lltype.ALLOCATED) == 1
- assert lltype.ALLOCATED.keys() == [buf._obj]
+ assert len(leakfinder.ALLOCATED) == 1
+ assert leakfinder.ALLOCATED.keys() == [buf._obj]
free(buf, flavor="raw")
- assert not lltype.ALLOCATED
+ assert not leakfinder.ALLOCATED
def test_str_from_buffer(self):
"""gc-managed memory does not need to be freed"""
@@ -828,16 +827,22 @@
for i in range(size): raw_buf[i] = 'a'
rstr = rffi.str_from_buffer(raw_buf, gc_buf, size, size)
rffi.keep_buffer_alive_until_here(raw_buf, gc_buf)
- assert not lltype.ALLOCATED
+ assert not leakfinder.ALLOCATED
def test_leak_traceback(self):
"""Test info stored for allocated items"""
buf = malloc(Array(Signed), 1, flavor="raw")
- traceback = lltype.ALLOCATED.keys()[0]._traceback
+ traceback = leakfinder.ALLOCATED.values()[0]
lines = traceback.splitlines()
assert 'malloc(' in lines[-1] and 'flavor="raw")' in lines[-1]
- # XXX The traceback should not be too long
+ # The traceback should not be too long
print traceback
free(buf, flavor="raw")
+
+ def test_no_tracking(self):
+ p1 = malloc(Array(Signed), 1, flavor='raw', track_allocation=False)
+ p2 = malloc(Array(Signed), 1, flavor='raw', track_allocation=False)
+ free(p2, flavor='raw', track_allocation=False)
+ # p1 is not freed
Modified: pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_rffi.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_rffi.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/lltypesystem/test/test_rffi.py Fri Oct 15 14:37:16 2010
@@ -9,7 +9,7 @@
from pypy.rpython.lltypesystem.rstr import STR
from pypy.rpython.lltypesystem import lltype
from pypy.tool.udir import udir
-from pypy.rpython.test.test_llinterp import interpret, MallocMismatch
+from pypy.rpython.test.test_llinterp import interpret
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.annotation.annrpython import RPythonAnnotator
from pypy.rpython.rtyper import RPythonTyper
Modified: pypy/branch/leak-finder/pypy/rpython/memory/gctransform/transform.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/memory/gctransform/transform.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/memory/gctransform/transform.py Fri Oct 15 14:37:16 2010
@@ -430,7 +430,8 @@
return self.parenttransformer.gct_malloc_varsize(hop)
def gct_free(self, hop):
- flavor = hop.spaceop.args[1].value
+ flags = hop.spaceop.args[1].value
+ flavor = flags['flavor']
assert flavor == 'raw'
return self.parenttransformer.gct_free(hop)
@@ -606,7 +607,8 @@
def gct_free(self, hop):
op = hop.spaceop
- flavor = op.args[1].value
+ flags = op.args[1].value
+ flavor = flags['flavor']
v = op.args[0]
assert flavor != 'cpy', "cannot free CPython objects directly"
if flavor == 'raw':
Modified: pypy/branch/leak-finder/pypy/rpython/memory/gcwrapper.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/memory/gcwrapper.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/memory/gcwrapper.py Fri Oct 15 14:37:16 2010
@@ -42,7 +42,8 @@
#
# Interface for the llinterp
#
- def malloc(self, TYPE, n=None, flavor='gc', zero=False):
+ def malloc(self, TYPE, n=None, flavor='gc', zero=False,
+ track_allocation=True):
if flavor == 'gc':
typeid = self.get_type_id(TYPE)
addr = self.gc.malloc(typeid, n, zero=zero)
@@ -51,7 +52,8 @@
gctypelayout.zero_gc_pointers(result)
return result
else:
- return lltype.malloc(TYPE, n, flavor=flavor, zero=zero)
+ return lltype.malloc(TYPE, n, flavor=flavor, zero=zero,
+ track_allocation=track_allocation)
def malloc_nonmovable(self, TYPE, n=None, zero=False):
typeid = self.get_type_id(TYPE)
@@ -69,9 +71,10 @@
return self.gc.shrink_array(addr, smallersize)
return False
- def free(self, TYPE, flavor='gc'):
+ def free(self, TYPE, flavor='gc', track_allocation=True):
assert flavor != 'gc'
- return lltype.free(TYPE, flavor=flavor)
+ return lltype.free(TYPE, flavor=flavor,
+ track_allocation=track_allocation)
def setfield(self, obj, fieldname, fieldvalue):
STRUCT = lltype.typeOf(obj).TO
Modified: pypy/branch/leak-finder/pypy/rpython/memory/support.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/memory/support.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/memory/support.py Fri Oct 15 14:37:16 2010
@@ -30,7 +30,8 @@
# we zero-initialize the chunks to make the translation
# backends happy, but we don't need to do it at run-time.
zero = not we_are_translated()
- return lltype.malloc(CHUNK, flavor="raw", zero=zero)
+ return lltype.malloc(CHUNK, flavor="raw", zero=zero,
+ track_allocation=False)
result = self.free_list
self.free_list = result.next
@@ -44,7 +45,7 @@
# Don't cache the old chunks but free them immediately.
# Helps debugging, and avoids that old chunks full of
# addresses left behind by a test end up in genc...
- lltype.free(chunk, flavor="raw")
+ lltype.free(chunk, flavor="raw", track_allocation=False)
unused_chunks = FreeList()
cache[chunk_size] = unused_chunks, null_chunk
Modified: pypy/branch/leak-finder/pypy/rpython/rbuiltin.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/rbuiltin.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/rbuiltin.py Fri Oct 15 14:37:16 2010
@@ -345,17 +345,22 @@
BUILTIN_TYPER[object.__init__] = rtype_object__init__
# annotation of low-level types
-def rtype_malloc(hop, i_flavor=None, i_zero=None):
+def rtype_malloc(hop, i_flavor=None, i_zero=None, i_track_allocation=None):
assert hop.args_s[0].is_constant()
vlist = [hop.inputarg(lltype.Void, arg=0)]
opname = 'malloc'
- v_flavor, v_zero = parse_kwds(hop, (i_flavor, lltype.Void), (i_zero, None))
+ v_flavor, v_zero, v_track_allocation = parse_kwds(hop,
+ (i_flavor, lltype.Void),
+ (i_zero, None),
+ (i_track_allocation, None))
flags = {'flavor': 'gc'}
if v_flavor is not None:
flags['flavor'] = v_flavor.value
if i_zero is not None:
flags['zero'] = v_zero.value
+ if i_track_allocation is not None:
+ flags['track_allocation'] = v_track_allocation.value
vlist.append(hop.inputconst(lltype.Void, flags))
if hop.nb_args == 2:
@@ -366,10 +371,19 @@
hop.exception_is_here()
return hop.genop(opname, vlist, resulttype = hop.r_result.lowleveltype)
-def rtype_free(hop, i_flavor):
- assert i_flavor == 1
+def rtype_free(hop, i_flavor, i_track_allocation=None):
+ vlist = [hop.inputarg(hop.args_r[0], arg=0)]
+ v_flavor, v_track_allocation = parse_kwds(hop,
+ (i_flavor, lltype.Void),
+ (i_track_allocation, None))
+ #
+ assert v_flavor is not None and v_flavor.value == 'raw'
+ flags = {'flavor': 'raw'}
+ if i_track_allocation is not None:
+ flags['track_allocation'] = v_track_allocation.value
+ vlist.append(hop.inputconst(lltype.Void, flags))
+ #
hop.exception_cannot_occur()
- vlist = hop.inputargs(hop.args_r[0], lltype.Void)
hop.genop('free', vlist)
def rtype_const_result(hop):
@@ -584,8 +598,9 @@
vinst, = hop.inputargs(hop.args_r[0])
flavor = hop.args_r[0].gcflavor
assert flavor != 'gc'
- cflavor = hop.inputconst(lltype.Void, flavor)
- return hop.genop('free', [vinst, cflavor])
+ flags = {'flavor': flavor}
+ cflags = hop.inputconst(lltype.Void, flags)
+ return hop.genop('free', [vinst, cflags])
BUILTIN_TYPER[objectmodel.free_non_gc_object] = rtype_free_non_gc_object
Modified: pypy/branch/leak-finder/pypy/rpython/test/test_llinterp.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/test/test_llinterp.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/test/test_llinterp.py Fri Oct 15 14:37:16 2010
@@ -12,13 +12,11 @@
from pypy.annotation.model import lltype_to_annotation
from pypy.rlib.rarithmetic import r_uint, ovfcheck
from pypy.rpython.ootypesystem import ootype
+from pypy.tool import leakfinder
from pypy import conftest
# switch on logging of interp to show more info on failing tests
-class MallocMismatch(Exception):
- pass
-
def setup_module(mod):
mod.logstate = py.log._getstate()
py.log.setconsumer("llinterp", py.log.STDOUT)
@@ -72,7 +70,7 @@
def get_interpreter(func, values, view='auto', viewbefore='auto', policy=None,
someobjects=False, type_system="lltype", backendopt=False,
- config=None, malloc_check=True, **extraconfigopts):
+ config=None, **extraconfigopts):
extra_key = [(key, value) for key, value in extraconfigopts.iteritems()]
extra_key.sort()
extra_key = tuple(extra_key)
@@ -97,7 +95,7 @@
viewbefore, policy, type_system=type_system,
backendopt=backendopt, config=config,
**extraconfigopts)
- interp = LLInterpreter(typer, malloc_check=malloc_check)
+ interp = LLInterpreter(typer)
_tcache[key] = (t, interp, graph)
# keep the cache small
_lastinterpreted.append(key)
@@ -115,10 +113,17 @@
interp, graph = get_interpreter(func, values, view, viewbefore, policy,
someobjects, type_system=type_system,
backendopt=backendopt, config=config,
- malloc_check=malloc_check, **kwargs)
- result = interp.eval_graph(graph, values)
- if malloc_check and interp.mallocs:
- raise MallocMismatch(interp.mallocs)
+ **kwargs)
+ if not malloc_check:
+ result = interp.eval_graph(graph, values)
+ else:
+ prev = leakfinder.start_tracking_allocations()
+ try:
+ result = interp.eval_graph(graph, values)
+ finally:
+ leaks = leakfinder.stop_tracking_allocations(False, prev)
+ if leaks:
+ raise leakfinder.MallocMismatch(leaks)
return result
def interpret_raises(exc, func, values, view='auto', viewbefore='auto',
@@ -418,6 +423,7 @@
assert result
def test_stack_malloc():
+ py.test.skip("stack-flavored mallocs no longer supported")
class A(object):
pass
def f():
@@ -430,6 +436,7 @@
assert result == 1
def test_invalid_stack_access():
+ py.test.skip("stack-flavored mallocs no longer supported")
class A(object):
pass
globala = A()
@@ -605,7 +612,7 @@
if x:
free(t, flavor='raw')
interpret(f, [1])
- py.test.raises(MallocMismatch, "interpret(f, [0])")
+ py.test.raises(leakfinder.MallocMismatch, "interpret(f, [0])")
def f():
t1 = malloc(T, flavor='raw')
Modified: pypy/branch/leak-finder/pypy/rpython/test/test_nongc.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/test/test_nongc.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/test/test_nongc.py Fri Oct 15 14:37:16 2010
@@ -79,7 +79,7 @@
py.test.raises(TypeError,rtyper.specialize) # results in an invalid cast
def test_isinstance():
- class A:
+ class A(object):
_alloc_flavor_ = "raw"
class B(A):
pass
@@ -95,7 +95,24 @@
o = B()
else:
o = C()
- return 100*isinstance(o, A)+10*isinstance(o, B)+1*isinstance(o ,C)
+ res = 100*isinstance(o, A) + 10*isinstance(o, B) + 1*isinstance(o, C)
+ if i == 0:
+ pass
+ elif i == 1:
+ assert isinstance(o, A)
+ free_non_gc_object(o)
+ elif i == 2:
+ assert isinstance(o, B)
+ free_non_gc_object(o)
+ else:
+ assert isinstance(o, C)
+ free_non_gc_object(o)
+ return res
+
+ assert f(1) == 100
+ assert f(2) == 110
+ assert f(3) == 111
+ assert f(0) == 0
a = RPythonAnnotator()
#does not raise:
@@ -131,10 +148,14 @@
d = b
elif i == 2:
e = c
- return (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
+ res = (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
0x0008*(a is e) | 0x0010*(b is c) | 0x0020*(b is d) |
0x0040*(b is e) | 0x0080*(c is d) | 0x0100*(c is e) |
0x0200*(d is e))
+ free_non_gc_object(a)
+ free_non_gc_object(b)
+ free_non_gc_object(c)
+ return res
a = RPythonAnnotator()
#does not raise:
s = a.build_types(f, [int])
@@ -169,10 +190,13 @@
d = b
elif i == 2:
e = c
- return (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
+ res = (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
0x0008*(a is e) | 0x0010*(b is c) | 0x0020*(b is d) |
0x0040*(b is e) | 0x0080*(c is d) | 0x0100*(c is e) |
0x0200*(d is e))
+ free_non_gc_object(a)
+ free_non_gc_object(b)
+ return res
a = RPythonAnnotator()
#does not raise:
s = a.build_types(f, [int])
Modified: pypy/branch/leak-finder/pypy/rpython/test/test_rptr.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/test/test_rptr.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/test/test_rptr.py Fri Oct 15 14:37:16 2010
@@ -212,10 +212,31 @@
S = Struct('S', ('x', Signed))
def fn(n):
- p = malloc(S, flavor='whatever')
+ p = malloc(S, flavor='raw')
p.x = n
result = p.x
- free(p, flavor='whatever')
+ free(p, flavor='raw')
+ return n
+
+ res = interpret(fn, [23])
+ assert res == 23
+
+ S = Struct('S', ('x', Signed))
+ def fn(n):
+ p = malloc(S, flavor='raw', track_allocation=False)
+ p.x = n
+ result = p.x
+ return n
+
+ res = interpret(fn, [23])
+ assert res == 23
+
+ S = Struct('S', ('x', Signed))
+ def fn(n):
+ p = malloc(S, flavor='raw', track_allocation=False)
+ p.x = n
+ result = p.x
+ free(p, flavor='raw', track_allocation=False)
return n
res = interpret(fn, [23])
Added: pypy/branch/leak-finder/pypy/tool/leakfinder.py
==============================================================================
--- (empty file)
+++ pypy/branch/leak-finder/pypy/tool/leakfinder.py Fri Oct 15 14:37:16 2010
@@ -0,0 +1,73 @@
+import sys, gc
+import cStringIO
+import traceback
+
+# Track allocations to detect memory leaks.
+# So far, this is used for lltype.malloc(flavor='raw').
+TRACK_ALLOCATIONS = False
+ALLOCATED = {}
+
+class MallocMismatch(Exception):
+ def __str__(self):
+ dict = self.args[0]
+ dict2 = {}
+ for obj, traceback in dict.items():
+ traceback = traceback.splitlines()
+ if len(traceback) > 8:
+ traceback = [' ...'] + traceback[-6:]
+ traceback = '\n'.join(traceback)
+ dict2.setdefault(traceback, [])
+ dict2[traceback].append(obj)
+ lines = ['{']
+ for traceback, objs in dict2.items():
+ lines.append('')
+ for obj in objs:
+ lines.append('%s:' % (obj,))
+ lines.append(traceback)
+ lines.append('}')
+ return '\n'.join(lines)
+
+def start_tracking_allocations():
+ global TRACK_ALLOCATIONS
+ if TRACK_ALLOCATIONS:
+ result = ALLOCATED.copy() # nested start
+ else:
+ result = None
+ TRACK_ALLOCATIONS = True
+ ALLOCATED.clear()
+ return result
+
+def stop_tracking_allocations(check, prev=None):
+ global TRACK_ALLOCATIONS
+ assert TRACK_ALLOCATIONS
+ for i in range(5):
+ if not ALLOCATED:
+ break
+ gc.collect()
+ result = ALLOCATED.copy()
+ ALLOCATED.clear()
+ if prev is None:
+ TRACK_ALLOCATIONS = False
+ else:
+ ALLOCATED.update(prev)
+ if check and result:
+ raise MallocMismatch(result)
+ return result
+
+def remember_malloc(obj, framedepth=1):
+ if TRACK_ALLOCATIONS:
+ frame = sys._getframe(framedepth)
+ sio = cStringIO.StringIO()
+ traceback.print_stack(frame, limit=10, file=sio)
+ tb = sio.getvalue()
+ ALLOCATED[obj] = tb
+
+def remember_free(obj):
+ if TRACK_ALLOCATIONS:
+ if obj not in ALLOCATED:
+ # rehashing is needed because some objects' hash may change
+ # e.g. when lltype objects are turned into
+ items = ALLOCATED.items()
+ ALLOCATED.clear()
+ ALLOCATED.update(items)
+ del ALLOCATED[obj]
Added: pypy/branch/leak-finder/pypy/tool/test/test_leakfinder.py
==============================================================================
--- (empty file)
+++ pypy/branch/leak-finder/pypy/tool/test/test_leakfinder.py Fri Oct 15 14:37:16 2010
@@ -0,0 +1,70 @@
+import py
+from pypy.tool import leakfinder
+
+def test_start_stop():
+ leakfinder.start_tracking_allocations()
+ assert leakfinder.TRACK_ALLOCATIONS
+ leakfinder.stop_tracking_allocations(True)
+ assert not leakfinder.TRACK_ALLOCATIONS
+
+def test_start_stop_nested():
+ leakfinder.start_tracking_allocations()
+ p2 = leakfinder.start_tracking_allocations()
+ assert leakfinder.TRACK_ALLOCATIONS
+ leakfinder.stop_tracking_allocations(True, prev=p2)
+ assert leakfinder.TRACK_ALLOCATIONS
+ leakfinder.stop_tracking_allocations(True)
+ assert not leakfinder.TRACK_ALLOCATIONS
+
+def test_remember_free():
+ leakfinder.start_tracking_allocations()
+ x = 1234
+ leakfinder.remember_malloc(x)
+ leakfinder.remember_free(x)
+ leakfinder.stop_tracking_allocations(True)
+
+def test_remember_forget():
+ leakfinder.start_tracking_allocations()
+ x = 1234
+ leakfinder.remember_malloc(x)
+ py.test.raises(leakfinder.MallocMismatch,
+ leakfinder.stop_tracking_allocations, True)
+
+def test_nested_remember_forget_1():
+ leakfinder.start_tracking_allocations()
+ x = 1234
+ leakfinder.remember_malloc(x)
+ p2 = leakfinder.start_tracking_allocations()
+ leakfinder.stop_tracking_allocations(True, prev=p2)
+ py.test.raises(leakfinder.MallocMismatch,
+ leakfinder.stop_tracking_allocations, True)
+
+def test_nested_remember_forget_2():
+ p2 = leakfinder.start_tracking_allocations()
+ x = 1234
+ leakfinder.remember_malloc(x)
+ py.test.raises(leakfinder.MallocMismatch,
+ leakfinder.stop_tracking_allocations, True, prev=p2)
+ leakfinder.stop_tracking_allocations(True)
+
+def test_traceback():
+ leakfinder.start_tracking_allocations()
+ x = 1234
+ leakfinder.remember_malloc(x)
+ res = leakfinder.stop_tracking_allocations(check=False)
+ assert res.keys() == [x]
+ print res[x]
+ assert isinstance(res[x], str)
+ assert 'test_traceback' in res[x]
+ assert 'leakfinder.remember_malloc(x)' in res[x]
+
+def test_malloc_mismatch():
+ import sys, traceback, cStringIO
+ sio = cStringIO.StringIO()
+ traceback.print_stack(sys._getframe(), limit=10, file=sio)
+ tb = sio.getvalue()
+ e = leakfinder.MallocMismatch({1234: tb, 2345: tb})
+ print str(e)
+ # grouped entries for 1234 and 2345
+ assert '1234:\n2345:\n' in str(e) or '2345:\n1234:\n' in str(e)
+ assert tb[-80:] in str(e)
Modified: pypy/branch/leak-finder/pypy/translator/c/test/test_lltyped.py
==============================================================================
--- pypy/branch/leak-finder/pypy/translator/c/test/test_lltyped.py (original)
+++ pypy/branch/leak-finder/pypy/translator/c/test/test_lltyped.py Fri Oct 15 14:37:16 2010
@@ -401,6 +401,7 @@
for i in range(n):
p = malloc(S, flavor='raw', zero=True)
if p.x != 0 or p.y != 0:
+ free(p, flavor='raw')
return -1
p.x = i
p.y = i
@@ -418,14 +419,16 @@
def f(n):
for length in range(n-1, -1, -1):
p = malloc(S, length, flavor='raw', zero=True)
- if p.x != 0:
- return -1
- p.x = n
- for j in range(length):
- if p.y[j] != 0:
- return -3
- p.y[j] = n^j
- free(p, flavor='raw')
+ try:
+ if p.x != 0:
+ return -1
+ p.x = n
+ for j in range(length):
+ if p.y[j] != 0:
+ return -3
+ p.y[j] = n^j
+ finally:
+ free(p, flavor='raw')
return 42
fn = self.getcompiled(f, [int])
@@ -655,7 +658,7 @@
def test_prebuilt_ll2ctypes_array(self):
from pypy.rpython.lltypesystem import rffi, ll2ctypes
A = rffi.CArray(Char)
- a = malloc(A, 6, flavor='raw')
+ a = malloc(A, 6, flavor='raw', immortal=True)
a[0] = 'a'
a[1] = 'b'
a[2] = 'c'
@@ -676,7 +679,7 @@
def test_ll2ctypes_array_from_c(self):
from pypy.rpython.lltypesystem import rffi, ll2ctypes
A = rffi.CArray(Char)
- a = malloc(A, 6, flavor='raw')
+ a = malloc(A, 6, flavor='raw', immortal=True)
a[0] = 'a'
a[1] = 'b'
a[2] = 'c'
From arigo at codespeak.net Fri Oct 15 14:48:26 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 14:48:26 +0200 (CEST)
Subject: [pypy-svn] r77980 - pypy/extradoc/talk/pepm2011
Message-ID: <20101015124826.12C8E36E0C3@codespeak.net>
Author: arigo
Date: Fri Oct 15 14:48:25 2010
New Revision: 77980
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
Semicolonization.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 15 14:48:25 2010
@@ -928,7 +928,7 @@
into a local variable, it is stored into the frame-object, which makes it
escape. We implemented a technique to treat such frames objects in a
special way to solve this problem. This is a common approach in VM
-implementations \cite{miranda_context_1999,andreas_gal_trace-based_2009}, the
+implementations \cite{miranda_context_1999,andreas_gal_trace-based_2009}; the
novelty of our approach is that we generalized it enough to be usable for
different interpreter.
From arigo at codespeak.net Fri Oct 15 14:51:58 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 14:51:58 +0200 (CEST)
Subject: [pypy-svn] r77981 - in pypy/branch/leak-finder/pypy/rpython:
lltypesystem memory/gctransform
Message-ID: <20101015125158.CB37036E0C3@codespeak.net>
Author: arigo
Date: Fri Oct 15 14:51:57 2010
New Revision: 77981
Modified:
pypy/branch/leak-finder/pypy/rpython/lltypesystem/lloperation.py
pypy/branch/leak-finder/pypy/rpython/memory/gctransform/transform.py
Log:
Add tracking operations, to be implemented by the C backend in debug mode.
Modified: pypy/branch/leak-finder/pypy/rpython/lltypesystem/lloperation.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/lltypesystem/lloperation.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/lltypesystem/lloperation.py Fri Oct 15 14:51:57 2010
@@ -403,6 +403,8 @@
'raw_load': LLOp(sideeffects=False),
'raw_store': LLOp(),
'stack_malloc': LLOp(), # mmh
+ 'track_alloc_start': LLOp(),
+ 'track_alloc_stop': LLOp(),
'adr_add': LLOp(canfold=True),
'adr_sub': LLOp(canfold=True),
'adr_delta': LLOp(canfold=True),
Modified: pypy/branch/leak-finder/pypy/rpython/memory/gctransform/transform.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/memory/gctransform/transform.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/memory/gctransform/transform.py Fri Oct 15 14:51:57 2010
@@ -533,6 +533,8 @@
resulttype=llmemory.Address)
if flags.get('zero'):
hop.genop("raw_memclear", [v_raw, c_size])
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_start", [v_raw])
return v_raw
def gct_fv_stack_malloc(self, hop, flags, TYPE, c_size):
@@ -603,6 +605,8 @@
[self.raw_malloc_varsize_ptr, v_length,
c_const_size, c_item_size, c_offset_to_length],
resulttype=llmemory.Address)
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_start", [v_raw])
return v_raw
def gct_free(self, hop):
@@ -613,6 +617,8 @@
assert flavor != 'cpy', "cannot free CPython objects directly"
if flavor == 'raw':
v = hop.genop("cast_ptr_to_adr", [v], resulttype=llmemory.Address)
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_stop", [v])
hop.genop('raw_free', [v])
else:
assert False, "%s has no support for free with flavor %r" % (self, flavor)
From arigo at codespeak.net Fri Oct 15 15:04:36 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 15:04:36 +0200 (CEST)
Subject: [pypy-svn] r77982 - pypy/branch/leak-finder/pypy/rpython
Message-ID: <20101015130436.0C84536C22E@codespeak.net>
Author: arigo
Date: Fri Oct 15 15:04:35 2010
New Revision: 77982
Modified:
pypy/branch/leak-finder/pypy/rpython/llinterp.py
Log:
Don't do any checking when running the GC-transformed graphs on llinterp.
Probably not worth it, and messy because the GC allocates some structures
like AddressStack.
Modified: pypy/branch/leak-finder/pypy/rpython/llinterp.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/llinterp.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/llinterp.py Fri Oct 15 15:04:35 2010
@@ -1013,6 +1013,13 @@
def op_stack_malloc(self, size): # mmh
raise NotImplementedError("backend only")
+ def op_track_alloc_start(self, addr):
+ # we don't do tracking at this level
+ checkadr(addr)
+
+ def op_track_alloc_stop(self, addr):
+ checkadr(addr)
+
# ____________________________________________________________
# Overflow-detecting variants
From cfbolz at codespeak.net Fri Oct 15 15:05:30 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 15 Oct 2010 15:05:30 +0200 (CEST)
Subject: [pypy-svn] r77983 - pypy/extradoc/talk/pepm2011
Message-ID: <20101015130530.B537F36C22E@codespeak.net>
Author: cfbolz
Date: Fri Oct 15 15:05:29 2010
New Revision: 77983
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
add lifting example
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 15 15:05:29 2010
@@ -872,8 +872,21 @@
twice, the \lstinline{liftfield} operation removes it from the static heap \emph{before}
recursively lifting its fields.
+As an example for lifting, consider the static heap $$\{v^* \mapsto (T_1, w^*,
+v^*), w^* \mapsto (T_2, u^*, u^*)\}$$ which contains two static objects. If $v^*$
+now needs to be lifted, the following residual operations are produced:
+
+\begin{lstlisting}[mathescape,xleftmargin=20pt]
+$v^*$ = new($T_1$)
+$w^*$ = new($T_2$)
+set($w^*$, $L$, $u^*$)
+set($w^*$, $R$, $u^*$)
+set($v^*$, $L$, $w^*$)
+set($v^*$, $R$, $v^*$)
+\end{lstlisting}
-XXX Apply the rules of the algorithm to parts of the sample trace
+After the lifting the static heap is the empty set, because both static objects
+were lifted.
% subsection Optimizing Traces (end)
From antocuni at codespeak.net Fri Oct 15 15:12:13 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri, 15 Oct 2010 15:12:13 +0200 (CEST)
Subject: [pypy-svn] r77984 - in pypy/branch/jitffi/pypy/jit/metainterp:
optimizeopt test
Message-ID: <20101015131213.E405A36E0C1@codespeak.net>
Author: antocuni
Date: Fri Oct 15 15:12:12 2010
New Revision: 77984
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
Log:
nicely optimize the case in which we have virtualizables around, and thus we
get force_token between the call_may_force
Modified: pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py Fri Oct 15 15:12:12 2010
@@ -18,6 +18,7 @@
argtypes, restype = self._get_signature(funcval)
self.descr = cpu.calldescrof_dynamic(argtypes, restype)
self.prepare_op = prepare_op
+ self.force_token_op = None
def _get_signature(self, funcval):
"""
@@ -83,6 +84,8 @@
self.emit_operation(funcinfo.prepare_op)
for op in funcinfo.opargs:
self.emit_operation(op)
+ if funcinfo.force_token_op:
+ self.emit_operation(funcinfo.force_token_op)
def emit_operation(self, op):
# we cannot emit any operation during the optimization
@@ -91,46 +94,80 @@
def optimize_CALL(self, op):
oopspec = self._get_oopspec(op)
+ ops = [op]
if oopspec == EffectInfo.OS_LIBFFI_PREPARE:
- op = self.do_prepare_call(op)
+ ops = self.do_prepare_call(op)
elif oopspec == EffectInfo.OS_LIBFFI_PUSH_ARG:
- op = self.do_push_arg(op)
+ ops = self.do_push_arg(op)
elif oopspec == EffectInfo.OS_LIBFFI_CALL:
- op = self.do_call(op)
+ ops = self.do_call(op)
#
- if op:
+ for op in ops:
self.emit_operation(op)
optimize_CALL_MAY_FORCE = optimize_CALL
+ def optimize_FORCE_TOKEN(self, op):
+ # The handling of force_token needs a bit of exaplanation.
+ # The original trace which is getting optimized looks like this:
+ # i1 = force_token()
+ # setfield_gc(p0, i1, ...)
+ # call_may_force(...)
+ #
+ # In theory, fficall should take care of both force_token and
+ # setfield_gc. However, the lazy setfield optimization in heap.py
+ # delays the setfield_gc, with the effect that fficall.py sees them in
+ # this order:
+ # i1 = force_token()
+ # call_may_force(...)
+ # setfield_gc(p0, i1, ...)
+ #
+ # This means that see the setfield_gc only the call_may_force, when
+ # the optimization has already been done, and thus we need to take
+ # special care just of force_token.
+ #
+ # Finally, the method force_lazy_setfield in heap.py reorders the
+ # call_may_force and the setfield_gc, so the final result we get is
+ # again force_token/setfield_gc/call_may_force.
+ #
+ if self.funcinfo is None:
+ self.emit_operation(op)
+ else:
+ self.funcinfo.force_token_op = op
+
def do_prepare_call(self, op):
self.rollback_maybe()
funcval = self._get_funcval(op)
if not funcval.is_constant():
- return op # cannot optimize
+ return [op] # cannot optimize
self.begin_optimization(funcval, op)
- return None
+ return []
def do_push_arg(self, op):
funcval = self._get_funcval(op)
if not self.funcinfo or self.funcinfo.funcval is not funcval:
- return op # cannot optimize
+ return [op] # cannot optimize
self.funcinfo.opargs.append(op)
- return None
+ return []
def do_call(self, op):
funcval = self._get_funcval(op)
- if not self.funcinfo or self.funcinfo.funcval is not funcval:
- return op # cannot optimize
+ funcinfo = self.funcinfo
+ if not funcinfo or funcinfo.funcval is not funcval:
+ return [op] # cannot optimize
funcsymval = self.getvalue(op.getarg(2))
arglist = [funcsymval.force_box()]
- for push_op in self.funcinfo.opargs:
+ for push_op in funcinfo.opargs:
argval = self.getvalue(push_op.getarg(2))
arglist.append(argval.force_box())
newop = ResOperation(rop.CALL_MAY_FORCE, arglist, op.result,
- descr=self.funcinfo.descr)
+ descr=funcinfo.descr)
self.commit_optimization()
- return newop
+ ops = []
+ if funcinfo.force_token_op:
+ ops.append(funcinfo.force_token_op)
+ ops.append(newop)
+ return ops
def propagate_forward(self, op):
opnum = op.getopnum()
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py Fri Oct 15 15:12:12 2010
@@ -35,6 +35,7 @@
class namespace:
cpu = LLtypeMixin.cpu
FUNC = LLtypeMixin.FUNC
+ vable_token_descr = LLtypeMixin.valuedescr
int_float__int = MyCallDescr('if', 'i')
funcptr = FakeLLObject()
func = FakeLLObject(_fake_class=Func,
@@ -92,6 +93,32 @@
expected = ops
loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
+ def test_handle_virtualizables(self):
+ # this test needs an explanation to understand what goes on: see the
+ # coment in optimize_FORCE_TOKEN
+ ops = """
+ [i0, f1, p2]
+ call(0, ConstPtr(func), descr=libffi_prepare)
+ call(0, ConstPtr(func), i0, descr=libffi_push_arg)
+ call(0, ConstPtr(func), f1, descr=libffi_push_arg)
+ i4 = force_token()
+ setfield_gc(p2, i4, descr=vable_token_descr)
+ i3 = call_may_force(0, ConstPtr(func), 12345, descr=libffi_call)
+ guard_not_forced() [p2]
+ guard_no_exception() [p2]
+ jump(i3, f1, p2)
+ """
+ expected = """
+ [i0, f1, p2]
+ i4 = force_token()
+ setfield_gc(p2, i4, descr=vable_token_descr)
+ i3 = call_may_force(12345, i0, f1, descr=int_float__int)
+ guard_not_forced() [p2]
+ guard_no_exception() [p2]
+ jump(i3, f1, p2)
+ """
+ loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
+
# ----------------------------------------------------------------------
# in pratice, the situations described in these tests should never happen,
# but we still want to ensure correctness
@@ -195,3 +222,19 @@
"""
loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
+ def test_rollback_force_token(self):
+ ops = """
+ [i0, f1, p2]
+ call(0, ConstPtr(func), descr=libffi_prepare)
+ call(0, ConstPtr(func), i0, descr=libffi_push_arg)
+ call(0, ConstPtr(func), f1, descr=libffi_push_arg)
+ i4 = force_token()
+ i5 = int_add(i0, 1) # culprit!
+ setfield_gc(p2, i4, descr=vable_token_descr)
+ i3 = call_may_force(0, ConstPtr(func), 12345, descr=libffi_call)
+ guard_not_forced() [p2]
+ guard_no_exception() [p2]
+ jump(i3, f1, p2)
+ """
+ expected = ops
+ loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
From cfbolz at codespeak.net Fri Oct 15 15:13:04 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 15 Oct 2010 15:13:04 +0200 (CEST)
Subject: [pypy-svn] r77985 - pypy/extradoc/talk/pepm2011
Message-ID: <20101015131304.D1C2336E0C1@codespeak.net>
Author: cfbolz
Date: Fri Oct 15 15:13:03 2010
New Revision: 77985
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
enhance example
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 15 15:13:03 2010
@@ -888,6 +888,19 @@
After the lifting the static heap is the empty set, because both static objects
were lifted.
+If from the same heap $w^*$ is lifted instead, the following operations are
+produced:
+
+\begin{lstlisting}[mathescape,xleftmargin=20pt]
+$w^*$ = new($T_2$)
+set($w^*$, $L$, $u^*$)
+set($w^*$, $R$, $u^*$)
+\end{lstlisting}
+
+The static heap afterwards is $\{v^* \mapsto (T_1, w^*, v^*)\}$.
+
+
+
% subsection Optimizing Traces (end)
\subsection{Analysis of the Algorithm}
From arigo at codespeak.net Fri Oct 15 15:13:45 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 15:13:45 +0200 (CEST)
Subject: [pypy-svn] r77986 - pypy/branch/32ptr-on-64bit/pypy/translator/c
Message-ID: <20101015131345.9A52536E0C3@codespeak.net>
Author: arigo
Date: Fri Oct 15 15:13:44 2010
New Revision: 77986
Modified:
pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py
Log:
Change the value of DEBUGFLAGS in non-asmgcc builds to
remove -O1. This is fine with the extra stack that we
get by the recent change to c/src/stack.h.
Modified: pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py
==============================================================================
--- pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py (original)
+++ pypy/branch/32ptr-on-64bit/pypy/translator/c/genc.py Fri Oct 15 15:13:44 2010
@@ -628,7 +628,7 @@
python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py $(GCMAPFILES) > $@')
else:
- mk.definition('DEBUGFLAGS', '-O1 -g')
+ mk.definition('DEBUGFLAGS', '-g')
mk.write()
#self.translator.platform,
# ,
From arigo at codespeak.net Fri Oct 15 15:16:31 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 15:16:31 +0200 (CEST)
Subject: [pypy-svn] r77987 - pypy/extradoc/talk/pepm2011
Message-ID: <20101015131631.67D6D36E0C1@codespeak.net>
Author: arigo
Date: Fri Oct 15 15:16:30 2010
New Revision: 77987
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
Rewording.
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 15 15:16:30 2010
@@ -887,9 +887,8 @@
After the lifting the static heap is the empty set, because both static objects
were lifted.
-
-If from the same heap $w^*$ is lifted instead, the following operations are
-produced:
+If we had lifted $w^*$ instead of $v^*$, then the following operations would
+have been produced:
\begin{lstlisting}[mathescape,xleftmargin=20pt]
$w^*$ = new($T_2$)
@@ -897,7 +896,8 @@
set($w^*$, $R$, $u^*$)
\end{lstlisting}
-The static heap afterwards is $\{v^* \mapsto (T_1, w^*, v^*)\}$.
+In this case, the static heap afterwards would be
+$\{v^* \mapsto (T_1, w^*, v^*)\}$.
From arigo at codespeak.net Fri Oct 15 15:50:14 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 15:50:14 +0200 (CEST)
Subject: [pypy-svn] r77989 - pypy/branch/leak-finder/pypy/translator/c/src
Message-ID: <20101015135014.D2DAF36E0C1@codespeak.net>
Author: arigo
Date: Fri Oct 15 15:50:13 2010
New Revision: 77989
Added:
pypy/branch/leak-finder/pypy/translator/c/src/debug_alloc.h
Modified:
pypy/branch/leak-finder/pypy/translator/c/src/g_include.h
pypy/branch/leak-finder/pypy/translator/c/src/main.h
Log:
In RPY_ASSERT mode, print a warning at the end of running a translated
program if there are malloc()s without a corresponding free(). If you
get more than a few ones, you have a leak.
Added: pypy/branch/leak-finder/pypy/translator/c/src/debug_alloc.h
==============================================================================
--- (empty file)
+++ pypy/branch/leak-finder/pypy/translator/c/src/debug_alloc.h Fri Oct 15 15:50:13 2010
@@ -0,0 +1,81 @@
+/**************************************************************/
+ /*** tracking raw mallocs and frees for debugging ***/
+
+#ifndef RPY_ASSERT
+
+# define OP_TRACK_ALLOC_START(addr, r) /* nothing */
+# define OP_TRACK_ALLOC_STOP(addr, r) /* nothing */
+
+#else /* ifdef RPY_ASSERT */
+
+# define OP_TRACK_ALLOC_START(addr, r) pypy_debug_alloc_start(addr, \
+ __FUNCTION__)
+# define OP_TRACK_ALLOC_STOP(addr, r) pypy_debug_alloc_stop(addr)
+
+void pypy_debug_alloc_start(void*, const char*);
+void pypy_debug_alloc_stop(void*);
+void pypy_debug_alloc_results(void);
+
+/************************************************************/
+
+
+#ifndef PYPY_NOT_MAIN_FILE
+
+struct pypy_debug_alloc_s {
+ struct pypy_debug_alloc_s *next;
+ void *addr;
+ const char *funcname;
+};
+
+static struct pypy_debug_alloc_s *pypy_debug_alloc_list = NULL;
+
+void pypy_debug_alloc_start(void *addr, const char *funcname)
+{
+ struct pypy_debug_alloc_s *p = malloc(sizeof(struct pypy_debug_alloc_s));
+ RPyAssert(p, "out of memory");
+ p->next = pypy_debug_alloc_list;
+ p->addr = addr;
+ p->funcname = funcname;
+ pypy_debug_alloc_list = p;
+}
+
+void pypy_debug_alloc_stop(void *addr)
+{
+ struct pypy_debug_alloc_s **p;
+ for (p = &pypy_debug_alloc_list; *p; p = &((*p)->next))
+ if ((*p)->addr == addr)
+ {
+ struct pypy_debug_alloc_s *dying;
+ dying = *p;
+ *p = dying->next;
+ free(dying);
+ return;
+ }
+ RPyAssert(0, "free() of a never-malloc()ed object");
+}
+
+void pypy_debug_alloc_results(void)
+{
+ long count = 0;
+ struct pypy_debug_alloc_s *p;
+ for (p = pypy_debug_alloc_list; p; p = p->next)
+ count++;
+ if (count > 0)
+ {
+ fprintf(stderr, "debug_alloc.h: %ld mallocs left", count);
+ char *env = getenv("PYPY_ALLOC");
+ if (env && *env)
+ {
+ fprintf(stderr, " (most recent first):\n");
+ for (p = pypy_debug_alloc_list; p; p = p->next)
+ fprintf(stderr, " %p %s\n", p->addr, p->funcname);
+ }
+ else
+ fprintf(stderr, " (use PYPY_ALLOC=1 to see the list)\n");
+ }
+}
+
+#endif
+
+
+#endif /* RPY_ASSERT */
Modified: pypy/branch/leak-finder/pypy/translator/c/src/g_include.h
==============================================================================
--- pypy/branch/leak-finder/pypy/translator/c/src/g_include.h (original)
+++ pypy/branch/leak-finder/pypy/translator/c/src/g_include.h Fri Oct 15 15:50:13 2010
@@ -53,6 +53,7 @@
# include "src/rtyper.h"
# include "src/debug_print.h"
# include "src/debug_traceback.h"
+# include "src/debug_alloc.h"
#ifndef AVR
# include "src/ll_os.h"
# include "src/ll_strtod.h"
Modified: pypy/branch/leak-finder/pypy/translator/c/src/main.h
==============================================================================
--- pypy/branch/leak-finder/pypy/translator/c/src/main.h (original)
+++ pypy/branch/leak-finder/pypy/translator/c/src/main.h Fri Oct 15 15:50:13 2010
@@ -53,10 +53,16 @@
}
exitcode = STANDALONE_ENTRY_POINT(list);
+
+#ifdef RPY_ASSERT
+ pypy_debug_alloc_results();
+#endif
+
if (RPyExceptionOccurred()) {
/* print the RPython traceback */
pypy_debug_catch_fatal_exception();
}
+
return exitcode;
memory_out:
From antocuni at codespeak.net Fri Oct 15 16:12:28 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri, 15 Oct 2010 16:12:28 +0200 (CEST)
Subject: [pypy-svn] r77990 - pypy/branch/jitffi/pypy/module/pypyjit/test
Message-ID: <20101015141228.090B736E0C1@codespeak.net>
Author: antocuni
Date: Fri Oct 15 16:12:27 2010
New Revision: 77990
Modified:
pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py
Log:
update the number of expected operations: we now have a force_token, setfield_gc and guard_not_forced more. Since we get two loops, it's 6 more ops in total
Modified: pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py Fri Oct 15 16:12:27 2010
@@ -1151,7 +1151,7 @@
i += 1
return res
''' % locals(),
- 70, ([], 8.0*2000), threshold=1000)
+ 76, ([], 8.0*2000), threshold=1000)
pow_addr = int(out.splitlines()[0])
ops = self.get_by_bytecode('CALL_FUNCTION')
assert len(ops) == 2 # we get two loops, because of specialization
From antocuni at codespeak.net Fri Oct 15 16:18:26 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri, 15 Oct 2010 16:18:26 +0200 (CEST)
Subject: [pypy-svn] r77991 - pypy/branch/jitffi/pypy/module/pypyjit/test
Message-ID: <20101015141826.5223836E0C1@codespeak.net>
Author: antocuni
Date: Fri Oct 15 16:18:24 2010
New Revision: 77991
Modified:
pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py
Log:
fix the test, and check that we get the expected operations
Modified: pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py Fri Oct 15 16:18:24 2010
@@ -1155,11 +1155,14 @@
pow_addr = int(out.splitlines()[0])
ops = self.get_by_bytecode('CALL_FUNCTION')
assert len(ops) == 2 # we get two loops, because of specialization
- op = ops[0]
- # XXX: there should be a guard_not_forced
- assert op[-1].getopname() == 'guard_no_exception'
- call = op[-2]
- assert call.getopname() == 'call'
+ call_function = ops[0]
+ last_ops = [op.getopname() for op in call_function[-5:]]
+ assert last_ops == ['force_token',
+ 'setfield_gc',
+ 'call_may_force',
+ 'guard_not_forced',
+ 'guard_no_exception']
+ call = call_function[-3]
assert call.getarg(0).value == pow_addr
assert call.getarg(1).value == 2.0
assert call.getarg(2).value == 3.0
From arigo at codespeak.net Fri Oct 15 16:25:29 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 16:25:29 +0200 (CEST)
Subject: [pypy-svn] r77992 - pypy/branch/leak-finder/pypy/rpython/memory/gc
Message-ID: <20101015142529.82D6636E076@codespeak.net>
Author: arigo
Date: Fri Oct 15 16:25:27 2010
New Revision: 77992
Modified:
pypy/branch/leak-finder/pypy/rpython/memory/gc/minimarkpage.py
Log:
Add track_allocation=False to the arenas; reporting them at process shutdown is unhelpful.
Modified: pypy/branch/leak-finder/pypy/rpython/memory/gc/minimarkpage.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/memory/gc/minimarkpage.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/memory/gc/minimarkpage.py Fri Oct 15 16:25:27 2010
@@ -100,11 +100,14 @@
# allocation of the given size.
length = small_request_threshold / WORD + 1
self.page_for_size = lltype.malloc(rffi.CArray(PAGE_PTR), length,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
self.full_page_for_size = lltype.malloc(rffi.CArray(PAGE_PTR), length,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
self.nblocks_for_size = lltype.malloc(rffi.CArray(lltype.Signed),
- length, flavor='raw')
+ length, flavor='raw',
+ immortal=True)
self.hdrsize = llmemory.raw_malloc_usage(llmemory.sizeof(PAGE_HEADER))
assert page_size > self.hdrsize
self.nblocks_for_size[0] = 0 # unused
@@ -114,11 +117,13 @@
self.max_pages_per_arena = arena_size // page_size
self.arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
self.max_pages_per_arena,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
# this is used in mass_free() only
self.old_arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
self.max_pages_per_arena,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
#
# the arena currently consumed; it must have at least one page
# available, or be NULL. The arena object that we point to is
@@ -281,7 +286,7 @@
npages = (arena_end - firstpage) // self.page_size
#
# Allocate an ARENA object and initialize it
- arena = lltype.malloc(ARENA, flavor='raw')
+ arena = lltype.malloc(ARENA, flavor='raw', track_allocation=False)
arena.base = arena_base
arena.nfreepages = 0 # they are all uninitialized pages
arena.totalpages = npages
@@ -332,7 +337,7 @@
#
# The whole arena is empty. Free it.
llarena.arena_free(arena.base)
- lltype.free(arena, flavor='raw')
+ lltype.free(arena, flavor='raw', track_allocation=False)
#
else:
# Insert 'arena' in the correct arenas_lists[n]
From cfbolz at codespeak.net Fri Oct 15 16:47:12 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 15 Oct 2010 16:47:12 +0200 (CEST)
Subject: [pypy-svn] r77993 - pypy/extradoc/talk/pepm2011
Message-ID: <20101015144712.4A62636E0C1@codespeak.net>
Author: cfbolz
Date: Fri Oct 15 16:47:10 2010
New Revision: 77993
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
incorporate first round of comments by stephan
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 15 16:47:10 2010
@@ -106,7 +106,7 @@
The performance of many dynamic language implementations suffers from
high allocation rates and runtime type checks. This makes dynamic
languages less applicable to purely algorithmic problems, despite their
-growing popularity. In this paper, we present a simple optimization
+growing popularity. In this paper we present a simple compiler optimization
based on online partial evaluation to remove object allocations and
runtime type checks in the context of a tracing JIT. We evaluate the
optimization using a Python VM and find that it gives good results for
@@ -130,7 +130,7 @@
\section{Introduction}
-The goal of a just-in-time (JIT) compiler for a dynamic language is obviously to
+The objective of a just-in-time (JIT) compiler for a dynamic language is to
improve the speed of the language over an implementation of the language that
uses interpretation. The first goal of a JIT is therefore to remove the
interpretation overhead, i.e. the overhead of bytecode (or AST) dispatch and the
@@ -142,38 +142,37 @@
Boxing of primitive types is necessary because dynamic languages need to be able to handle
all objects, even integers, floats, booleans etc. in the same way as user-defined
-instances. Thus those primitive types are usually \emph{boxed}, i.e. a small
-heap-structure is allocated for them, that contains the actual value. Boxing
+instances. Thus those primitive types are usually \emph{boxed}, \ie a small
+heap-structure is allocated for them that contains the actual value. Boxing
primitive types can be very costly, because a lot of common operations,
-particularly all arithmetic operations, have to produce a new box, in addition
+particularly all arithmetic operations, have to produce new boxes, in addition
to the actual computation they do. Because the boxes are allocated on the heap,
-producing a lot of them puts pressure on the garbage collector.
+producing many of them puts pressure on the garbage collector.
Type dispatching is the process of finding the concrete implementation that is
-applicable to the objects at hand when doing a generic operation on them. An
-example would be the addition of two objects: The addition needs to check what
-the concrete objects that should be added are, and choose the implementation
-that is fitting for them. Type dispatching is a very common operation in
+applicable to the objects at hand when performing a generic operation on them. An
+example would be the addition of two objects: For addition the types of the
+concrete objects need to be checked and the suiting implementation chosen.
+Type dispatching is a very common operation in
modern\footnote{For languages in the LISP family, basic arithmetic operations
are typically not overloaded; even in Smalltalk, type dispatching is much
simpler than in Python or JavaScript.}
-dynamic languages because no types are known at compile time, so all operations
-need it.
+dynamic languages because no types are known at compile time. Therefore all
+operations need it.
A recently popular approach to implementing just-in-time compilers for dynamic
languages is that of a tracing JIT. A tracing JIT works by observing the running
-program and recording its hot spots into linear execution traces. Working on
-traces is the central idea of a tracing JIT. Those traces are optimized and
-turned into machine code.
+program and recording its hot spots into \emph{linear execution traces}. Those
+traces are optimized and turned into machine code.
One reason for the popularity of tracing JITs is their relative
-simplicity. They can often be added to an interpreter and a lot of the
-infrastructure of the interpreter can be reused. They give some important
+simplicity. They can often be added to an existing interpreter, reusing a lot of
+the interpreter's infrastructure. They give some important
optimizations like inlining and constant-folding for free. A tracing JIT always
-produces linear pieces of code, which simplifies many algorithms that are usually
-hard in a compiler, such as register allocation.
+produces linear pieces of code, which simplifies many of the hard algorithms in
+a compiler, such as register allocation.
-The usage of a tracing JIT can remove the overhead of bytecode dispatch and that
+The use of a tracing JIT can remove the overhead of bytecode dispatch and that
of the interpreter data structures. In this paper we want to present a new
optimization that can be added to a tracing JIT that further removes some of the
overhead more closely associated to dynamic languages, such as boxing overhead
@@ -190,14 +189,15 @@
informally described in Section~\ref{sec:statics}; a more formal description is
given in Section~\ref{sec:formal}. The introduced
techniques are evaluated in Section~\ref{sec:Evaluation} using PyPy's Python
-interpreter as a case study.
+interpreter.
-The contributions of this paper are:
+The contributions made by this paper are:
\begin{enumerate}
- \item An efficient and effective algorithm for removing object allocations in a tracing JIT.
+ \item A description of an efficient and effective algorithm for removing
+ object allocations in a tracing JIT.
\item A characterization of this algorithm as partial evaluation.
- \item A rigorous evaluation of this algorithm.
+ \item Performance benchmarks for this algorithm.
\end{enumerate}
@@ -215,7 +215,7 @@
\emph{RPython} \cite{davide_ancona_rpython:_2007}. RPython ("restricted Python")
is a subset of Python chosen in such a way that type inference becomes
possible. The language interpreter can then be compiled (``translated'') with
-PyPy's tools into a VM on the C level. During translation to C, many low-level
+PyPy's tools into a VM on C level. During translation to C, many low-level
aspects of the final VM, such as object layout, garbage collection and memory
model, are woven into the generated code. Therefore the interpreter itself can
remain at a relatively high level of abstraction.
@@ -234,13 +234,13 @@
language that the interpreter is implementing. This process is mostly
automatic; it only needs to be guided by the language implementer using a small number of
source-code hints. Mostly-automatically generating a JIT compiler has many advantages
-over writing one manually, which is an error-prone and tedious process.
+over writing one manually, an error-prone and tedious process.
By construction, the generated JIT has the same semantics as the interpreter.
-Many optimizations can benefit all languages implemented as an interpreter in RPython.
+Optimizations can be shared between different languages implemented with PyPy.
Moreover, thanks to the internal design of the JIT generator, it is very easy
to add new \emph{backends} for producing the actual machine code. Examples of
-JIT backends that are implemented are the one for Intel x86 and x86-64 and an
+JIT backends that are implemented are those for Intel x86 and x86-64 and an
experimental one for the CLI .NET Virtual Machine \cite{cuni_high_2010}.
\subsection{Tracing JIT Compilers}
@@ -256,7 +256,7 @@
and now Python (and other languages) via PyPy.
The core idea of tracing JITs is to focus the optimization effort of the JIT
-compiler on the hot paths of the core loops of the program and to just use an
+compiler on the commonly executed, \ie \emph{hot} paths of the core loops of the program and to just use an
interpreter for the less commonly executed parts. VMs that use a tracing JIT are
mostly mixed-mode execution environments, they contain both an interpreter and a
JIT compiler. By default the interpreter is used to execute the program, doing
@@ -269,24 +269,23 @@
it always ends with a jump to its own beginning. The trace also contains all
operations that are performed in functions that were called in the loop, thus a
tracing JIT automatically performs inlining.
-
-This trace of operations is then the basis of the generated code. The trace is
+This trace of operations subsequently forms the basis of the generated code. The trace is
first optimized, and then turned into machine code. Both optimization
and machine code generation are simple, because the traces are linear. This
linearity makes many optimizations a lot more tractable, and the inlining that
happens gives the optimizations automatically more context to work with.
Since the trace corresponds to one concrete execution of a loop,
-the code generated from it is only one possible path through it.
-To make sure that the trace is maintaining the correct semantics, it contains a
+the code generated from it is only one possible path through the loop.
+To make sure that the trace maintains the correct semantics, it contains a
\emph{guard} at all places where the execution could have diverged from the
path. Those guards check the assumptions under which execution can stay on the
-trace. As an example, if a loop contains an \lstinline{if} statement, the trace
+trace. As an example, if a loop contains an if-statement, the trace
will contain the execution of one of the paths only, which is the path that was
taken during the production of the trace. The trace will also contain a guard
-that checks that the condition of the \lstinline{if} statement is the same as
+that checks that the condition of the if-statement is the same as
during tracing, because if
-it isn't, the rest of the trace is not valid. \cfbolz{The "if" shouldn't be bold}
+it isn't, the rest of the trace would not be valid.
When generating machine code, every guard is be turned into a quick check to
see whether the assumption still holds. When such a guard is hit during the
@@ -367,11 +366,11 @@
\label{fig:objmodel}
\end{figure}
-Using these classes to implement arithmetic shows the basic problem that a
-dynamic language implementation has. All the numbers are instances of either
+Using these classes to implement arithmetic shows the basic problem of a
+dynamic language implementation. All the numbers are instances of either
\lstinline{BoxedInteger} or \lstinline{BoxedFloat}, therefore they consume space on the
heap. Performing many arithmetic operations produces lots of garbage quickly,
-which puts pressure on the garbage collector. Using double dispatching to
+putthing pressure on the garbage collector. Using double dispatching to
implement the numeric tower needs two method calls per arithmetic operation,
which is costly due to the method dispatch.
@@ -384,7 +383,7 @@
calls inside the loop, one for each \lstinline{is_positive} and even two for each
call to \lstinline{add}. These method calls need to check the type of the involved
objects repeatedly and redundantly. In addition, a lot of objects are created
-when executing that loop, many of these objects do not survive for very long.
+when executing that loop, many of these objects are short-lived.
The actual computation that is performed by \lstinline{f} is simply a sequence of
float or integer additions.
@@ -589,7 +588,7 @@
the type check the guard does is statically known.
In the example from last section, the following operations in the upper half
-of Fig.~\ref{fig:unopt-trace} produce two
+of Figure~\ref{fig:unopt-trace} produce two
static objects, and can be completely removed from the optimized trace:
\begin{lstlisting}[mathescape,xleftmargin=20pt]
@@ -605,7 +604,7 @@
one associated with $p_{6}$ would know that it is a \lstinline{BoxedInteger}
whose \lstinline{intval} field contains the constant -100.
-The subsequent operations in Fig.~\ref{fig:unopt-trace},
+The subsequent operations in Figure~\ref{fig:unopt-trace},
which use $p_{5}$ and $p_{6}$, could then be
optimized using that knowledge:
@@ -628,7 +627,7 @@
$i_{9}$ = int_add($i_{4}$, -100)
\end{lstlisting}
-The rest of the trace from Fig.~\ref{fig:unopt-trace} is optimized similarly.
+The rest of the trace from Figure~\ref{fig:unopt-trace} is optimized similarly.
So far we have only described what happens when static objects are used in guards and in
operations that read and write fields. When the static
@@ -640,7 +639,7 @@
necessary to put operations into the residual code that allocate the
static object at runtime.
-This is what happens at the end of the trace in Fig.~\ref{fig:unopt-trace}, when the \lstinline{jump} operation
+This is what happens at the end of the trace in Figure~\ref{fig:unopt-trace}, when the \lstinline{jump} operation
is hit. The arguments of the jump are at this point static objects. Before the
jump is emitted, they are \emph{lifted}. This means that the optimizer produces code
that allocates a new object of the right type and sets its fields to the field
@@ -897,7 +896,7 @@
\end{lstlisting}
In this case, the static heap afterwards would be
-$\{v^* \mapsto (T_1, w^*, v^*)\}$.
+$$\{v^* \mapsto (T_1, w^*, v^*)\}$$.
From cfbolz at codespeak.net Fri Oct 15 17:17:15 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 15 Oct 2010 17:17:15 +0200 (CEST)
Subject: [pypy-svn] r77994 - pypy/extradoc/talk/pepm2011
Message-ID: <20101015151715.6408E36E0C1@codespeak.net>
Author: cfbolz
Date: Fri Oct 15 17:17:13 2010
New Revision: 77994
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
more suggestions by stephan
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 15 17:17:13 2010
@@ -374,10 +374,8 @@
implement the numeric tower needs two method calls per arithmetic operation,
which is costly due to the method dispatch.
-To understand the problems more directly, let us consider the simple
-interpreter function \lstinline{f} that uses the object model (see the bottom of
-Figure~\ref{fig:objmodel}).
-
+Let us now consider a simple interpreter function \lstinline{f} that uses the
+object model (see the bottom of Figure~\ref{fig:objmodel}).
The loop in \lstinline{f} iterates \lstinline{y} times, and computes something in the process.
Simply running this function is slow, because there are lots of virtual method
calls inside the loop, one for each \lstinline{is_positive} and even two for each
@@ -451,24 +449,25 @@
\end{figure}
If the function is executed using the tracing JIT, with \lstinline{y} being a
-\lstinline{BoxedInteger}, the produced trace looks like
-Figure~\ref{fig:unopt-trace} (lines starting with the hash ``\#'' are comments).
+\lstinline{BoxedInteger}, the produced trace looks like the one of
+Figure~\ref{fig:unopt-trace} (lines starting with a hash ``\#'' are comments).
+The trace corresponds to one iteration of the while-loop in \lstinline{f}.
-The operations in the trace are shown indented to
-correspond to the stack level of the function that contains the traced
+The operations in the trace are indented
+corresponding to the stack level of the function that contains the traced
operation. The trace is in single-assignment form, meaning that each variable is
-assigned to exactly once. The arguments $p_0$ and $p_1$ of the loop correspond
+assigned a value exactly once. The arguments $p_0$ and $p_1$ of the loop correspond
to the live variables \lstinline{y} and \lstinline{res} in the original function.
The operations in the trace correspond to the operations in the RPython program
in Figure~\ref{fig:objmodel}:
\begin{itemize}
- \item \lstinline{new} corresponds to object creation.
- \item \lstinline{get} correspond to attribute reads.
- \item \lstinline{set} correspond to attribute writes.
- \item \lstinline{guard_class} correspond to method calls and are followed by
- the trace of the called method.
+ \item \lstinline{new} creates a new object.
+ \item \lstinline{get} reads an attribute of an object.
+ \item \lstinline{set} writes to an attribute of an object.
+ \item \lstinline{guard_class} precedes an (inlined) method call and is
+ followed by the trace of the called method.
\item \lstinline{int_add} and \lstinline{int_gt} are integer addition and
comparison (``greater than''), respectively.
\end{itemize}
@@ -477,9 +476,9 @@
operation, to check that the class of the receiver is the same as the one that
was observed during tracing.\footnote{\lstinline{guard_class} performs a precise
class check, not checking for subclasses.} These guards make the trace specific
-to the situation where \lstinline{y} is really a \lstinline{BoxedInteger}, it can
-already be said to be specialized for \lstinline{BoxedIntegers}. When the trace is
-turned into machine code and then executed with \lstinline{BoxedFloats}, the
+to the situation where \lstinline{y} is really a \lstinline{BoxedInteger}. When
+the trace is turned into machine code and afterwards executed with
+\lstinline{BoxedFloat}, the
first \lstinline{guard_class} instruction will fail and execution will continue
using the interpreter.
@@ -488,8 +487,8 @@
operations. The number of \lstinline{guard_class} operation is particularly
problematic, not only because of the time it takes to run them. All guards also
have additional information attached that makes it possible to return to the
-interpreter, should the guard fail. This means that many guard operations also
-lead to a memory problem.
+interpreter, should the guard fail. This means that too many guard operations also
+consume a lot of memory.
In the rest of the paper we will see how this trace can be optimized using
partial evaluation.
@@ -499,7 +498,7 @@
% section Object Lifetimes in a Tracing JIT (end)
-To understand the problems that this paper is trying to solve some more, we
+To understand the problems that this paper is trying to solve in more detail, we
first need to understand various cases of object lifetimes that can occur in a
tracing JIT compiler.
@@ -520,7 +519,7 @@
aborted and interpretation resumes.
Some of the operations within this trace are \lstinline{new} operations, which each
-create a new instance of some class. These instances are used for a while, e.g.
+create a new instance of some class. These instances are used for some time, \eg
by calling methods on them (which are inlined into the trace), reading and
writing their fields. Some of these instances \emph{escape}, which means that
they are stored in some globally accessible place or are passed into a
@@ -530,19 +529,19 @@
created objects. The objects that are created within a trace using \lstinline{new}
fall into one of several categories:
-\begin{itemize}
- \item Category 1: Objects that live for a while, and are then just not
- used any more.
+\begin{enumerate}
+ \item Objects that live for some time, and are then just not
+ used any more afterwards.
- \item Category 2: Objects that live for a while and then escape.
+ \item Objects that live for some time and then escape.
- \item Category 3: Objects that live for a while, survive across the jump to
+ \item Objects that live for some time, survive across the jump to
the beginning of the loop, and are then not used any more.
- \item Category 4: Objects that live for a while, survive across the jump,
+ \item Objects that live for some time, survive across the jump,
and then escape. To these we also count the objects that live across several
jumps and then either escape or stop being used.
-\end{itemize}
+\end{enumerate}
The objects that are allocated in the example trace in
Figure~\ref{fig:unopt-trace} fall into categories 1 and 3. Objects stored in
@@ -551,7 +550,9 @@
The creation of objects in category 1 is removed by the optimization described
in Sections~\ref{sec:statics} and \ref{sec:formal}. Objects in the other
-categories are partially optimized by this approach as well.
+categories are partially optimized by this approach as well.\footnote{We also started to
+work on optimizing objects in category 3, which will be the subject of a later
+paper.}
\section{Allocation Removal in Traces}
\label{sec:statics}
@@ -895,8 +896,8 @@
set($w^*$, $R$, $u^*$)
\end{lstlisting}
-In this case, the static heap afterwards would be
-$$\{v^* \mapsto (T_1, w^*, v^*)\}$$.
+In this case, the static heap afterwards would be:
+$$\{v^* \mapsto (T_1, w^*, v^*)\}$$
From arigo at codespeak.net Fri Oct 15 17:19:36 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 17:19:36 +0200 (CEST)
Subject: [pypy-svn] r77995 - pypy/branch/leak-finder/pypy/jit/codewriter
Message-ID: <20101015151936.38FBC36E0C1@codespeak.net>
Author: arigo
Date: Fri Oct 15 17:19:34 2010
New Revision: 77995
Modified:
pypy/branch/leak-finder/pypy/jit/codewriter/jtransform.py
Log:
Fix.
Modified: pypy/branch/leak-finder/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/branch/leak-finder/pypy/jit/codewriter/jtransform.py Fri Oct 15 17:19:34 2010
@@ -429,7 +429,8 @@
op.result)
def rewrite_op_free(self, op):
- assert op.args[1].value == 'raw'
+ flags = op.args[1].value
+ assert flags['flavor'] == 'raw'
ARRAY = op.args[0].concretetype.TO
return self._do_builtin_call(op, 'raw_free', [op.args[0]],
extra = (ARRAY,), extrakey = ARRAY)
From arigo at codespeak.net Fri Oct 15 17:21:24 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 17:21:24 +0200 (CEST)
Subject: [pypy-svn] r77996 - pypy/branch/leak-finder/pypy/module/_socket/test
Message-ID: <20101015152124.56B8F36E0C1@codespeak.net>
Author: arigo
Date: Fri Oct 15 17:21:22 2010
New Revision: 77996
Modified:
pypy/branch/leak-finder/pypy/module/_socket/test/test_sock_app.py
Log:
Fix the test.
Modified: pypy/branch/leak-finder/pypy/module/_socket/test/test_sock_app.py
==============================================================================
--- pypy/branch/leak-finder/pypy/module/_socket/test/test_sock_app.py (original)
+++ pypy/branch/leak-finder/pypy/module/_socket/test/test_sock_app.py Fri Oct 15 17:21:22 2010
@@ -254,6 +254,7 @@
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
fd = s.fileno()
w_obj = rsocket.make_address(c_addr, addrlen).as_object(fd, space)
+ lltype.free(c_addr_ll, flavor='raw')
assert space.is_true(space.eq(w_obj, space.newtuple([
space.wrap('lo'),
space.wrap(socket.ntohs(8)),
From cfbolz at codespeak.net Fri Oct 15 17:22:31 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 15 Oct 2010 17:22:31 +0200 (CEST)
Subject: [pypy-svn] r77997 - pypy/extradoc/talk/pepm2011
Message-ID: <20101015152231.3FCC536E0C1@codespeak.net>
Author: cfbolz
Date: Fri Oct 15 17:22:29 2010
New Revision: 77997
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
make the paper sort of presentable
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 15 17:22:29 2010
@@ -1113,7 +1113,6 @@
garbage-collection pressure but does not optimize away the actual accesses to
the stack-allocated object. In our case, an object is not needed at all any
more.
-XXX more papers?
Chang \etal describe a tracing JIT for JavaScript running on top of a JVM
\cite{mason_chang_efficient_2007}. They mention in passing an approach to
@@ -1142,8 +1141,6 @@
short-lived, but the type check removals are similar to what our optimization
achieves.
-partial evaluation:
-
Partially known data structures are built directly into Prolog (via unbound
logic variables) and thus the treatment of partially static data structures
was part of partial evaluation of Prolog programs from the early stages
@@ -1152,15 +1149,15 @@
residual program. This is similar to what our optimization does for an
imperative language.
In functional programming this idea was introduced as
- constructor specialisation by Mogensen \cite{Mogensen:PEPM93}.)
+ constructor specialisation by Mogensen \cite{Mogensen:PEPM93}.
%xxx:
%partially static data structures: kenichi asai's thesis?
-xxx:
-Other related work is compile-time garbage collection \cite{mazur_practical_2001}, which
-tries to ....;
- separation logic
+%xxx:
+%Other related work is compile-time garbage collection \cite{mazur_practical_2001}, which
+%tries to ....;
+% separation logic
%; John Hughes: type specialization
\section{Conclusion}
From arigo at codespeak.net Fri Oct 15 17:30:43 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 17:30:43 +0200 (CEST)
Subject: [pypy-svn] r77998 - pypy/branch/leak-finder/pypy/module/thread
Message-ID: <20101015153043.52D1F36E0C1@codespeak.net>
Author: arigo
Date: Fri Oct 15 17:30:41 2010
New Revision: 77998
Modified:
pypy/branch/leak-finder/pypy/module/thread/ll_thread.py
Log:
Set track_allocation=False on the malloced locks.
Explain why.
Modified: pypy/branch/leak-finder/pypy/module/thread/ll_thread.py
==============================================================================
--- pypy/branch/leak-finder/pypy/module/thread/ll_thread.py (original)
+++ pypy/branch/leak-finder/pypy/module/thread/ll_thread.py Fri Oct 15 17:30:41 2010
@@ -111,7 +111,7 @@
c_thread_releaselock(self._lock)
def __del__(self):
- lltype.free(self._lock, flavor='raw')
+ lltype.free(self._lock, flavor='raw', track_allocation=False)
# ____________________________________________________________
#
@@ -128,10 +128,13 @@
null_ll_lock = lltype.nullptr(TLOCKP.TO)
def allocate_ll_lock():
- ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw')
+ # track_allocation=False here; be careful to lltype.free() it. The
+ # reason it is set to False is that we get it from all app-level
+ # lock objects, as well as from the GIL, which exists at shutdown.
+ ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw', track_allocation=False)
res = c_thread_lock_init(ll_lock)
if res == -1:
- lltype.free(ll_lock, flavor='raw')
+ lltype.free(ll_lock, flavor='raw', track_allocation=False)
raise error("out of resources")
return ll_lock
From arigo at codespeak.net Fri Oct 15 17:35:04 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 17:35:04 +0200 (CEST)
Subject: [pypy-svn] r77999 - pypy/branch/leak-finder/pypy/translator/c/test
Message-ID: <20101015153504.16F2836E0C1@codespeak.net>
Author: arigo
Date: Fri Oct 15 17:35:02 2010
New Revision: 77999
Modified:
pypy/branch/leak-finder/pypy/translator/c/test/test_genc.py
Log:
Fix test.
Modified: pypy/branch/leak-finder/pypy/translator/c/test/test_genc.py
==============================================================================
--- pypy/branch/leak-finder/pypy/translator/c/test/test_genc.py (original)
+++ pypy/branch/leak-finder/pypy/translator/c/test/test_genc.py Fri Oct 15 17:35:02 2010
@@ -426,6 +426,7 @@
if py.test.config.option.view:
t.view()
assert ' BarStruct ' in t.driver.cbuilder.c_source_filename.read()
+ free(foo, flavor="raw")
def test_recursive_llhelper():
from pypy.rpython.annlowlevel import llhelper
@@ -473,7 +474,7 @@
return f(s)
a_f = A(f, "f")
a_g = A(g, "g")
- t = lltype.malloc(STRUCT, flavor="raw")
+ t = lltype.malloc(STRUCT, flavor="raw", immortal=True)
t.bar = llhelper(FTPTR, a_f.make_func())
fn = compile(chooser, [bool])
assert fn(True)
From arigo at codespeak.net Fri Oct 15 17:36:31 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 17:36:31 +0200 (CEST)
Subject: [pypy-svn] r78000 -
pypy/branch/leak-finder/pypy/rpython/memory/gctransform
Message-ID: <20101015153631.7635E36E0C1@codespeak.net>
Author: arigo
Date: Fri Oct 15 17:36:29 2010
New Revision: 78000
Modified:
pypy/branch/leak-finder/pypy/rpython/memory/gctransform/asmgcroot.py
Log:
Fix.
Modified: pypy/branch/leak-finder/pypy/rpython/memory/gctransform/asmgcroot.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/memory/gctransform/asmgcroot.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/memory/gctransform/asmgcroot.py Fri Oct 15 17:36:29 2010
@@ -75,7 +75,8 @@
key = (TYPE, num)
if key not in sradict:
CONTAINER = lltype.FixedSizeArray(TYPE, 1)
- p = lltype.malloc(CONTAINER, flavor='raw', zero=True)
+ p = lltype.malloc(CONTAINER, flavor='raw', zero=True,
+ immortal=True)
sradict[key] = Constant(p, lltype.Ptr(CONTAINER))
sra.append(sradict[key])
#
From afa at codespeak.net Fri Oct 15 18:00:15 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 15 Oct 2010 18:00:15 +0200 (CEST)
Subject: [pypy-svn] r78001 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101015160015.7FBC836E0C1@codespeak.net>
Author: afa
Date: Fri Oct 15 18:00:10 2010
New Revision: 78001
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
Log:
_io.open() now accepts a file descriptor
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Fri Oct 15 18:00:10 2010
@@ -88,6 +88,8 @@
if space.isinstance_w(w_name, space.w_float):
raise OperationError(space.w_TypeError, space.wrap(
"integer argument expected, got float"))
+
+ fd = -1
try:
fd = space.int_w(w_name)
except OperationError, e:
@@ -99,12 +101,16 @@
self.readable, self.writable, flags = decode_mode(space, mode)
- from pypy.module.posix.interp_posix import dispatch_filename, rposix
- try:
- self.fd = dispatch_filename(rposix.open)(
- space, w_name, flags, 0666)
- except OSError, e:
- raise wrap_oserror2(space, e, w_name)
+ if fd >= 0:
+ self.fd = fd
+ else:
+ from pypy.module.posix.interp_posix import (
+ dispatch_filename, rposix)
+ try:
+ self.fd = dispatch_filename(rposix.open)(
+ space, w_name, flags, 0666)
+ except OSError, e:
+ raise wrap_oserror2(space, e, w_name)
self.closefd = bool(closefd)
def _check_closed(self, space):
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Fri Oct 15 18:00:10 2010
@@ -1,4 +1,5 @@
from pypy.conftest import gettestobjspace
+import os
class AppTestIoModule:
def setup_class(cls):
@@ -42,8 +43,18 @@
from pypy.tool.udir import udir
tmpfile = udir.join('tmpfile').ensure()
cls.w_tmpfile = cls.space.wrap(str(tmpfile))
+ cls.w_posix = cls.space.appexec([], """():
+ import %s as m;
+ return m""" % os.name)
def test_open(self):
import io
f = io.open(self.tmpfile, "rb")
f.close()
+
+ def test_open_fd(self):
+ import io
+ os = self.posix
+ fd = os.open(self.tmpfile, os.O_RDONLY, 0666)
+ f = io.open(fd, "rb")
+ f.close()
From arigo at codespeak.net Fri Oct 15 18:35:46 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 18:35:46 +0200 (CEST)
Subject: [pypy-svn] r78002 - in pypy/branch/leak-finder/pypy: rlib
rpython/lltypesystem rpython/memory/test
Message-ID: <20101015163546.B0D7D36E0C1@codespeak.net>
Author: arigo
Date: Fri Oct 15 18:35:43 2010
New Revision: 78002
Modified:
pypy/branch/leak-finder/pypy/rlib/rgc.py
pypy/branch/leak-finder/pypy/rpython/lltypesystem/rffi.py
pypy/branch/leak-finder/pypy/rpython/memory/test/test_gc.py
Log:
Test and fix for *a real leak*! Yay :-)
Modified: pypy/branch/leak-finder/pypy/rlib/rgc.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rlib/rgc.py (original)
+++ pypy/branch/leak-finder/pypy/rlib/rgc.py Fri Oct 15 18:35:43 2010
@@ -170,7 +170,14 @@
return hop.genop('gc_set_max_heap_size', [v_nbytes],
resulttype=lltype.Void)
-def can_move(p): # NB. must not be called with NULL pointers
+def can_move(p):
+ """Check if the GC object 'p' is at an address that can move.
+ Must not be called with None. With non-moving GCs, it is always False.
+ With some moving GCs like the SemiSpace GC, it is always True.
+ With other moving GCs like the MiniMark GC, it can be True for some
+ time, then False for the same object, when we are sure that it won't
+ move any more.
+ """
return True
class CanMoveEntry(ExtRegistryEntry):
Modified: pypy/branch/leak-finder/pypy/rpython/lltypesystem/rffi.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/lltypesystem/rffi.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/lltypesystem/rffi.py Fri Oct 15 18:35:43 2010
@@ -644,7 +644,14 @@
"""
Either free a non-moving buffer or keep the original storage alive.
"""
- if rgc.can_move(data):
+ # We cannot rely on rgc.can_move(data) here, because its result
+ # might have changed since get_nonmovingbuffer(). Instead we check
+ # if 'buf' points inside 'data'. This is only possible if we
+ # followed the 2nd case in get_nonmovingbuffer(); in the first case,
+ # 'buf' points to its own raw-malloced memory.
+ data_start = cast_ptr_to_adr(llstrtype(data)) + \
+ offsetof(STRTYPE, 'chars') + itemoffsetof(STRTYPE.chars, 0)
+ if buf != cast(TYPEP, data_start):
lltype.free(buf, flavor='raw')
else:
keepalive_until_here(data)
Modified: pypy/branch/leak-finder/pypy/rpython/memory/test/test_gc.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/memory/test/test_gc.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/memory/test/test_gc.py Fri Oct 15 18:35:43 2010
@@ -368,6 +368,14 @@
res = self.interpret(f, [4, 42])
assert res == 12
+ def test_print_leak(self):
+ def f(n):
+ for i in range(n):
+ print i
+ return 42
+ res = self.interpret(f, [10])
+ assert res == 42
+
def test_weakref_across_minor_collection(self):
import weakref
class A:
From arigo at codespeak.net Fri Oct 15 18:42:24 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 18:42:24 +0200 (CEST)
Subject: [pypy-svn] r78003 -
pypy/branch/leak-finder/pypy/rpython/lltypesystem
Message-ID: <20101015164224.C384C36E0C1@codespeak.net>
Author: arigo
Date: Fri Oct 15 18:42:23 2010
New Revision: 78003
Modified:
pypy/branch/leak-finder/pypy/rpython/lltypesystem/rffi.py
Log:
Fix.
Modified: pypy/branch/leak-finder/pypy/rpython/lltypesystem/rffi.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rpython/lltypesystem/rffi.py (original)
+++ pypy/branch/leak-finder/pypy/rpython/lltypesystem/rffi.py Fri Oct 15 18:42:23 2010
@@ -649,12 +649,13 @@
# if 'buf' points inside 'data'. This is only possible if we
# followed the 2nd case in get_nonmovingbuffer(); in the first case,
# 'buf' points to its own raw-malloced memory.
- data_start = cast_ptr_to_adr(llstrtype(data)) + \
+ data = llstrtype(data)
+ data_start = cast_ptr_to_adr(data) + \
offsetof(STRTYPE, 'chars') + itemoffsetof(STRTYPE.chars, 0)
- if buf != cast(TYPEP, data_start):
+ followed_2nd_path = (buf == cast(TYPEP, data_start))
+ keepalive_until_here(data)
+ if not followed_2nd_path:
lltype.free(buf, flavor='raw')
- else:
- keepalive_until_here(data)
# int -> (char*, str)
def alloc_buffer(count):
From arigo at codespeak.net Fri Oct 15 18:59:47 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 18:59:47 +0200 (CEST)
Subject: [pypy-svn] r78004 - pypy/branch/leak-finder/pypy/module/cpyext/test
Message-ID: <20101015165947.60F5B36E0C1@codespeak.net>
Author: arigo
Date: Fri Oct 15 18:59:45 2010
New Revision: 78004
Modified:
pypy/branch/leak-finder/pypy/module/cpyext/test/test_cpyext.py
Log:
Fix this test to depend on the default leakfinder.
Modified: pypy/branch/leak-finder/pypy/module/cpyext/test/test_cpyext.py
==============================================================================
--- pypy/branch/leak-finder/pypy/module/cpyext/test/test_cpyext.py (original)
+++ pypy/branch/leak-finder/pypy/module/cpyext/test/test_cpyext.py Fri Oct 15 18:59:45 2010
@@ -16,6 +16,7 @@
from pypy.module.cpyext.pyobject import Py_DecRef, InvalidPointerException
from pypy.translator.goal import autopath
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
@api.cpython_api([], api.PyObject)
def PyPy_Crash1(space):
@@ -78,7 +79,6 @@
self.frozen_refcounts[w_obj] = obj.c_ob_refcnt
#state.print_refcounts()
self.frozen_ll2callocations = set(ll2ctypes.ALLOCATED.values())
- lltype.start_tracking_allocations()
class LeakCheckingTest(object):
def check_and_print_leaks(self):
@@ -126,17 +126,8 @@
for w_obj in lost_objects_w:
print >>sys.stderr, "Lost object %r" % (w_obj, )
leaking = True
-## for llvalue in set(ll2ctypes.ALLOCATED.values()) - self.frozen_ll2callocations:
-## if llvalue in lltype.ALLOCATED:
-## leaking = True
-## print >>sys.stderr, "Did not deallocate %r (ll2ctypes)" % (llvalue, )
-## print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines())
- for llvalue, traceback in lltype.ALLOCATED.items():
- leaking = True
- print >>sys.stderr, "Did not deallocate %r (llvalue)" % (llvalue, )
- print >>sys.stderr, "\t" + "\n\t".join(traceback.splitlines())
-
- lltype.stop_tracking_allocations(check=False)
+ # the actual low-level leak checking is done by pypy.tool.leakfinder,
+ # enabled automatically by pypy.conftest.
return leaking
class AppTestCpythonExtensionBase(LeakCheckingTest):
From arigo at codespeak.net Fri Oct 15 19:03:29 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 15 Oct 2010 19:03:29 +0200 (CEST)
Subject: [pypy-svn] r78005 - pypy/trunk/pypy/module/cpyext/test
Message-ID: <20101015170329.D9B7836E0C1@codespeak.net>
Author: arigo
Date: Fri Oct 15 19:03:28 2010
New Revision: 78005
Modified:
pypy/trunk/pypy/module/cpyext/test/test_unicodeobject.py
Log:
64-bit fix.
Modified: pypy/trunk/pypy/module/cpyext/test/test_unicodeobject.py
==============================================================================
--- pypy/trunk/pypy/module/cpyext/test/test_unicodeobject.py (original)
+++ pypy/trunk/pypy/module/cpyext/test/test_unicodeobject.py Fri Oct 15 19:03:28 2010
@@ -177,13 +177,14 @@
encoded_charp = rffi.str2charp(encoded)
strict_charp = rffi.str2charp("strict")
if endian is not None:
- pendian = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
if endian < 0:
- pendian[0] = -1
+ value = -1
elif endian > 0:
- pendian[0] = 1
+ value = 1
else:
- pendian[0] = 0
+ value = 0
+ pendian = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
+ pendian[0] = rffi.cast(rffi.INT, value)
else:
pendian = None
From afa at codespeak.net Fri Oct 15 19:26:06 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 15 Oct 2010 19:26:06 +0200 (CEST)
Subject: [pypy-svn] r78006 - pypy/branch/fast-forward/pypy/module/imp
Message-ID: <20101015172606.9244336E0C3@codespeak.net>
Author: afa
Date: Fri Oct 15 19:26:04 2010
New Revision: 78006
Modified:
pypy/branch/fast-forward/pypy/module/imp/__init__.py
Log:
Grab and release the import lock when forking the interpreter.
I could not find any test for this change, but:
- CPython 2.7 has similar code
- when translated, it helps a lot test_multiprocessing, which did not even start a single test before.
Modified: pypy/branch/fast-forward/pypy/module/imp/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/imp/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/imp/__init__.py Fri Oct 15 19:26:04 2010
@@ -38,9 +38,9 @@
def __init__(self, space, *args):
"NOT_RPYTHON"
MixedModule.__init__(self, space, *args)
- # from pypy.module.posix.interp_posix import add_fork_hook
- # from pypy.module.imp import interp_imp
- # add_fork_hook('before', interp_imp.acquire_lock)
- # add_fork_hook('parent', interp_imp.release_lock)
- # add_fork_hook('child', interp_imp.reinit_lock)
+ from pypy.module.posix.interp_posix import add_fork_hook
+ from pypy.module.imp import interp_imp
+ add_fork_hook('before', interp_imp.acquire_lock)
+ add_fork_hook('parent', interp_imp.release_lock)
+ add_fork_hook('child', interp_imp.reinit_lock)
From afa at codespeak.net Fri Oct 15 19:30:03 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 15 Oct 2010 19:30:03 +0200 (CEST)
Subject: [pypy-svn] r78007 - in pypy/branch/fast-forward:
lib-python/modified-2.7.0/test pypy/config
Message-ID: <20101015173003.0E60436E0CB@codespeak.net>
Author: afa
Date: Fri Oct 15 19:30:02 2010
New Revision: 78007
Added:
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_multiprocessing.py
- copied, changed from r77992, pypy/branch/fast-forward/lib-python/2.7.0/test/test_multiprocessing.py
Modified:
pypy/branch/fast-forward/pypy/config/pypyoption.py
Log:
Enable the _mulitprocessing module, and be careful to skip tests that are known to block.
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_multiprocessing.py (from r77992, pypy/branch/fast-forward/lib-python/2.7.0/test/test_multiprocessing.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/test/test_multiprocessing.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_multiprocessing.py Fri Oct 15 19:30:02 2010
@@ -501,6 +501,7 @@
p.join()
+ @unittest.skipIf(os.name == 'posix', "PYPY: FIXME")
def test_qsize(self):
q = self.Queue()
try:
@@ -521,6 +522,7 @@
time.sleep(DELTA)
q.task_done()
+ @unittest.skipIf(os.name == 'posix', "PYPY: FIXME")
def test_task_done(self):
queue = self.JoinableQueue()
@@ -1074,6 +1076,7 @@
class _TestPoolWorkerLifetime(BaseTestCase):
ALLOWED_TYPES = ('processes', )
+ @unittest.skipIf(os.name == 'posix', "PYPY: FIXME")
def test_pool_worker_lifetime(self):
p = multiprocessing.Pool(3, maxtasksperchild=10)
self.assertEqual(3, len(p._pool))
@@ -1260,6 +1263,7 @@
queue = manager.get_queue()
queue.put('hello world')
+ @unittest.skipIf(os.name == 'posix', "PYPY: FIXME")
def test_rapid_restart(self):
authkey = os.urandom(32)
manager = QueueManager(
@@ -1546,6 +1550,7 @@
ALLOWED_TYPES = ('processes',)
+ @unittest.skipIf(os.name == 'posix', "PYPY: FIXME")
def test_heap(self):
iterations = 5000
maxblocks = 50
Modified: pypy/branch/fast-forward/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/fast-forward/pypy/config/pypyoption.py (original)
+++ pypy/branch/fast-forward/pypy/config/pypyoption.py Fri Oct 15 19:30:02 2010
@@ -30,7 +30,8 @@
"rctime" , "select", "zipimport", "_lsprof",
"crypt", "signal", "_rawffi", "termios", "zlib", "bz2",
"struct", "_hashlib", "_md5", "_sha", "_minimal_curses", "cStringIO",
- "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array"]
+ "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array",
+ "_multiprocessing"]
))
working_oo_modules = default_modules.copy()
From afa at codespeak.net Fri Oct 15 22:26:52 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 15 Oct 2010 22:26:52 +0200 (CEST)
Subject: [pypy-svn] r78008 - pypy/branch/fast-forward/pypy/module/_io
Message-ID: <20101015202652.B96C036E35D@codespeak.net>
Author: afa
Date: Fri Oct 15 22:26:50 2010
New Revision: 78008
Added:
pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (contents, props changed)
Modified:
pypy/branch/fast-forward/pypy/module/_io/__init__.py
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
pypy/branch/fast-forward/pypy/module/_io/interp_io.py
Log:
Move _io.IOBase into its own file, like CPython's _io/iobase.c
Modified: pypy/branch/fast-forward/pypy/module/_io/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/__init__.py Fri Oct 15 22:26:50 2010
@@ -9,8 +9,8 @@
interpleveldefs = {
'DEFAULT_BUFFER_SIZE': 'space.wrap(interp_io.DEFAULT_BUFFER_SIZE)',
'BlockingIOError': 'interp_io.W_BlockingIOError',
- '_IOBase': 'interp_io.W_IOBase',
- '_RawIOBase': 'interp_io.W_RawIOBase',
+ '_IOBase': 'interp_iobase.W_IOBase',
+ '_RawIOBase': 'interp_iobase.W_RawIOBase',
'_BufferedIOBase': 'interp_io.W_BufferedIOBase',
'_TextIOBase': 'interp_io.W_TextIOBase',
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Fri Oct 15 22:26:50 2010
@@ -1,4 +1,4 @@
-from pypy.module._io.interp_io import W_RawIOBase
+from pypy.module._io.interp_iobase import W_RawIOBase
from pypy.interpreter.typedef import TypeDef
from pypy.interpreter.gateway import interp2app, unwrap_spec, Arguments
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
@@ -84,7 +84,7 @@
return space.wrap(self)
@unwrap_spec('self', ObjSpace, W_Root, str, int)
- def descr_init(self, space, w_name, mode, closefd):
+ def descr_init(self, space, w_name, mode='r', closefd=True):
if space.isinstance_w(w_name, space.w_float):
raise OperationError(space.w_TypeError, space.wrap(
"integer argument expected, got float"))
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_io.py Fri Oct 15 22:26:50 2010
@@ -1,10 +1,9 @@
-from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root
+from pypy.interpreter.baseobjspace import ObjSpace, W_Root
from pypy.interpreter.typedef import (
- TypeDef, interp_attrproperty, GetSetProperty, generic_new_descr)
+ TypeDef, interp_attrproperty, generic_new_descr)
from pypy.interpreter.gateway import interp2app, Arguments, unwrap_spec
-from pypy.interpreter.error import OperationError
from pypy.module.exceptions.interp_exceptions import W_IOError
-from pypy.tool.sourcetools import func_renamer
+from pypy.module._io.interp_iobase import W_IOBase
DEFAULT_BUFFER_SIZE = 8192
@@ -27,112 +26,6 @@
characters_written = interp_attrproperty('written', W_BlockingIOError),
)
-class W_IOBase(Wrappable):
- def __init__(self, space):
- # XXX: IOBase thinks it has to maintain its own internal state in
- # `__IOBase_closed` and call flush() by itself, but it is redundant
- # with whatever behaviour a non-trivial derived class will implement.
- self.__IOBase_closed = False
-
- def _closed(self, space):
- # This gets the derived attribute, which is *not* __IOBase_closed
- # in most cases!
- w_closed = space.findattr(self, space.wrap('closed'))
- if w_closed is not None and space.is_true(w_closed):
- return True
- return False
-
- def _CLOSED(self):
- # Use this macro whenever you want to check the internal `closed`
- # status of the IOBase object rather than the virtual `closed`
- # attribute as returned by whatever subclass.
- return self.__IOBase_closed
-
- def _check_closed(self, space):
- if self._closed(space):
- raise OperationError(
- space.w_ValueError,
- space.wrap("I/O operation on closed file"))
-
- def closed_get_w(space, self):
- return space.newbool(self.__IOBase_closed)
-
- @unwrap_spec('self', ObjSpace)
- def close_w(self, space):
- if self._CLOSED():
- return
- try:
- space.call_method(self, "flush")
- finally:
- self.__IOBase_closed = True
-
- @unwrap_spec('self', ObjSpace)
- def flush_w(self, space):
- if self._CLOSED():
- raise OperationError(
- space.w_ValueError,
- space.wrap("I/O operation on closed file"))
-
- @unwrap_spec('self', ObjSpace)
- def enter_w(self, space):
- self._check_closed(space)
- return space.wrap(self)
-
- @unwrap_spec('self', ObjSpace, Arguments)
- def exit_w(self, space, __args__):
- space.call_method(self, "close")
-
- @unwrap_spec('self', ObjSpace)
- def iter_w(self, space):
- self._check_closed(space)
- return space.wrap(self)
-
- @unwrap_spec('self', ObjSpace)
- def next_w(self, space):
- w_line = space.call_method(self, "readline")
- if space.int_w(space.len(w_line)) == 0:
- raise OperationError(space.w_StopIteration, space.w_None)
- return w_line
-
- @unwrap_spec('self', ObjSpace)
- def isatty_w(self, space):
- return space.w_False
-
- @unwrap_spec('self', ObjSpace)
- def readable_w(self, space):
- return space.w_False
-
- @unwrap_spec('self', ObjSpace)
- def writable_w(self, space):
- return space.w_False
-
- @unwrap_spec('self', ObjSpace)
- def seekable_w(self, space):
- return space.w_False
-
-W_IOBase.typedef = TypeDef(
- '_IOBase',
- __new__ = generic_new_descr(W_IOBase),
- __enter__ = interp2app(W_IOBase.enter_w),
- __exit__ = interp2app(W_IOBase.exit_w),
- __iter__ = interp2app(W_IOBase.iter_w),
- next = interp2app(W_IOBase.next_w),
- close = interp2app(W_IOBase.close_w),
- flush = interp2app(W_IOBase.flush_w),
- isatty = interp2app(W_IOBase.isatty_w),
- readable = interp2app(W_IOBase.readable_w),
- writable = interp2app(W_IOBase.writable_w),
- seekable = interp2app(W_IOBase.seekable_w),
- closed = GetSetProperty(W_IOBase.closed_get_w),
- )
-
-class W_RawIOBase(W_IOBase):
- pass
-W_RawIOBase.typedef = TypeDef(
- '_RawIOBase', W_IOBase.typedef,
- __new__ = generic_new_descr(W_RawIOBase),
- )
-
class W_BufferedIOBase(W_IOBase):
pass
Added: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Fri Oct 15 22:26:50 2010
@@ -0,0 +1,112 @@
+from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root
+from pypy.interpreter.typedef import (
+ TypeDef, GetSetProperty, generic_new_descr)
+from pypy.interpreter.gateway import interp2app, Arguments, unwrap_spec
+from pypy.interpreter.error import OperationError
+
+class W_IOBase(Wrappable):
+ def __init__(self, space):
+ # XXX: IOBase thinks it has to maintain its own internal state in
+ # `__IOBase_closed` and call flush() by itself, but it is redundant
+ # with whatever behaviour a non-trivial derived class will implement.
+ self.__IOBase_closed = False
+
+ def _closed(self, space):
+ # This gets the derived attribute, which is *not* __IOBase_closed
+ # in most cases!
+ w_closed = space.findattr(self, space.wrap('closed'))
+ if w_closed is not None and space.is_true(w_closed):
+ return True
+ return False
+
+ def _CLOSED(self):
+ # Use this macro whenever you want to check the internal `closed`
+ # status of the IOBase object rather than the virtual `closed`
+ # attribute as returned by whatever subclass.
+ return self.__IOBase_closed
+
+ def _check_closed(self, space):
+ if self._closed(space):
+ raise OperationError(
+ space.w_ValueError,
+ space.wrap("I/O operation on closed file"))
+
+ def closed_get_w(space, self):
+ return space.newbool(self.__IOBase_closed)
+
+ @unwrap_spec('self', ObjSpace)
+ def close_w(self, space):
+ if self._CLOSED():
+ return
+ try:
+ space.call_method(self, "flush")
+ finally:
+ self.__IOBase_closed = True
+
+ @unwrap_spec('self', ObjSpace)
+ def flush_w(self, space):
+ if self._CLOSED():
+ raise OperationError(
+ space.w_ValueError,
+ space.wrap("I/O operation on closed file"))
+
+ @unwrap_spec('self', ObjSpace)
+ def enter_w(self, space):
+ self._check_closed(space)
+ return space.wrap(self)
+
+ @unwrap_spec('self', ObjSpace, Arguments)
+ def exit_w(self, space, __args__):
+ space.call_method(self, "close")
+
+ @unwrap_spec('self', ObjSpace)
+ def iter_w(self, space):
+ self._check_closed(space)
+ return space.wrap(self)
+
+ @unwrap_spec('self', ObjSpace)
+ def next_w(self, space):
+ w_line = space.call_method(self, "readline")
+ if space.int_w(space.len(w_line)) == 0:
+ raise OperationError(space.w_StopIteration, space.w_None)
+ return w_line
+
+ @unwrap_spec('self', ObjSpace)
+ def isatty_w(self, space):
+ return space.w_False
+
+ @unwrap_spec('self', ObjSpace)
+ def readable_w(self, space):
+ return space.w_False
+
+ @unwrap_spec('self', ObjSpace)
+ def writable_w(self, space):
+ return space.w_False
+
+ @unwrap_spec('self', ObjSpace)
+ def seekable_w(self, space):
+ return space.w_False
+
+W_IOBase.typedef = TypeDef(
+ '_IOBase',
+ __new__ = generic_new_descr(W_IOBase),
+ __enter__ = interp2app(W_IOBase.enter_w),
+ __exit__ = interp2app(W_IOBase.exit_w),
+ __iter__ = interp2app(W_IOBase.iter_w),
+ next = interp2app(W_IOBase.next_w),
+ close = interp2app(W_IOBase.close_w),
+ flush = interp2app(W_IOBase.flush_w),
+ isatty = interp2app(W_IOBase.isatty_w),
+ readable = interp2app(W_IOBase.readable_w),
+ writable = interp2app(W_IOBase.writable_w),
+ seekable = interp2app(W_IOBase.seekable_w),
+ closed = GetSetProperty(W_IOBase.closed_get_w),
+ )
+
+class W_RawIOBase(W_IOBase):
+ pass
+W_RawIOBase.typedef = TypeDef(
+ '_RawIOBase', W_IOBase.typedef,
+ __new__ = generic_new_descr(W_RawIOBase),
+ )
+
From afa at codespeak.net Fri Oct 15 22:47:55 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 15 Oct 2010 22:47:55 +0200 (CEST)
Subject: [pypy-svn] r78009 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101015204755.ADBCC36E363@codespeak.net>
Author: afa
Date: Fri Oct 15 22:47:54 2010
New Revision: 78009
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
Log:
More progress in the _io module,
one more instruction passed by some functions of the test suite.
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Fri Oct 15 22:47:54 2010
@@ -3,6 +3,7 @@
from pypy.interpreter.gateway import interp2app, unwrap_spec, Arguments
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
from pypy.interpreter.error import OperationError, wrap_oserror2
+from pypy.rlib.rarithmetic import r_longlong
from os import O_RDONLY, O_WRONLY, O_RDWR, O_CREAT, O_TRUNC
import os
@@ -118,6 +119,12 @@
raise OperationError(space.w_ValueError, space.wrap(
"I/O operation on closed file"))
+ @unwrap_spec('self', ObjSpace, r_longlong, int)
+ def seek_w(self, space, pos, whence):
+ self._check_closed(space)
+ pos = os.lseek(self.fd, pos, whence)
+ return space.wrap(pos)
+
@unwrap_spec('self', ObjSpace)
def readable_w(self, space):
self._check_closed(space)
@@ -128,11 +135,27 @@
self._check_closed(space)
return space.wrap(self.writable)
+ @unwrap_spec('self', ObjSpace)
+ def seekable_w(self, space):
+ self._check_closed(space)
+ if self.seekable < 0:
+ try:
+ pos = os.lseek(self.fd, 0, os.SEEK_CUR)
+ except OSError:
+ self.seekable = 0
+ else:
+ self.seekable = 1
+ return space.newbool(self.seekable)
+
W_FileIO.typedef = TypeDef(
'FileIO', W_RawIOBase.typedef,
__new__ = interp2app(W_FileIO.descr_new.im_func),
__init__ = interp2app(W_FileIO.descr_init),
+
+ seek = interp2app(W_FileIO.seek_w),
+
readable = interp2app(W_FileIO.readable_w),
writable = interp2app(W_FileIO.writable_w),
+ seekable = interp2app(W_FileIO.seekable_w),
)
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Fri Oct 15 22:47:54 2010
@@ -87,6 +87,13 @@
def seekable_w(self, space):
return space.w_False
+ @unwrap_spec('self', ObjSpace)
+ def check_seekable_w(self, space):
+ if not space.is_true(space.call_method(self, 'seekable')):
+ raise OperationError(
+ space.w_IOError,
+ space.wrap("file or stream is not seekable"))
+
W_IOBase.typedef = TypeDef(
'_IOBase',
__new__ = generic_new_descr(W_IOBase),
@@ -100,6 +107,7 @@
readable = interp2app(W_IOBase.readable_w),
writable = interp2app(W_IOBase.writable_w),
seekable = interp2app(W_IOBase.seekable_w),
+ _checkSeekable = interp2app(W_IOBase.check_seekable_w),
closed = GetSetProperty(W_IOBase.closed_get_w),
)
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Fri Oct 15 22:47:54 2010
@@ -52,6 +52,12 @@
f = io.open(self.tmpfile, "rb")
f.close()
+ def test_open_writable(self):
+ import io
+ f = io.open(self.tmpfile, "w+b")
+ f.seek(0)
+ f.close()
+
def test_open_fd(self):
import io
os = self.posix
From afa at codespeak.net Fri Oct 15 23:22:42 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 15 Oct 2010 23:22:42 +0200 (CEST)
Subject: [pypy-svn] r78010 - pypy/branch/fast-forward/pypy/module/_io
Message-ID: <20101015212242.DA96136E393@codespeak.net>
Author: afa
Date: Fri Oct 15 23:22:41 2010
New Revision: 78010
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
Log:
Fix translation
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Fri Oct 15 23:22:41 2010
@@ -145,7 +145,7 @@
self.seekable = 0
else:
self.seekable = 1
- return space.newbool(self.seekable)
+ return space.newbool(self.seekable == 1)
W_FileIO.typedef = TypeDef(
'FileIO', W_RawIOBase.typedef,
From afa at codespeak.net Sat Oct 16 00:34:51 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Sat, 16 Oct 2010 00:34:51 +0200 (CEST)
Subject: [pypy-svn] r78011 - in pypy/branch/fast-forward/pypy:
module/__builtin__ module/__builtin__/test objspace/std
objspace/std/test
Message-ID: <20101015223451.AA78736E3B5@codespeak.net>
Author: afa
Date: Sat Oct 16 00:34:43 2010
New Revision: 78011
Modified:
pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py
pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py
pypy/branch/fast-forward/pypy/objspace/std/inttype.py
pypy/branch/fast-forward/pypy/objspace/std/longtype.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_intobject.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py
Log:
Make int() and long() fall back to __trunc__().
This is the same as CPython r60566, issue2002.
I hope I did it right.
Modified: pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py (original)
+++ pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py Sat Oct 16 00:34:43 2010
@@ -595,6 +595,27 @@
space.wrap("__hash__ must return int or long"))
return w_ret
+ def descr_int(self, space):
+ w_func = self.getattr(space, space.wrap('__int__'), False)
+ if w_func is not None:
+ return space.call_function(w_func)
+
+ w_truncated = space.trunc(self)
+ # int() needs to return an int
+ try:
+ return space.int(w_truncated)
+ except OperationError:
+ # Raise a different error
+ raise OperationError(
+ space.w_TypeError,
+ space.wrap("__trunc__ returned non-Integral"))
+
+ def descr_long(self, space):
+ w_func = self.getattr(space, space.wrap('__long__'), False)
+ if w_func is not None:
+ return space.call_function(w_func)
+ return self.descr_int(space)
+
def descr_index(self, space):
w_func = self.getattr(space, space.wrap('__index__'), False)
if w_func is not None:
@@ -685,7 +706,7 @@
rawdict = {}
# unary operations
-for op in "neg pos abs invert int long float oct hex enter reversed".split():
+for op in "neg pos abs invert trunc float oct hex enter reversed".split():
specialname = "__%s__" % (op, )
# fool the gateway logic by giving it a real unbound method
meth = new.instancemethod(
@@ -762,6 +783,10 @@
unwrap_spec=['self', ObjSpace, W_Root]),
__hash__ = interp2app(W_InstanceObject.descr_hash,
unwrap_spec=['self', ObjSpace]),
+ __int__ = interp2app(W_InstanceObject.descr_int,
+ unwrap_spec=['self', ObjSpace]),
+ __long__ = interp2app(W_InstanceObject.descr_long,
+ unwrap_spec=['self', ObjSpace]),
__index__ = interp2app(W_InstanceObject.descr_index,
unwrap_spec=['self', ObjSpace]),
__contains__ = interp2app(W_InstanceObject.descr_contains,
Modified: pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py (original)
+++ pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py Sat Oct 16 00:34:43 2010
@@ -898,10 +898,18 @@
"long": long}
for opname, opfunc in op_by_name.items():
assert opfunc(b) == 42
- assert b.called == ("__" + opname + "__", ())
+ called = b.called
+ assert called == ("__" + opname + "__", ())
assert oct(a) == '__oct__()'
assert hex(a) == '__hex__()'
#
+ class JustTrunc:
+ def __trunc__(self):
+ return 42
+ assert int(JustTrunc()) == 42
+ assert long(JustTrunc()) == 42
+ #
+ #
class C:
def __getattr__(self, name):
return lambda: 5.5
Modified: pypy/branch/fast-forward/pypy/objspace/std/inttype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/inttype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/inttype.py Sat Oct 16 00:34:43 2010
@@ -101,8 +101,13 @@
except ParseStringOverflowError, e:
w_longval = retry_to_w_long(space, e.parser)
else:
- # otherwise, use the __int__() method
- w_obj = space.int(w_value)
+ # otherwise, use the __int__() then __trunc__() methods
+ try:
+ w_obj = space.int(w_value)
+ except OperationError, e:
+ if not e.match(space,space.w_TypeError):
+ raise
+ w_obj = space.trunc(w_value)
# 'int(x)' should return whatever x.__int__() returned
if space.is_w(w_inttype, space.w_int):
return w_obj
Modified: pypy/branch/fast-forward/pypy/objspace/std/longtype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/longtype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/longtype.py Sat Oct 16 00:34:43 2010
@@ -36,8 +36,13 @@
raise OperationError(space.w_ValueError,
space.wrap(e.msg))
else:
- # otherwise, use the __long__() method
- w_obj = space.long(w_value)
+ # otherwise, use the __long__(), then the __trunc__ methods
+ try:
+ w_obj = space.long(w_value)
+ except OperationError, e:
+ if not e.match(space,space.w_TypeError):
+ raise
+ w_obj = space.trunc(w_value)
# 'long(x)' should return whatever x.__long__() returned
if space.is_w(w_longtype, space.w_long):
return w_obj
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_intobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_intobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_intobject.py Sat Oct 16 00:34:43 2010
@@ -418,6 +418,12 @@
pass
raises((AttributeError,TypeError), long, b())
+ def test_just_trunc(self):
+ class myint(object):
+ def __trunc__(self):
+ return 42
+ assert int(myint()) == 42
+
def test_override___int__(self):
class myint(int):
def __int__(self):
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py Sat Oct 16 00:34:43 2010
@@ -246,6 +246,12 @@
raises(OverflowError, operator.truediv, huge, 3)
raises(OverflowError, operator.truediv, huge, 3L)
+ def test_just_trunc(self):
+ class myint(object):
+ def __trunc__(self):
+ return 42
+ assert long(myint()) == 42
+
def test_override___long__(self):
class mylong(long):
def __long__(self):
From trundle at codespeak.net Sat Oct 16 00:43:24 2010
From: trundle at codespeak.net (trundle at codespeak.net)
Date: Sat, 16 Oct 2010 00:43:24 +0200 (CEST)
Subject: [pypy-svn] r78012 - pypy/branch/fast-forward/lib-python
Message-ID: <20101015224324.C130036E3BC@codespeak.net>
Author: trundle
Date: Sat Oct 16 00:43:19 2010
New Revision: 78012
Modified:
pypy/branch/fast-forward/lib-python/TODO
Log:
Add note about catching string exceptions to TODO
Modified: pypy/branch/fast-forward/lib-python/TODO
==============================================================================
--- pypy/branch/fast-forward/lib-python/TODO (original)
+++ pypy/branch/fast-forward/lib-python/TODO Sat Oct 16 00:43:19 2010
@@ -35,7 +35,9 @@
test_pickle()
- "exceptions must be old-style classes or derived from BaseException, not str"
- in the 'raise' statement and generator.throw()
+ in the 'raise' statement and generator.throw(), catching a string
+ should emit a DeprecationWarning ("catching of string exceptions is
+ deprecated")
- missing builtin: memoryview
From arigo at codespeak.net Sat Oct 16 16:38:29 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 16 Oct 2010 16:38:29 +0200 (CEST)
Subject: [pypy-svn] r78013 - in pypy/branch/leak-finder/pypy/rlib/rsre: .
test
Message-ID: <20101016143829.5D61B36E393@codespeak.net>
Author: arigo
Date: Sat Oct 16 16:38:26 2010
New Revision: 78013
Added:
pypy/branch/leak-finder/pypy/rlib/rsre/test/conftest.py
pypy/branch/leak-finder/pypy/rlib/rsre/test/test_zjit.py
Modified:
pypy/branch/leak-finder/pypy/rlib/rsre/rsre_core.py
Log:
In-progress. Start with some tests.
Modified: pypy/branch/leak-finder/pypy/rlib/rsre/rsre_core.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rlib/rsre/rsre_core.py (original)
+++ pypy/branch/leak-finder/pypy/rlib/rsre/rsre_core.py Sat Oct 16 16:38:26 2010
@@ -4,6 +4,7 @@
from pypy.rlib.rsre import rsre_char
from pypy.tool.sourcetools import func_with_new_name
from pypy.rlib.objectmodel import we_are_translated
+from pypy.rlib import jit
OPCODE_FAILURE = 0
@@ -56,15 +57,18 @@
_seen_specname[specname] = True
# Install a copy of the function under the name '_spec_funcname' in each
# concrete subclass
+ specialized_methods = []
for prefix, concreteclass in [('str', StrMatchContext),
('uni', UnicodeMatchContext)]:
newfunc = func_with_new_name(func, prefix + specname)
assert not hasattr(concreteclass, specname)
setattr(concreteclass, specname, newfunc)
+ specialized_methods.append(newfunc)
# Return a dispatcher function, specialized on the exact type of 'ctx'
def dispatch(ctx, *args):
return getattr(ctx, specname)(*args)
dispatch._annspecialcase_ = 'specialize:argtype(0)'
+ dispatch._specialized_methods_ = specialized_methods
return dispatch
# ____________________________________________________________
@@ -75,6 +79,7 @@
class AbstractMatchContext(object):
"""Abstract base class"""
+ _immutable_fields_ = ['pattern[*]', 'flags']
match_start = 0
match_end = 0
match_marks = None
@@ -238,6 +243,7 @@
self.start_ptr = ptr
self.start_marks = marks
+ @jit.unroll_safe
def find_first_result(self, ctx):
ppos = self.ppos
while ctx.pat(ppos):
@@ -250,6 +256,8 @@
find_next_result = find_first_result
class RepeatOneMatchResult(MatchResult):
+ jitdriver = jit.JitDriver(greens=['nextppos', 'pattern'],
+ reds=['ptr', 'self', 'ctx'])
def __init__(self, nextppos, minptr, ptr, marks):
self.nextppos = nextppos
@@ -259,8 +267,18 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
+ nextppos = self.nextppos
while ptr >= self.minptr:
- result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
+ #
+ pattern = ctx.pattern
+ self.jitdriver.can_enter_jit(self=self, ptr=ptr, ctx=ctx,
+ nextppos=nextppos, pattern=pattern)
+ self.jitdriver.jit_merge_point(self=self, ptr=ptr, ctx=ctx,
+ nextppos=nextppos, pattern=pattern)
+ if jit.we_are_jitted():
+ ctx.pattern = pattern
+ #
+ result = sre_match(ctx, nextppos, ptr, self.start_marks)
ptr -= 1
if result is not None:
self.subresult = result
@@ -270,6 +288,8 @@
class MinRepeatOneMatchResult(MatchResult):
+ jitdriver = jit.JitDriver(greens=['nextppos', 'ppos3', 'pattern'],
+ reds=['ptr', 'self', 'ctx'])
def __init__(self, nextppos, ppos3, maxptr, ptr, marks):
self.nextppos = nextppos
@@ -280,27 +300,39 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
+ nextppos = self.nextppos
+ ppos3 = self.ppos3
while ptr <= self.maxptr:
- result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
+ #
+ pattern = ctx.pattern
+ self.jitdriver.can_enter_jit(self=self, ptr=ptr, ctx=ctx,
+ nextppos=nextppos, pattern=pattern,
+ ppos3=ppos3)
+ self.jitdriver.jit_merge_point(self=self, ptr=ptr, ctx=ctx,
+ nextppos=nextppos, pattern=pattern,
+ ppos3=ppos3)
+ if jit.we_are_jitted():
+ ctx.pattern = pattern
+ #
+ result = sre_match(ctx, nextppos, ptr, self.start_marks)
if result is not None:
self.subresult = result
self.start_ptr = ptr
return self
- if not self.next_char_ok(ctx, ptr):
+ if not self.next_char_ok(ctx, ptr, ppos3):
break
ptr += 1
def find_next_result(self, ctx):
ptr = self.start_ptr
- if not self.next_char_ok(ctx, ptr):
+ if not self.next_char_ok(ctx, ptr, self.ppos3):
return
self.start_ptr = ptr + 1
return self.find_first_result(ctx)
- def next_char_ok(self, ctx, ptr):
+ def next_char_ok(self, ctx, ptr, ppos):
if ptr == ctx.end:
return False
- ppos = self.ppos3
op = ctx.pat(ppos)
for op1, (checkerfn, _) in unroll_char_checker:
if op1 == op:
@@ -429,6 +461,7 @@
# ____________________________________________________________
@specializectx
+ at jit.unroll_safe
def sre_match(ctx, ppos, ptr, marks):
"""Returns either None or a MatchResult object. Usually we only need
the first result, but there is the case of REPEAT...UNTIL where we
Added: pypy/branch/leak-finder/pypy/rlib/rsre/test/conftest.py
==============================================================================
--- (empty file)
+++ pypy/branch/leak-finder/pypy/rlib/rsre/test/conftest.py Sat Oct 16 16:38:26 2010
@@ -0,0 +1,5 @@
+# import the option --viewloops from the JIT
+
+def pytest_addoption(parser):
+ from pypy.jit.conftest import pytest_addoption
+ pytest_addoption(parser)
Added: pypy/branch/leak-finder/pypy/rlib/rsre/test/test_zjit.py
==============================================================================
--- (empty file)
+++ pypy/branch/leak-finder/pypy/rlib/rsre/test/test_zjit.py Sat Oct 16 16:38:26 2010
@@ -0,0 +1,69 @@
+from pypy.jit.metainterp.test import test_basic
+from pypy.rlib.nonconst import NonConstant
+from pypy.rlib.debug import make_sure_not_modified
+from pypy.rlib.rsre.test.test_match import get_code
+from pypy.rlib.rsre import rsre_core
+from pypy.rpython.lltypesystem import lltype
+from pypy.rpython.annlowlevel import llstr, hlstr
+
+def entrypoint1(r, string):
+ r = array2list(r)
+ string = hlstr(string)
+ make_sure_not_modified(r)
+ match = rsre_core.match(r, string)
+ if match is None:
+ return -1
+ else:
+ return match.match_end
+
+def list2array(lst):
+ a = lltype.malloc(lltype.GcArray(lltype.Signed), len(lst))
+ for i, x in enumerate(lst):
+ a[i] = x
+ return a
+
+def array2list(a):
+ return [a[i] for i in range(len(a))]
+
+
+def test_jit_unroll_safe():
+ # test that the decorators are applied in the right order
+ assert not hasattr(rsre_core.sre_match, '_jit_unroll_safe_')
+ for m in rsre_core.sre_match._specialized_methods_:
+ assert m._jit_unroll_safe_
+
+
+class TestJitRSre(test_basic.LLJitMixin):
+
+ def meta_interp_match(self, pattern, string):
+ r = get_code(pattern)
+ return self.meta_interp(entrypoint1, [list2array(r), llstr(string)],
+ listcomp=True, backendopt=True)
+
+ def test_simple_match_1(self):
+ res = self.meta_interp_match(r"ab*bbbbbbbc", "abbbbbbbbbcdef")
+ assert res == 11
+
+ def test_simple_match_2(self):
+ res = self.meta_interp_match(r".*abc", "xxabcyyyyyyyyyyyyy")
+ assert res == 5
+
+ def test_match_minrepeat_1(self):
+ res = self.meta_interp_match(r".*?abc", "xxxxxxxxxxxxxxabc")
+ assert res == 17
+
+ #def test_match_maxuntil_1(self):
+ # res = self.meta_interp_match(r"(ab)*c", "ababababababababc")
+ # assert res == 17
+
+ def test_branch_1(self):
+ res = self.meta_interp_match(r".*?(ab|x)c", "xxxxxxxxxxxxxxabc")
+ assert res == 17
+
+ def test_match_minrepeat_2(self):
+ s = ("xxxxxxxxxxabbbbbbbbbb" +
+ "xxxxxxxxxxabbbbbbbbbb" +
+ "xxxxxxxxxxabbbbbbbbbb" +
+ "xxxxxxxxxxabbbbbbbbbbc")
+ res = self.meta_interp_match(r".*?ab+?c", s)
+ assert res == len(s)
From arigo at codespeak.net Sat Oct 16 16:39:34 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 16 Oct 2010 16:39:34 +0200 (CEST)
Subject: [pypy-svn] r78014 - pypy/branch/leak-finder/pypy/jit/metainterp
Message-ID: <20101016143934.0E7EE36E393@codespeak.net>
Author: arigo
Date: Sat Oct 16 16:39:33 2010
New Revision: 78014
Modified:
pypy/branch/leak-finder/pypy/jit/metainterp/warmspot.py
Log:
Bug fix: backend optimizations may remove some operations and invalidate
the 'index' of an existing operation inside its block.
Modified: pypy/branch/leak-finder/pypy/jit/metainterp/warmspot.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/metainterp/warmspot.py (original)
+++ pypy/branch/leak-finder/pypy/jit/metainterp/warmspot.py Sat Oct 16 16:39:33 2010
@@ -211,9 +211,9 @@
"there are multiple jit_merge_points with the same jitdriver"
def split_graph_and_record_jitdriver(self, graph, block, pos):
- jd = JitDriverStaticData()
- jd._jit_merge_point_pos = (graph, block, pos)
op = block.operations[pos]
+ jd = JitDriverStaticData()
+ jd._jit_merge_point_pos = (graph, op)
args = op.args[2:]
s_binding = self.translator.annotator.binding
jd._portal_args_s = [s_binding(v) for v in args]
@@ -457,8 +457,7 @@
self.make_args_specification(jd)
def make_args_specification(self, jd):
- graph, block, index = jd._jit_merge_point_pos
- op = block.operations[index]
+ graph, op = jd._jit_merge_point_pos
greens_v, reds_v = support.decode_hp_hint_args(op)
ALLARGS = [v.concretetype for v in (greens_v + reds_v)]
jd._green_args_spec = [v.concretetype for v in greens_v]
@@ -709,8 +708,14 @@
# ____________________________________________________________
# Now mutate origportalgraph to end with a call to portal_runner_ptr
#
- _, origblock, origindex = jd._jit_merge_point_pos
- op = origblock.operations[origindex]
+ _, op = jd._jit_merge_point_pos
+ for origblock in origportalgraph.iterblocks():
+ if op in origblock.operations:
+ break
+ else:
+ assert False, "lost the operation %r in the graph %r" % (
+ op, origportalgraph)
+ origindex = origblock.operations.index(op)
assert op.opname == 'jit_marker'
assert op.args[0].value == 'jit_merge_point'
greens_v, reds_v = support.decode_hp_hint_args(op)
From arigo at codespeak.net Sat Oct 16 16:40:51 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 16 Oct 2010 16:40:51 +0200 (CEST)
Subject: [pypy-svn] r78015 - in pypy/branch/leak-finder: dotviewer
pypy/jit/metainterp
Message-ID: <20101016144051.E216936E393@codespeak.net>
Author: arigo
Date: Sat Oct 16 16:40:50 2010
New Revision: 78015
Modified:
pypy/branch/leak-finder/dotviewer/drawgraph.py
pypy/branch/leak-finder/pypy/jit/metainterp/graphpage.py
pypy/branch/leak-finder/pypy/jit/metainterp/resoperation.py
Log:
Improve the readability of the JIT flow graphs by changing the color of
the prefix to the operations (the name of where it comes from).
Modified: pypy/branch/leak-finder/dotviewer/drawgraph.py
==============================================================================
--- pypy/branch/leak-finder/dotviewer/drawgraph.py (original)
+++ pypy/branch/leak-finder/dotviewer/drawgraph.py Sat Oct 16 16:40:50 2010
@@ -423,20 +423,43 @@
else:
for line in lines:
raw_line = line.replace('\\l','').replace('\r','') or ' '
- img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
- w, h = img.get_size()
- if w>wmax: wmax = w
- if raw_line.strip():
- if line.endswith('\\l'):
- def cmd(img=img, y=hmax):
- img.draw(xleft, ytop+y)
- elif line.endswith('\r'):
- def cmd(img=img, y=hmax, w=w):
- img.draw(xright-w, ytop+y)
- else:
- def cmd(img=img, y=hmax, w=w):
- img.draw(xcenter-w//2, ytop+y)
+ if '\f' in raw_line: # grayed out parts of the line
+ imgs = []
+ graytext = True
+ h = 16
+ w_total = 0
+ for linepart in raw_line.split('\f'):
+ graytext = not graytext
+ if not linepart.strip():
+ continue
+ if graytext:
+ fgcolor = (128, 160, 160)
+ else:
+ fgcolor = (0, 0, 0)
+ img = TextSnippet(self, linepart, fgcolor, bgcolor)
+ imgs.append((w_total, img))
+ w, h = img.get_size()
+ w_total += w
+ if w_total > wmax: wmax = w_total
+ def cmd(imgs=imgs, y=hmax):
+ for x, img in imgs:
+ img.draw(xleft+x, ytop+y)
commands.append(cmd)
+ else:
+ img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
+ w, h = img.get_size()
+ if w>wmax: wmax = w
+ if raw_line.strip():
+ if line.endswith('\\l'):
+ def cmd(img=img, y=hmax):
+ img.draw(xleft, ytop+y)
+ elif line.endswith('\r'):
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xright-w, ytop+y)
+ else:
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xcenter-w//2, ytop+y)
+ commands.append(cmd)
hmax += h
#hmax += 8
Modified: pypy/branch/leak-finder/pypy/jit/metainterp/graphpage.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/metainterp/graphpage.py (original)
+++ pypy/branch/leak-finder/pypy/jit/metainterp/graphpage.py Sat Oct 16 16:40:50 2010
@@ -153,7 +153,7 @@
opindex = opstartindex
while True:
op = operations[opindex]
- lines.append(repr(op))
+ lines.append(op.repr(graytext=True))
if is_interesting_guard(op):
tgt = op.getdescr()._debug_suboperations[0]
tgt_g, tgt_i = self.all_operations[tgt]
Modified: pypy/branch/leak-finder/pypy/jit/metainterp/resoperation.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/metainterp/resoperation.py (original)
+++ pypy/branch/leak-finder/pypy/jit/metainterp/resoperation.py Sat Oct 16 16:40:50 2010
@@ -93,7 +93,7 @@
def __repr__(self):
return self.repr()
- def repr(self):
+ def repr(self, graytext=False):
# RPython-friendly version
if self.result is not None:
sres = '%s = ' % (self.result,)
@@ -101,6 +101,8 @@
sres = ''
if self.name:
prefix = "%s:%s " % (self.name, self.pc)
+ if graytext:
+ prefix = "\f%s\f" % prefix
else:
prefix = ""
args = self.getarglist()
From arigo at codespeak.net Sat Oct 16 16:44:49 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 16 Oct 2010 16:44:49 +0200 (CEST)
Subject: [pypy-svn] r78016 - pypy/branch/leak-finder/pypy/jit/codewriter/test
Message-ID: <20101016144449.A53AF36E393@codespeak.net>
Author: arigo
Date: Sat Oct 16 16:44:48 2010
New Revision: 78016
Modified:
pypy/branch/leak-finder/pypy/jit/codewriter/test/test_jtransform.py
Log:
An extra test.
Modified: pypy/branch/leak-finder/pypy/jit/codewriter/test/test_jtransform.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/codewriter/test/test_jtransform.py (original)
+++ pypy/branch/leak-finder/pypy/jit/codewriter/test/test_jtransform.py Sat Oct 16 16:44:48 2010
@@ -687,6 +687,38 @@
assert block.operations[1].result is None
assert block.exits[0].args == [v1]
+def test_jit_merge_point_1():
+ class FakeJitDriverSD:
+ index = 42
+ class jitdriver:
+ greens = ['green1', 'green2']
+ reds = ['red1', 'red2']
+ jd = FakeJitDriverSD()
+ v1 = varoftype(lltype.Signed)
+ v2 = varoftype(lltype.Signed)
+ v3 = varoftype(lltype.Signed)
+ v4 = varoftype(lltype.Signed)
+ v5 = varoftype(lltype.Void)
+ op = SpaceOperation('jit_marker',
+ [Constant('jit_merge_point', lltype.Void),
+ Constant(jd.jitdriver, lltype.Void),
+ v1, v2, v3, v4], v5)
+ tr = Transformer()
+ tr.portal_jd = jd
+ oplist = tr.rewrite_operation(op)
+ assert len(oplist) == 6
+ assert oplist[0].opname == '-live-'
+ assert oplist[1].opname == 'int_guard_value'
+ assert oplist[1].args == [v1]
+ assert oplist[2].opname == '-live-'
+ assert oplist[3].opname == 'int_guard_value'
+ assert oplist[3].args == [v2]
+ assert oplist[4].opname == 'jit_merge_point'
+ assert oplist[4].args[0].value == 42
+ assert list(oplist[4].args[1]) == [v1, v2]
+ assert list(oplist[4].args[4]) == [v3, v4]
+ assert oplist[5].opname == '-live-'
+
def test_int_abs():
v1 = varoftype(lltype.Signed)
v2 = varoftype(lltype.Signed)
From arigo at codespeak.net Sat Oct 16 17:01:32 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 16 Oct 2010 17:01:32 +0200 (CEST)
Subject: [pypy-svn] r78017 - pypy/branch/leak-finder/pypy/jit/codewriter/test
Message-ID: <20101016150132.2D65836E393@codespeak.net>
Author: arigo
Date: Sat Oct 16 17:01:30 2010
New Revision: 78017
Modified:
pypy/branch/leak-finder/pypy/jit/codewriter/test/test_jtransform.py
Log:
Add voids.
Modified: pypy/branch/leak-finder/pypy/jit/codewriter/test/test_jtransform.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/codewriter/test/test_jtransform.py (original)
+++ pypy/branch/leak-finder/pypy/jit/codewriter/test/test_jtransform.py Sat Oct 16 17:01:30 2010
@@ -691,18 +691,20 @@
class FakeJitDriverSD:
index = 42
class jitdriver:
- greens = ['green1', 'green2']
- reds = ['red1', 'red2']
+ greens = ['green1', 'green2', 'voidgreen3']
+ reds = ['red1', 'red2', 'voidred3']
jd = FakeJitDriverSD()
v1 = varoftype(lltype.Signed)
v2 = varoftype(lltype.Signed)
+ vvoid1 = varoftype(lltype.Void)
v3 = varoftype(lltype.Signed)
v4 = varoftype(lltype.Signed)
+ vvoid2 = varoftype(lltype.Void)
v5 = varoftype(lltype.Void)
op = SpaceOperation('jit_marker',
[Constant('jit_merge_point', lltype.Void),
Constant(jd.jitdriver, lltype.Void),
- v1, v2, v3, v4], v5)
+ v1, v2, vvoid1, v3, v4, vvoid2], v5)
tr = Transformer()
tr.portal_jd = jd
oplist = tr.rewrite_operation(op)
From arigo at codespeak.net Sat Oct 16 17:46:02 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 16 Oct 2010 17:46:02 +0200 (CEST)
Subject: [pypy-svn] r78018 - pypy/branch/rsre-jit
Message-ID: <20101016154602.86E2E36E393@codespeak.net>
Author: arigo
Date: Sat Oct 16 17:46:00 2010
New Revision: 78018
Added:
pypy/branch/rsre-jit/
- copied from r78017, pypy/trunk/
Log:
A branch in which to play again with JITting the _sre module.
From arigo at codespeak.net Sat Oct 16 17:47:50 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 16 Oct 2010 17:47:50 +0200 (CEST)
Subject: [pypy-svn] r78019 - in pypy/branch/leak-finder: dotviewer
pypy/jit/codewriter/test pypy/jit/metainterp pypy/rlib/rsre
pypy/rlib/rsre/test
Message-ID: <20101016154750.789C736E393@codespeak.net>
Author: arigo
Date: Sat Oct 16 17:47:48 2010
New Revision: 78019
Removed:
pypy/branch/leak-finder/pypy/rlib/rsre/test/conftest.py
pypy/branch/leak-finder/pypy/rlib/rsre/test/test_zjit.py
Modified:
pypy/branch/leak-finder/dotviewer/drawgraph.py
pypy/branch/leak-finder/pypy/jit/codewriter/test/test_jtransform.py
pypy/branch/leak-finder/pypy/jit/metainterp/graphpage.py
pypy/branch/leak-finder/pypy/jit/metainterp/resoperation.py
pypy/branch/leak-finder/pypy/jit/metainterp/warmspot.py
pypy/branch/leak-finder/pypy/rlib/rsre/rsre_core.py
Log:
Oups! Wrong branch. Thanks agaynor.
Modified: pypy/branch/leak-finder/dotviewer/drawgraph.py
==============================================================================
--- pypy/branch/leak-finder/dotviewer/drawgraph.py (original)
+++ pypy/branch/leak-finder/dotviewer/drawgraph.py Sat Oct 16 17:47:48 2010
@@ -423,43 +423,20 @@
else:
for line in lines:
raw_line = line.replace('\\l','').replace('\r','') or ' '
- if '\f' in raw_line: # grayed out parts of the line
- imgs = []
- graytext = True
- h = 16
- w_total = 0
- for linepart in raw_line.split('\f'):
- graytext = not graytext
- if not linepart.strip():
- continue
- if graytext:
- fgcolor = (128, 160, 160)
- else:
- fgcolor = (0, 0, 0)
- img = TextSnippet(self, linepart, fgcolor, bgcolor)
- imgs.append((w_total, img))
- w, h = img.get_size()
- w_total += w
- if w_total > wmax: wmax = w_total
- def cmd(imgs=imgs, y=hmax):
- for x, img in imgs:
- img.draw(xleft+x, ytop+y)
+ img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
+ w, h = img.get_size()
+ if w>wmax: wmax = w
+ if raw_line.strip():
+ if line.endswith('\\l'):
+ def cmd(img=img, y=hmax):
+ img.draw(xleft, ytop+y)
+ elif line.endswith('\r'):
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xright-w, ytop+y)
+ else:
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xcenter-w//2, ytop+y)
commands.append(cmd)
- else:
- img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
- w, h = img.get_size()
- if w>wmax: wmax = w
- if raw_line.strip():
- if line.endswith('\\l'):
- def cmd(img=img, y=hmax):
- img.draw(xleft, ytop+y)
- elif line.endswith('\r'):
- def cmd(img=img, y=hmax, w=w):
- img.draw(xright-w, ytop+y)
- else:
- def cmd(img=img, y=hmax, w=w):
- img.draw(xcenter-w//2, ytop+y)
- commands.append(cmd)
hmax += h
#hmax += 8
Modified: pypy/branch/leak-finder/pypy/jit/codewriter/test/test_jtransform.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/codewriter/test/test_jtransform.py (original)
+++ pypy/branch/leak-finder/pypy/jit/codewriter/test/test_jtransform.py Sat Oct 16 17:47:48 2010
@@ -687,40 +687,6 @@
assert block.operations[1].result is None
assert block.exits[0].args == [v1]
-def test_jit_merge_point_1():
- class FakeJitDriverSD:
- index = 42
- class jitdriver:
- greens = ['green1', 'green2', 'voidgreen3']
- reds = ['red1', 'red2', 'voidred3']
- jd = FakeJitDriverSD()
- v1 = varoftype(lltype.Signed)
- v2 = varoftype(lltype.Signed)
- vvoid1 = varoftype(lltype.Void)
- v3 = varoftype(lltype.Signed)
- v4 = varoftype(lltype.Signed)
- vvoid2 = varoftype(lltype.Void)
- v5 = varoftype(lltype.Void)
- op = SpaceOperation('jit_marker',
- [Constant('jit_merge_point', lltype.Void),
- Constant(jd.jitdriver, lltype.Void),
- v1, v2, vvoid1, v3, v4, vvoid2], v5)
- tr = Transformer()
- tr.portal_jd = jd
- oplist = tr.rewrite_operation(op)
- assert len(oplist) == 6
- assert oplist[0].opname == '-live-'
- assert oplist[1].opname == 'int_guard_value'
- assert oplist[1].args == [v1]
- assert oplist[2].opname == '-live-'
- assert oplist[3].opname == 'int_guard_value'
- assert oplist[3].args == [v2]
- assert oplist[4].opname == 'jit_merge_point'
- assert oplist[4].args[0].value == 42
- assert list(oplist[4].args[1]) == [v1, v2]
- assert list(oplist[4].args[4]) == [v3, v4]
- assert oplist[5].opname == '-live-'
-
def test_int_abs():
v1 = varoftype(lltype.Signed)
v2 = varoftype(lltype.Signed)
Modified: pypy/branch/leak-finder/pypy/jit/metainterp/graphpage.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/metainterp/graphpage.py (original)
+++ pypy/branch/leak-finder/pypy/jit/metainterp/graphpage.py Sat Oct 16 17:47:48 2010
@@ -153,7 +153,7 @@
opindex = opstartindex
while True:
op = operations[opindex]
- lines.append(op.repr(graytext=True))
+ lines.append(repr(op))
if is_interesting_guard(op):
tgt = op.getdescr()._debug_suboperations[0]
tgt_g, tgt_i = self.all_operations[tgt]
Modified: pypy/branch/leak-finder/pypy/jit/metainterp/resoperation.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/metainterp/resoperation.py (original)
+++ pypy/branch/leak-finder/pypy/jit/metainterp/resoperation.py Sat Oct 16 17:47:48 2010
@@ -93,7 +93,7 @@
def __repr__(self):
return self.repr()
- def repr(self, graytext=False):
+ def repr(self):
# RPython-friendly version
if self.result is not None:
sres = '%s = ' % (self.result,)
@@ -101,8 +101,6 @@
sres = ''
if self.name:
prefix = "%s:%s " % (self.name, self.pc)
- if graytext:
- prefix = "\f%s\f" % prefix
else:
prefix = ""
args = self.getarglist()
Modified: pypy/branch/leak-finder/pypy/jit/metainterp/warmspot.py
==============================================================================
--- pypy/branch/leak-finder/pypy/jit/metainterp/warmspot.py (original)
+++ pypy/branch/leak-finder/pypy/jit/metainterp/warmspot.py Sat Oct 16 17:47:48 2010
@@ -211,9 +211,9 @@
"there are multiple jit_merge_points with the same jitdriver"
def split_graph_and_record_jitdriver(self, graph, block, pos):
- op = block.operations[pos]
jd = JitDriverStaticData()
- jd._jit_merge_point_pos = (graph, op)
+ jd._jit_merge_point_pos = (graph, block, pos)
+ op = block.operations[pos]
args = op.args[2:]
s_binding = self.translator.annotator.binding
jd._portal_args_s = [s_binding(v) for v in args]
@@ -457,7 +457,8 @@
self.make_args_specification(jd)
def make_args_specification(self, jd):
- graph, op = jd._jit_merge_point_pos
+ graph, block, index = jd._jit_merge_point_pos
+ op = block.operations[index]
greens_v, reds_v = support.decode_hp_hint_args(op)
ALLARGS = [v.concretetype for v in (greens_v + reds_v)]
jd._green_args_spec = [v.concretetype for v in greens_v]
@@ -708,14 +709,8 @@
# ____________________________________________________________
# Now mutate origportalgraph to end with a call to portal_runner_ptr
#
- _, op = jd._jit_merge_point_pos
- for origblock in origportalgraph.iterblocks():
- if op in origblock.operations:
- break
- else:
- assert False, "lost the operation %r in the graph %r" % (
- op, origportalgraph)
- origindex = origblock.operations.index(op)
+ _, origblock, origindex = jd._jit_merge_point_pos
+ op = origblock.operations[origindex]
assert op.opname == 'jit_marker'
assert op.args[0].value == 'jit_merge_point'
greens_v, reds_v = support.decode_hp_hint_args(op)
Modified: pypy/branch/leak-finder/pypy/rlib/rsre/rsre_core.py
==============================================================================
--- pypy/branch/leak-finder/pypy/rlib/rsre/rsre_core.py (original)
+++ pypy/branch/leak-finder/pypy/rlib/rsre/rsre_core.py Sat Oct 16 17:47:48 2010
@@ -4,7 +4,6 @@
from pypy.rlib.rsre import rsre_char
from pypy.tool.sourcetools import func_with_new_name
from pypy.rlib.objectmodel import we_are_translated
-from pypy.rlib import jit
OPCODE_FAILURE = 0
@@ -57,18 +56,15 @@
_seen_specname[specname] = True
# Install a copy of the function under the name '_spec_funcname' in each
# concrete subclass
- specialized_methods = []
for prefix, concreteclass in [('str', StrMatchContext),
('uni', UnicodeMatchContext)]:
newfunc = func_with_new_name(func, prefix + specname)
assert not hasattr(concreteclass, specname)
setattr(concreteclass, specname, newfunc)
- specialized_methods.append(newfunc)
# Return a dispatcher function, specialized on the exact type of 'ctx'
def dispatch(ctx, *args):
return getattr(ctx, specname)(*args)
dispatch._annspecialcase_ = 'specialize:argtype(0)'
- dispatch._specialized_methods_ = specialized_methods
return dispatch
# ____________________________________________________________
@@ -79,7 +75,6 @@
class AbstractMatchContext(object):
"""Abstract base class"""
- _immutable_fields_ = ['pattern[*]', 'flags']
match_start = 0
match_end = 0
match_marks = None
@@ -243,7 +238,6 @@
self.start_ptr = ptr
self.start_marks = marks
- @jit.unroll_safe
def find_first_result(self, ctx):
ppos = self.ppos
while ctx.pat(ppos):
@@ -256,8 +250,6 @@
find_next_result = find_first_result
class RepeatOneMatchResult(MatchResult):
- jitdriver = jit.JitDriver(greens=['nextppos', 'pattern'],
- reds=['ptr', 'self', 'ctx'])
def __init__(self, nextppos, minptr, ptr, marks):
self.nextppos = nextppos
@@ -267,18 +259,8 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
- nextppos = self.nextppos
while ptr >= self.minptr:
- #
- pattern = ctx.pattern
- self.jitdriver.can_enter_jit(self=self, ptr=ptr, ctx=ctx,
- nextppos=nextppos, pattern=pattern)
- self.jitdriver.jit_merge_point(self=self, ptr=ptr, ctx=ctx,
- nextppos=nextppos, pattern=pattern)
- if jit.we_are_jitted():
- ctx.pattern = pattern
- #
- result = sre_match(ctx, nextppos, ptr, self.start_marks)
+ result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
ptr -= 1
if result is not None:
self.subresult = result
@@ -288,8 +270,6 @@
class MinRepeatOneMatchResult(MatchResult):
- jitdriver = jit.JitDriver(greens=['nextppos', 'ppos3', 'pattern'],
- reds=['ptr', 'self', 'ctx'])
def __init__(self, nextppos, ppos3, maxptr, ptr, marks):
self.nextppos = nextppos
@@ -300,39 +280,27 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
- nextppos = self.nextppos
- ppos3 = self.ppos3
while ptr <= self.maxptr:
- #
- pattern = ctx.pattern
- self.jitdriver.can_enter_jit(self=self, ptr=ptr, ctx=ctx,
- nextppos=nextppos, pattern=pattern,
- ppos3=ppos3)
- self.jitdriver.jit_merge_point(self=self, ptr=ptr, ctx=ctx,
- nextppos=nextppos, pattern=pattern,
- ppos3=ppos3)
- if jit.we_are_jitted():
- ctx.pattern = pattern
- #
- result = sre_match(ctx, nextppos, ptr, self.start_marks)
+ result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
if result is not None:
self.subresult = result
self.start_ptr = ptr
return self
- if not self.next_char_ok(ctx, ptr, ppos3):
+ if not self.next_char_ok(ctx, ptr):
break
ptr += 1
def find_next_result(self, ctx):
ptr = self.start_ptr
- if not self.next_char_ok(ctx, ptr, self.ppos3):
+ if not self.next_char_ok(ctx, ptr):
return
self.start_ptr = ptr + 1
return self.find_first_result(ctx)
- def next_char_ok(self, ctx, ptr, ppos):
+ def next_char_ok(self, ctx, ptr):
if ptr == ctx.end:
return False
+ ppos = self.ppos3
op = ctx.pat(ppos)
for op1, (checkerfn, _) in unroll_char_checker:
if op1 == op:
@@ -461,7 +429,6 @@
# ____________________________________________________________
@specializectx
- at jit.unroll_safe
def sre_match(ctx, ppos, ptr, marks):
"""Returns either None or a MatchResult object. Usually we only need
the first result, but there is the case of REPEAT...UNTIL where we
From arigo at codespeak.net Sat Oct 16 17:49:59 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 16 Oct 2010 17:49:59 +0200 (CEST)
Subject: [pypy-svn] r78020 - in pypy/branch/rsre-jit/pypy/rlib/rsre: . test
Message-ID: <20101016154959.C595136E393@codespeak.net>
Author: arigo
Date: Sat Oct 16 17:49:58 2010
New Revision: 78020
Added:
pypy/branch/rsre-jit/pypy/rlib/rsre/test/conftest.py
- copied unchanged from r78013, pypy/branch/leak-finder/pypy/rlib/rsre/test/conftest.py
pypy/branch/rsre-jit/pypy/rlib/rsre/test/test_zjit.py
- copied unchanged from r78013, pypy/branch/leak-finder/pypy/rlib/rsre/test/test_zjit.py
Modified:
pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_core.py
Log:
In-progress. Start with some tests.
Modified: pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_core.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_core.py (original)
+++ pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_core.py Sat Oct 16 17:49:58 2010
@@ -4,6 +4,7 @@
from pypy.rlib.rsre import rsre_char
from pypy.tool.sourcetools import func_with_new_name
from pypy.rlib.objectmodel import we_are_translated
+from pypy.rlib import jit
OPCODE_FAILURE = 0
@@ -56,15 +57,18 @@
_seen_specname[specname] = True
# Install a copy of the function under the name '_spec_funcname' in each
# concrete subclass
+ specialized_methods = []
for prefix, concreteclass in [('str', StrMatchContext),
('uni', UnicodeMatchContext)]:
newfunc = func_with_new_name(func, prefix + specname)
assert not hasattr(concreteclass, specname)
setattr(concreteclass, specname, newfunc)
+ specialized_methods.append(newfunc)
# Return a dispatcher function, specialized on the exact type of 'ctx'
def dispatch(ctx, *args):
return getattr(ctx, specname)(*args)
dispatch._annspecialcase_ = 'specialize:argtype(0)'
+ dispatch._specialized_methods_ = specialized_methods
return dispatch
# ____________________________________________________________
@@ -75,6 +79,7 @@
class AbstractMatchContext(object):
"""Abstract base class"""
+ _immutable_fields_ = ['pattern[*]', 'flags']
match_start = 0
match_end = 0
match_marks = None
@@ -238,6 +243,7 @@
self.start_ptr = ptr
self.start_marks = marks
+ @jit.unroll_safe
def find_first_result(self, ctx):
ppos = self.ppos
while ctx.pat(ppos):
@@ -250,6 +256,8 @@
find_next_result = find_first_result
class RepeatOneMatchResult(MatchResult):
+ jitdriver = jit.JitDriver(greens=['nextppos', 'pattern'],
+ reds=['ptr', 'self', 'ctx'])
def __init__(self, nextppos, minptr, ptr, marks):
self.nextppos = nextppos
@@ -259,8 +267,18 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
+ nextppos = self.nextppos
while ptr >= self.minptr:
- result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
+ #
+ pattern = ctx.pattern
+ self.jitdriver.can_enter_jit(self=self, ptr=ptr, ctx=ctx,
+ nextppos=nextppos, pattern=pattern)
+ self.jitdriver.jit_merge_point(self=self, ptr=ptr, ctx=ctx,
+ nextppos=nextppos, pattern=pattern)
+ if jit.we_are_jitted():
+ ctx.pattern = pattern
+ #
+ result = sre_match(ctx, nextppos, ptr, self.start_marks)
ptr -= 1
if result is not None:
self.subresult = result
@@ -270,6 +288,8 @@
class MinRepeatOneMatchResult(MatchResult):
+ jitdriver = jit.JitDriver(greens=['nextppos', 'ppos3', 'pattern'],
+ reds=['ptr', 'self', 'ctx'])
def __init__(self, nextppos, ppos3, maxptr, ptr, marks):
self.nextppos = nextppos
@@ -280,27 +300,39 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
+ nextppos = self.nextppos
+ ppos3 = self.ppos3
while ptr <= self.maxptr:
- result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
+ #
+ pattern = ctx.pattern
+ self.jitdriver.can_enter_jit(self=self, ptr=ptr, ctx=ctx,
+ nextppos=nextppos, pattern=pattern,
+ ppos3=ppos3)
+ self.jitdriver.jit_merge_point(self=self, ptr=ptr, ctx=ctx,
+ nextppos=nextppos, pattern=pattern,
+ ppos3=ppos3)
+ if jit.we_are_jitted():
+ ctx.pattern = pattern
+ #
+ result = sre_match(ctx, nextppos, ptr, self.start_marks)
if result is not None:
self.subresult = result
self.start_ptr = ptr
return self
- if not self.next_char_ok(ctx, ptr):
+ if not self.next_char_ok(ctx, ptr, ppos3):
break
ptr += 1
def find_next_result(self, ctx):
ptr = self.start_ptr
- if not self.next_char_ok(ctx, ptr):
+ if not self.next_char_ok(ctx, ptr, self.ppos3):
return
self.start_ptr = ptr + 1
return self.find_first_result(ctx)
- def next_char_ok(self, ctx, ptr):
+ def next_char_ok(self, ctx, ptr, ppos):
if ptr == ctx.end:
return False
- ppos = self.ppos3
op = ctx.pat(ppos)
for op1, (checkerfn, _) in unroll_char_checker:
if op1 == op:
@@ -429,6 +461,7 @@
# ____________________________________________________________
@specializectx
+ at jit.unroll_safe
def sre_match(ctx, ppos, ptr, marks):
"""Returns either None or a MatchResult object. Usually we only need
the first result, but there is the case of REPEAT...UNTIL where we
From arigo at codespeak.net Sat Oct 16 17:50:30 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 16 Oct 2010 17:50:30 +0200 (CEST)
Subject: [pypy-svn] r78021 - pypy/branch/rsre-jit/pypy/jit/metainterp
Message-ID: <20101016155030.14B6A36E393@codespeak.net>
Author: arigo
Date: Sat Oct 16 17:50:28 2010
New Revision: 78021
Modified:
pypy/branch/rsre-jit/pypy/jit/metainterp/warmspot.py
Log:
Bug fix: backend optimizations may remove some operations and invalidate
the 'index' of an existing operation inside its block.
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/warmspot.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/warmspot.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/warmspot.py Sat Oct 16 17:50:28 2010
@@ -211,9 +211,9 @@
"there are multiple jit_merge_points with the same jitdriver"
def split_graph_and_record_jitdriver(self, graph, block, pos):
- jd = JitDriverStaticData()
- jd._jit_merge_point_pos = (graph, block, pos)
op = block.operations[pos]
+ jd = JitDriverStaticData()
+ jd._jit_merge_point_pos = (graph, op)
args = op.args[2:]
s_binding = self.translator.annotator.binding
jd._portal_args_s = [s_binding(v) for v in args]
@@ -457,8 +457,7 @@
self.make_args_specification(jd)
def make_args_specification(self, jd):
- graph, block, index = jd._jit_merge_point_pos
- op = block.operations[index]
+ graph, op = jd._jit_merge_point_pos
greens_v, reds_v = support.decode_hp_hint_args(op)
ALLARGS = [v.concretetype for v in (greens_v + reds_v)]
jd._green_args_spec = [v.concretetype for v in greens_v]
@@ -709,8 +708,14 @@
# ____________________________________________________________
# Now mutate origportalgraph to end with a call to portal_runner_ptr
#
- _, origblock, origindex = jd._jit_merge_point_pos
- op = origblock.operations[origindex]
+ _, op = jd._jit_merge_point_pos
+ for origblock in origportalgraph.iterblocks():
+ if op in origblock.operations:
+ break
+ else:
+ assert False, "lost the operation %r in the graph %r" % (
+ op, origportalgraph)
+ origindex = origblock.operations.index(op)
assert op.opname == 'jit_marker'
assert op.args[0].value == 'jit_merge_point'
greens_v, reds_v = support.decode_hp_hint_args(op)
From arigo at codespeak.net Sat Oct 16 17:50:59 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 16 Oct 2010 17:50:59 +0200 (CEST)
Subject: [pypy-svn] r78022 - in pypy/branch/rsre-jit: dotviewer
pypy/jit/metainterp
Message-ID: <20101016155059.CF96B36E393@codespeak.net>
Author: arigo
Date: Sat Oct 16 17:50:58 2010
New Revision: 78022
Modified:
pypy/branch/rsre-jit/dotviewer/drawgraph.py
pypy/branch/rsre-jit/pypy/jit/metainterp/graphpage.py
pypy/branch/rsre-jit/pypy/jit/metainterp/resoperation.py
Log:
Improve the readability of the JIT flow graphs by changing the color of
the prefix to the operations (the name of where it comes from).
Modified: pypy/branch/rsre-jit/dotviewer/drawgraph.py
==============================================================================
--- pypy/branch/rsre-jit/dotviewer/drawgraph.py (original)
+++ pypy/branch/rsre-jit/dotviewer/drawgraph.py Sat Oct 16 17:50:58 2010
@@ -423,20 +423,43 @@
else:
for line in lines:
raw_line = line.replace('\\l','').replace('\r','') or ' '
- img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
- w, h = img.get_size()
- if w>wmax: wmax = w
- if raw_line.strip():
- if line.endswith('\\l'):
- def cmd(img=img, y=hmax):
- img.draw(xleft, ytop+y)
- elif line.endswith('\r'):
- def cmd(img=img, y=hmax, w=w):
- img.draw(xright-w, ytop+y)
- else:
- def cmd(img=img, y=hmax, w=w):
- img.draw(xcenter-w//2, ytop+y)
+ if '\f' in raw_line: # grayed out parts of the line
+ imgs = []
+ graytext = True
+ h = 16
+ w_total = 0
+ for linepart in raw_line.split('\f'):
+ graytext = not graytext
+ if not linepart.strip():
+ continue
+ if graytext:
+ fgcolor = (128, 160, 160)
+ else:
+ fgcolor = (0, 0, 0)
+ img = TextSnippet(self, linepart, fgcolor, bgcolor)
+ imgs.append((w_total, img))
+ w, h = img.get_size()
+ w_total += w
+ if w_total > wmax: wmax = w_total
+ def cmd(imgs=imgs, y=hmax):
+ for x, img in imgs:
+ img.draw(xleft+x, ytop+y)
commands.append(cmd)
+ else:
+ img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
+ w, h = img.get_size()
+ if w>wmax: wmax = w
+ if raw_line.strip():
+ if line.endswith('\\l'):
+ def cmd(img=img, y=hmax):
+ img.draw(xleft, ytop+y)
+ elif line.endswith('\r'):
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xright-w, ytop+y)
+ else:
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xcenter-w//2, ytop+y)
+ commands.append(cmd)
hmax += h
#hmax += 8
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/graphpage.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/graphpage.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/graphpage.py Sat Oct 16 17:50:58 2010
@@ -153,7 +153,7 @@
opindex = opstartindex
while True:
op = operations[opindex]
- lines.append(repr(op))
+ lines.append(op.repr(graytext=True))
if is_interesting_guard(op):
tgt = op.getdescr()._debug_suboperations[0]
tgt_g, tgt_i = self.all_operations[tgt]
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/resoperation.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/resoperation.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/resoperation.py Sat Oct 16 17:50:58 2010
@@ -93,7 +93,7 @@
def __repr__(self):
return self.repr()
- def repr(self):
+ def repr(self, graytext=False):
# RPython-friendly version
if self.result is not None:
sres = '%s = ' % (self.result,)
@@ -101,6 +101,8 @@
sres = ''
if self.name:
prefix = "%s:%s " % (self.name, self.pc)
+ if graytext:
+ prefix = "\f%s\f" % prefix
else:
prefix = ""
args = self.getarglist()
From arigo at codespeak.net Sat Oct 16 17:51:23 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 16 Oct 2010 17:51:23 +0200 (CEST)
Subject: [pypy-svn] r78023 - pypy/branch/rsre-jit/pypy/jit/codewriter/test
Message-ID: <20101016155123.0C37A36E393@codespeak.net>
Author: arigo
Date: Sat Oct 16 17:51:21 2010
New Revision: 78023
Modified:
pypy/branch/rsre-jit/pypy/jit/codewriter/test/test_jtransform.py
Log:
Add an extra test, including voids.
Modified: pypy/branch/rsre-jit/pypy/jit/codewriter/test/test_jtransform.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/codewriter/test/test_jtransform.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/codewriter/test/test_jtransform.py Sat Oct 16 17:51:21 2010
@@ -687,6 +687,40 @@
assert block.operations[1].result is None
assert block.exits[0].args == [v1]
+def test_jit_merge_point_1():
+ class FakeJitDriverSD:
+ index = 42
+ class jitdriver:
+ greens = ['green1', 'green2', 'voidgreen3']
+ reds = ['red1', 'red2', 'voidred3']
+ jd = FakeJitDriverSD()
+ v1 = varoftype(lltype.Signed)
+ v2 = varoftype(lltype.Signed)
+ vvoid1 = varoftype(lltype.Void)
+ v3 = varoftype(lltype.Signed)
+ v4 = varoftype(lltype.Signed)
+ vvoid2 = varoftype(lltype.Void)
+ v5 = varoftype(lltype.Void)
+ op = SpaceOperation('jit_marker',
+ [Constant('jit_merge_point', lltype.Void),
+ Constant(jd.jitdriver, lltype.Void),
+ v1, v2, vvoid1, v3, v4, vvoid2], v5)
+ tr = Transformer()
+ tr.portal_jd = jd
+ oplist = tr.rewrite_operation(op)
+ assert len(oplist) == 6
+ assert oplist[0].opname == '-live-'
+ assert oplist[1].opname == 'int_guard_value'
+ assert oplist[1].args == [v1]
+ assert oplist[2].opname == '-live-'
+ assert oplist[3].opname == 'int_guard_value'
+ assert oplist[3].args == [v2]
+ assert oplist[4].opname == 'jit_merge_point'
+ assert oplist[4].args[0].value == 42
+ assert list(oplist[4].args[1]) == [v1, v2]
+ assert list(oplist[4].args[4]) == [v3, v4]
+ assert oplist[5].opname == '-live-'
+
def test_int_abs():
v1 = varoftype(lltype.Signed)
v2 = varoftype(lltype.Signed)
From arigo at codespeak.net Sat Oct 16 18:17:01 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 16 Oct 2010 18:17:01 +0200 (CEST)
Subject: [pypy-svn] r78024 - in pypy/branch/rsre-jit/pypy/rlib: . test
Message-ID: <20101016161701.4DDC036E393@codespeak.net>
Author: arigo
Date: Sat Oct 16 18:16:59 2010
New Revision: 78024
Modified:
pypy/branch/rsre-jit/pypy/rlib/jit.py
pypy/branch/rsre-jit/pypy/rlib/test/test_jit.py
Log:
Test showing the final goal: letting 'x.foo' be written
in the JitDriver, in the 'greens' list. Currently expands
as a extra argument in the jit_marker operation.
Modified: pypy/branch/rsre-jit/pypy/rlib/jit.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/rlib/jit.py (original)
+++ pypy/branch/rsre-jit/pypy/rlib/jit.py Sat Oct 16 18:16:59 2010
@@ -358,7 +358,8 @@
driver = self.instance.im_self
keys = kwds_s.keys()
keys.sort()
- expected = ['s_' + name for name in driver.greens + driver.reds]
+ expected = ['s_' + name for name in driver.greens + driver.reds
+ if '.' not in name]
expected.sort()
if keys != expected:
raise JitHintError("%s expects the following keyword "
@@ -403,7 +404,13 @@
uniquekey = 'jitdriver.%s' % func.func_name
args_s = args_s[:]
for name in variables:
- s_arg = kwds_s['s_' + name]
+ if '.' not in name:
+ s_arg = kwds_s['s_' + name]
+ else:
+ objname, fieldname = name.split('.')
+ s_instance = kwds_s['s_' + objname]
+ s_arg = s_instance.classdef.about_attribute(fieldname)
+ assert s_arg is not None
args_s.append(s_arg)
bk.emulate_pbc_call(uniquekey, s_func, args_s)
@@ -416,9 +423,33 @@
greens_v = []
reds_v = []
for name in driver.greens:
- i = kwds_i['i_' + name]
- r_green = hop.args_r[i]
- v_green = hop.inputarg(r_green, arg=i)
+ if '.' not in name:
+ i = kwds_i['i_' + name]
+ r_green = hop.args_r[i]
+ v_green = hop.inputarg(r_green, arg=i)
+ else:
+ if hop.rtyper.type_system.name == 'ootypesystem':
+ py.test.skip("lltype only")
+ objname, fieldname = name.split('.') # see test_green_field
+ assert objname in driver.reds
+ i = kwds_i['i_' + objname]
+ r_red = hop.args_r[i]
+ while True:
+ try:
+ mangled_name, r_field = r_red._get_field(fieldname)
+ break
+ except KeyError:
+ pass
+ assert r_red.rbase is not None, (
+ "field %r not found in %r" % (name,
+ r_red.lowleveltype.TO))
+ r_red = r_red.rbase
+ assert r_red.lowleveltype.TO._immutable_field(mangled_name), (
+ "field %r must be declared as immutable" % name)
+ v_red = hop.inputarg(r_red, arg=i)
+ c_llname = hop.inputconst(lltype.Void, mangled_name)
+ v_green = hop.genop('getfield', [v_red, c_llname],
+ resulttype = r_field)
greens_v.append(v_green)
for name in driver.reds:
i = kwds_i['i_' + name]
Modified: pypy/branch/rsre-jit/pypy/rlib/test/test_jit.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/rlib/test/test_jit.py (original)
+++ pypy/branch/rsre-jit/pypy/rlib/test/test_jit.py Sat Oct 16 18:16:59 2010
@@ -1,4 +1,5 @@
import py
+from pypy.conftest import option
from pypy.rlib.jit import hint, we_are_jitted, JitDriver, purefunction_promote
from pypy.rlib.jit import JitHintError
from pypy.translator.translator import TranslationContext, graphof
@@ -104,6 +105,26 @@
return n
py.test.raises(JitHintError, self.gengraph, fn, [int])
+ def test_green_field(self):
+ def get_printable_location(xfoo):
+ return str(ord(xfoo)) # xfoo must be annotated as a character
+ myjitdriver = JitDriver(greens=['x.foo'], reds=['n', 'x'],
+ get_printable_location=get_printable_location)
+ class A(object):
+ _immutable_fields_ = ['foo']
+ def fn(n):
+ x = A()
+ x.foo = chr(n)
+ while n > 0:
+ myjitdriver.can_enter_jit(x=x, n=n)
+ myjitdriver.jit_merge_point(x=x, n=n)
+ n -= 1
+ return n
+ t = self.gengraph(fn, [int])[0]
+ if option.view:
+ t.view()
+ # assert did not raise
+
class TestJITLLtype(BaseTestJIT, LLRtypeMixin):
pass
From arigo at codespeak.net Sat Oct 16 18:24:41 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 16 Oct 2010 18:24:41 +0200 (CEST)
Subject: [pypy-svn] r78025 - in pypy/branch/rsre-jit/pypy:
jit/metainterp/test rlib
Message-ID: <20101016162441.A295B36E393@codespeak.net>
Author: arigo
Date: Sat Oct 16 18:24:40 2010
New Revision: 78025
Modified:
pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_immutable.py
pypy/branch/rsre-jit/pypy/rlib/jit.py
Log:
The first test about it passes.
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_immutable.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_immutable.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_immutable.py Sat Oct 16 18:24:40 2010
@@ -1,4 +1,5 @@
from pypy.jit.metainterp.test.test_basic import LLJitMixin, OOJitMixin
+from pypy.rlib.jit import JitDriver
class ImmutableFieldsTests:
@@ -85,6 +86,29 @@
int_add=3)
+ def test_green_field(self):
+ myjitdriver = JitDriver(greens=['ctx.x'], reds=['ctx'])
+ class Ctx(object):
+ _immutable_fields_ = ['x']
+ def __init__(self, x, y):
+ self.x = x
+ self.y = y
+ def f(x, y):
+ ctx = Ctx(x, y)
+ while 1:
+ myjitdriver.can_enter_jit(ctx=ctx)
+ myjitdriver.jit_merge_point(ctx=ctx)
+ ctx.y -= 1
+ if ctx.y < 0:
+ return ctx.y
+ def g(y):
+ return f(5, y) + f(6, y)
+ #
+ res = self.meta_interp(g, [7])
+ assert res == -2
+ self.check_loop_count(2)
+
+
class TestLLtypeImmutableFieldsTests(ImmutableFieldsTests, LLJitMixin):
pass
Modified: pypy/branch/rsre-jit/pypy/rlib/jit.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/rlib/jit.py (original)
+++ pypy/branch/rsre-jit/pypy/rlib/jit.py Sat Oct 16 18:24:40 2010
@@ -433,6 +433,7 @@
objname, fieldname = name.split('.') # see test_green_field
assert objname in driver.reds
i = kwds_i['i_' + objname]
+ s_red = hop.args_s[i]
r_red = hop.args_r[i]
while True:
try:
@@ -450,6 +451,9 @@
c_llname = hop.inputconst(lltype.Void, mangled_name)
v_green = hop.genop('getfield', [v_red, c_llname],
resulttype = r_field)
+ s_green = s_red.classdef.about_attribute(fieldname)
+ assert s_green is not None
+ hop.rtyper.annotator.setbinding(v_green, s_green)
greens_v.append(v_green)
for name in driver.reds:
i = kwds_i['i_' + name]
From arigo at codespeak.net Sat Oct 16 20:05:30 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 16 Oct 2010 20:05:30 +0200 (CEST)
Subject: [pypy-svn] r78026 - in pypy/branch/rsre-jit/pypy: jit/codewriter
jit/metainterp jit/metainterp/test rlib
Message-ID: <20101016180530.2D95136E393@codespeak.net>
Author: arigo
Date: Sat Oct 16 20:05:28 2010
New Revision: 78026
Added:
pypy/branch/rsre-jit/pypy/jit/metainterp/greenfield.py
pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_greenfield.py
Modified:
pypy/branch/rsre-jit/pypy/jit/codewriter/call.py
pypy/branch/rsre-jit/pypy/jit/codewriter/jtransform.py
pypy/branch/rsre-jit/pypy/jit/metainterp/blackhole.py
pypy/branch/rsre-jit/pypy/jit/metainterp/compile.py
pypy/branch/rsre-jit/pypy/jit/metainterp/jitdriver.py
pypy/branch/rsre-jit/pypy/jit/metainterp/pyjitpl.py
pypy/branch/rsre-jit/pypy/jit/metainterp/resume.py
pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_immutable.py
pypy/branch/rsre-jit/pypy/jit/metainterp/warmspot.py
pypy/branch/rsre-jit/pypy/rlib/jit.py
Log:
Implement green fields. Probably buggy in various ways: I used a single test
to touch all these files... I will write more tests now.
Modified: pypy/branch/rsre-jit/pypy/jit/codewriter/call.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/codewriter/call.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/codewriter/call.py Sat Oct 16 20:05:28 2010
@@ -277,3 +277,11 @@
return seen.pop()
else:
return None
+
+ def could_be_green_field(self, GTYPE, fieldname):
+ GTYPE_fieldname = (GTYPE, fieldname)
+ for jd in self.jitdrivers_sd:
+ if jd.greenfield_info is not None:
+ if GTYPE_fieldname in jd.greenfield_info.green_fields:
+ return True
+ return False
Modified: pypy/branch/rsre-jit/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/codewriter/jtransform.py Sat Oct 16 20:05:28 2010
@@ -521,7 +521,12 @@
# check for deepfrozen structures that force constant-folding
immut = v_inst.concretetype.TO._immutable_field(c_fieldname.value)
if immut:
- pure = '_pure'
+ if (self.callcontrol is not None and
+ self.callcontrol.could_be_green_field(v_inst.concretetype.TO,
+ c_fieldname.value)):
+ pure = '_greenfield'
+ else:
+ pure = '_pure'
if immut == "[*]":
self.immutable_arrays[op.result] = True
else:
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/blackhole.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/blackhole.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/blackhole.py Sat Oct 16 20:05:28 2010
@@ -1073,6 +1073,10 @@
bhimpl_getfield_vable_r = bhimpl_getfield_gc_r
bhimpl_getfield_vable_f = bhimpl_getfield_gc_f
+ bhimpl_getfield_gc_i_greenfield = bhimpl_getfield_gc_i
+ bhimpl_getfield_gc_r_greenfield = bhimpl_getfield_gc_r
+ bhimpl_getfield_gc_f_greenfield = bhimpl_getfield_gc_f
+
@arguments("cpu", "i", "d", returns="i")
def bhimpl_getfield_raw_i(cpu, struct, fielddescr):
return cpu.bh_getfield_raw_i(struct, fielddescr)
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/compile.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/compile.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/compile.py Sat Oct 16 20:05:28 2010
@@ -370,7 +370,8 @@
from pypy.jit.metainterp.resume import force_from_resumedata
metainterp_sd = self.metainterp_sd
vinfo = self.jitdriver_sd.virtualizable_info
- all_virtuals = force_from_resumedata(metainterp_sd, self, vinfo)
+ ginfo = self.jitdriver_sd.greenfield_info
+ all_virtuals = force_from_resumedata(metainterp_sd, self, vinfo, ginfo)
# The virtualizable data was stored on the real virtualizable above.
# Handle all_virtuals: keep them for later blackholing from the
# future failure of the GUARD_NOT_FORCED
Added: pypy/branch/rsre-jit/pypy/jit/metainterp/greenfield.py
==============================================================================
--- (empty file)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/greenfield.py Sat Oct 16 20:05:28 2010
@@ -0,0 +1,27 @@
+from pypy.jit.metainterp.typesystem import deref
+
+
+class GreenFieldInfo(object):
+
+ def __init__(self, cpu, jd):
+ self.cpu = cpu
+ self.jitdriver_sd = jd
+ # XXX for now, only supports a single instance,
+ # but several fields of it can be green
+ seen = set()
+ for name in jd.jitdriver.greens:
+ if '.' in name:
+ objname, fieldname = name.split('.')
+ seen.add(objname)
+ assert len(seen) == 1, (
+ "Current limitation: you can only give one instance with green "
+ "fields. Found %r" % seen.keys())
+ self.red_index = jd.jitdriver.reds.index(objname)
+ #
+ # a list of (GTYPE, fieldname)
+ self.green_fields = jd.jitdriver.ll_greenfields.values()
+ self.green_field_descrs = [cpu.fielddescrof(GTYPE, fieldname)
+ for GTYPE, fieldname in self.green_fields]
+
+ def _freeze_(self):
+ return True
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/jitdriver.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/jitdriver.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/jitdriver.py Sat Oct 16 20:05:28 2010
@@ -13,6 +13,7 @@
# self.num_red_args ... pypy.jit.metainterp.warmspot
# self.result_type ... pypy.jit.metainterp.warmspot
# self.virtualizable_info... pypy.jit.metainterp.warmspot
+ # self.greenfield_info ... pypy.jit.metainterp.warmspot
# self.warmstate ... pypy.jit.metainterp.warmspot
# self.handle_jitexc_from_bh pypy.jit.metainterp.warmspot
# self.portal_finishtoken... pypy.jit.metainterp.pyjitpl
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/pyjitpl.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/pyjitpl.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/pyjitpl.py Sat Oct 16 20:05:28 2010
@@ -498,6 +498,22 @@
opimpl_getfield_gc_r_pure = _opimpl_getfield_gc_pure_any
opimpl_getfield_gc_f_pure = _opimpl_getfield_gc_pure_any
+ @arguments("orgpc", "box", "descr")
+ def _opimpl_getfield_gc_greenfield_any(self, pc, box, fielddescr):
+ ginfo = self.metainterp.jitdriver_sd.greenfield_info
+ if (ginfo is not None and fielddescr in ginfo.green_field_descrs
+ and not self._nonstandard_virtualizable(pc, box)):
+ # fetch the result, but consider it as a Const box and don't
+ # record any operation
+ resbox = executor.execute(self.metainterp.cpu, self.metainterp,
+ rop.GETFIELD_GC_PURE, fielddescr, box)
+ return resbox.constbox()
+ # fall-back
+ return self.execute_with_descr(rop.GETFIELD_GC_PURE, fielddescr, box)
+ opimpl_getfield_gc_i_greenfield = _opimpl_getfield_gc_greenfield_any
+ opimpl_getfield_gc_r_greenfield = _opimpl_getfield_gc_greenfield_any
+ opimpl_getfield_gc_f_greenfield = _opimpl_getfield_gc_greenfield_any
+
@arguments("box", "descr", "box")
def _opimpl_setfield_gc_any(self, box, fielddescr, valuebox):
self.execute_with_descr(rop.SETFIELD_GC, fielddescr, box, valuebox)
@@ -529,7 +545,8 @@
def _nonstandard_virtualizable(self, pc, box):
# returns True if 'box' is actually not the "standard" virtualizable
# that is stored in metainterp.virtualizable_boxes[-1]
- if self.metainterp.jitdriver_sd.virtualizable_info is None:
+ if (self.metainterp.jitdriver_sd.virtualizable_info is None and
+ self.metainterp.jitdriver_sd.greenfield_info is None):
return True # can occur in case of multiple JITs
standard_box = self.metainterp.virtualizable_boxes[-1]
if standard_box is box:
@@ -998,13 +1015,15 @@
guard_op = metainterp.history.record(opnum, moreargs, None,
descr=resumedescr)
virtualizable_boxes = None
- if metainterp.jitdriver_sd.virtualizable_info is not None:
+ if (metainterp.jitdriver_sd.virtualizable_info is not None or
+ metainterp.jitdriver_sd.greenfield_info is not None):
virtualizable_boxes = metainterp.virtualizable_boxes
saved_pc = self.pc
if resumepc >= 0:
self.pc = resumepc
resume.capture_resumedata(metainterp.framestack, virtualizable_boxes,
- metainterp.virtualref_boxes, resumedescr)
+ metainterp.virtualref_boxes,
+ resumedescr)
self.pc = saved_pc
self.metainterp.staticdata.profiler.count_ops(opnum, GUARDS)
# count
@@ -1646,6 +1665,7 @@
duplicates)
live_arg_boxes += self.virtualizable_boxes
live_arg_boxes.pop()
+ #
assert len(self.virtualref_boxes) == 0, "missing virtual_ref_finish()?"
# Called whenever we reach the 'loop_header' hint.
# First, attempt to make a bridge:
@@ -1832,6 +1852,7 @@
f.setup_call(original_boxes)
assert self.in_recursion == 0
self.virtualref_boxes = []
+ self.initialize_withgreenfields(original_boxes)
self.initialize_virtualizable(original_boxes)
def initialize_state_from_guard_failure(self, resumedescr):
@@ -1856,6 +1877,14 @@
self.virtualizable_boxes.append(virtualizable_box)
self.initialize_virtualizable_enter()
+ def initialize_withgreenfields(self, original_boxes):
+ ginfo = self.jitdriver_sd.greenfield_info
+ if ginfo is not None:
+ assert self.jitdriver_sd.virtualizable_info is None
+ index = (self.jitdriver_sd.num_green_args +
+ ginfo.red_index)
+ self.virtualizable_boxes = [original_boxes[index]]
+
def initialize_virtualizable_enter(self):
vinfo = self.jitdriver_sd.virtualizable_info
virtualizable_box = self.virtualizable_boxes[-1]
@@ -1949,8 +1978,10 @@
def rebuild_state_after_failure(self, resumedescr):
vinfo = self.jitdriver_sd.virtualizable_info
+ ginfo = self.jitdriver_sd.greenfield_info
self.framestack = []
- boxlists = resume.rebuild_from_resumedata(self, resumedescr, vinfo)
+ boxlists = resume.rebuild_from_resumedata(self, resumedescr, vinfo,
+ ginfo)
inputargs_and_holes, virtualizable_boxes, virtualref_boxes = boxlists
#
# virtual refs: make the vrefs point to the freshly allocated virtuals
@@ -1975,6 +2006,12 @@
assert not virtualizable.vable_token
# fill the virtualizable with the local boxes
self.synchronize_virtualizable()
+ #
+ elif self.jitdriver_sd.greenfield_info:
+ self.virtualizable_boxes = virtualizable_boxes
+ else:
+ assert not virtualizable_boxes
+ #
return inputargs_and_holes
def check_synchronized_virtualizable(self):
@@ -2048,7 +2085,8 @@
for i in range(len(boxes)):
if boxes[i] is oldbox:
boxes[i] = newbox
- if self.jitdriver_sd.virtualizable_info is not None:
+ if (self.jitdriver_sd.virtualizable_info is not None or
+ self.jitdriver_sd.greenfield_info is not None):
boxes = self.virtualizable_boxes
for i in range(len(boxes)):
if boxes[i] is oldbox:
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/resume.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/resume.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/resume.py Sat Oct 16 20:05:28 2010
@@ -689,9 +689,11 @@
# ---------- when resuming for pyjitpl.py, make boxes ----------
-def rebuild_from_resumedata(metainterp, storage, virtualizable_info):
+def rebuild_from_resumedata(metainterp, storage, virtualizable_info,
+ greenfield_info):
resumereader = ResumeDataBoxReader(storage, metainterp)
- boxes = resumereader.consume_vref_and_vable_boxes(virtualizable_info)
+ boxes = resumereader.consume_vref_and_vable_boxes(virtualizable_info,
+ greenfield_info)
virtualizable_boxes, virtualref_boxes = boxes
frameinfo = storage.rd_frame_info_list
while True:
@@ -901,8 +903,9 @@
resumereader = ResumeDataDirectReader(blackholeinterpbuilder.cpu, storage,
all_virtuals)
vinfo = jitdriver_sd.virtualizable_info
+ ginfo = jitdriver_sd.greenfield_info
vrefinfo = blackholeinterpbuilder.metainterp_sd.virtualref_info
- resumereader.consume_vref_and_vable(vrefinfo, vinfo)
+ resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo)
#
# First get a chain of blackhole interpreters whose length is given
# by the depth of rd_frame_info_list. The first one we get must be
@@ -932,11 +935,11 @@
resumereader.done()
return firstbh
-def force_from_resumedata(metainterp_sd, storage, vinfo=None):
+def force_from_resumedata(metainterp_sd, storage, vinfo, ginfo):
resumereader = ResumeDataDirectReader(metainterp_sd.cpu, storage)
resumereader.handling_async_forcing()
vrefinfo = metainterp_sd.virtualref_info
- resumereader.consume_vref_and_vable(vrefinfo, vinfo)
+ resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo)
return resumereader.force_all_virtuals()
class ResumeDataDirectReader(AbstractResumeDataReader):
@@ -1011,11 +1014,12 @@
return specialize_value(TYPE, x)
load_value_of_type._annspecialcase_ = 'specialize:arg(1)'
- def consume_vref_and_vable(self, vrefinfo, vinfo):
+ def consume_vref_and_vable(self, vrefinfo, vinfo, ginfo):
nums = self.cur_numb.nums
self.cur_numb = self.cur_numb.prev
if self.resume_after_guard_not_forced != 2:
end_vref = self.consume_vable_info(vinfo, nums)
+ if ginfo is not None: end_vref -= 1
self.consume_virtualref_info(vrefinfo, nums, end_vref)
def allocate_with_vtable(self, known_class):
Added: pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_greenfield.py
==============================================================================
--- (empty file)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_greenfield.py Sat Oct 16 20:05:28 2010
@@ -0,0 +1,35 @@
+from pypy.jit.metainterp.test.test_basic import LLJitMixin, OOJitMixin
+from pypy.rlib.jit import JitDriver
+
+
+class GreenFieldsTests:
+
+ def test_green_field_1(self):
+ myjitdriver = JitDriver(greens=['ctx.x'], reds=['ctx'])
+ class Ctx(object):
+ _immutable_fields_ = ['x']
+ def __init__(self, x, y):
+ self.x = x
+ self.y = y
+ def f(x, y):
+ ctx = Ctx(x, y)
+ while 1:
+ myjitdriver.can_enter_jit(ctx=ctx)
+ myjitdriver.jit_merge_point(ctx=ctx)
+ ctx.y -= 1
+ if ctx.y < 0:
+ return ctx.y
+ def g(y):
+ return f(5, y) + f(6, y)
+ #
+ res = self.meta_interp(g, [7])
+ assert res == -2
+ self.check_loop_count(2)
+ self.check_loops(guard_value=0)
+
+
+class TestLLtypeGreenFieldsTests(GreenFieldsTests, LLJitMixin):
+ pass
+
+class TestOOtypeGreenFieldsTests(GreenFieldsTests, OOJitMixin):
+ pass
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_immutable.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_immutable.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_immutable.py Sat Oct 16 20:05:28 2010
@@ -1,5 +1,4 @@
from pypy.jit.metainterp.test.test_basic import LLJitMixin, OOJitMixin
-from pypy.rlib.jit import JitDriver
class ImmutableFieldsTests:
@@ -86,29 +85,6 @@
int_add=3)
- def test_green_field(self):
- myjitdriver = JitDriver(greens=['ctx.x'], reds=['ctx'])
- class Ctx(object):
- _immutable_fields_ = ['x']
- def __init__(self, x, y):
- self.x = x
- self.y = y
- def f(x, y):
- ctx = Ctx(x, y)
- while 1:
- myjitdriver.can_enter_jit(ctx=ctx)
- myjitdriver.jit_merge_point(ctx=ctx)
- ctx.y -= 1
- if ctx.y < 0:
- return ctx.y
- def g(y):
- return f(5, y) + f(6, y)
- #
- res = self.meta_interp(g, [7])
- assert res == -2
- self.check_loop_count(2)
-
-
class TestLLtypeImmutableFieldsTests(ImmutableFieldsTests, LLJitMixin):
pass
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/warmspot.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/warmspot.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/warmspot.py Sat Oct 16 20:05:28 2010
@@ -286,10 +286,20 @@
def make_virtualizable_infos(self):
vinfos = {}
for jd in self.jitdrivers_sd:
+ #
+ jd.greenfield_info = None
+ for name in jd.jitdriver.greens:
+ if '.' in name:
+ from pypy.jit.metainterp.greenfield import GreenFieldInfo
+ jd.greenfield_info = GreenFieldInfo(self.cpu, jd)
+ break
+ #
if not jd.jitdriver.virtualizables:
jd.virtualizable_info = None
jd.index_of_virtualizable = -1
continue
+ else:
+ assert jd.greenfield_info is None, "XXX not supported yet"
#
jitdriver = jd.jitdriver
assert len(jitdriver.virtualizables) == 1 # for now
Modified: pypy/branch/rsre-jit/pypy/rlib/jit.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/rlib/jit.py (original)
+++ pypy/branch/rsre-jit/pypy/rlib/jit.py Sat Oct 16 20:05:28 2010
@@ -445,8 +445,13 @@
"field %r not found in %r" % (name,
r_red.lowleveltype.TO))
r_red = r_red.rbase
- assert r_red.lowleveltype.TO._immutable_field(mangled_name), (
+ GTYPE = r_red.lowleveltype.TO
+ assert GTYPE._immutable_field(mangled_name), (
"field %r must be declared as immutable" % name)
+ if not hasattr(driver, 'll_greenfields'):
+ driver.ll_greenfields = {}
+ driver.ll_greenfields[name] = GTYPE, mangled_name
+ #
v_red = hop.inputarg(r_red, arg=i)
c_llname = hop.inputconst(lltype.Void, mangled_name)
v_green = hop.genop('getfield', [v_red, c_llname],
From trundle at codespeak.net Sun Oct 17 01:29:32 2010
From: trundle at codespeak.net (trundle at codespeak.net)
Date: Sun, 17 Oct 2010 01:29:32 +0200 (CEST)
Subject: [pypy-svn] r78027 - in pypy/branch/fast-forward: lib-python
pypy/interpreter pypy/interpreter/test
Message-ID: <20101016232932.83C1836E3F0@codespeak.net>
Author: trundle
Date: Sun Oct 17 01:29:28 2010
New Revision: 78027
Modified:
pypy/branch/fast-forward/lib-python/TODO
pypy/branch/fast-forward/pypy/interpreter/error.py
pypy/branch/fast-forward/pypy/interpreter/test/test_generator.py
Log:
Raising a string as exception is not allowed anymore.
Modified: pypy/branch/fast-forward/lib-python/TODO
==============================================================================
--- pypy/branch/fast-forward/lib-python/TODO (original)
+++ pypy/branch/fast-forward/lib-python/TODO Sun Oct 17 01:29:28 2010
@@ -34,10 +34,8 @@
- Ast objects should be picklable, see in pypy/module/_ast/test/test_ast.py:
test_pickle()
-- "exceptions must be old-style classes or derived from BaseException, not str"
- in the 'raise' statement and generator.throw(), catching a string
- should emit a DeprecationWarning ("catching of string exceptions is
- deprecated")
+- catching a string should emit a DeprecationWarning ("catching of
+ string exceptions is deprecated")
- missing builtin: memoryview
Modified: pypy/branch/fast-forward/pypy/interpreter/error.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/error.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/error.py Sun Oct 17 01:29:28 2010
@@ -202,19 +202,14 @@
w_value = space.call_function(w_type, w_value)
w_type = space.exception_getclass(w_value)
- elif space.full_exceptions and space.is_w(space.type(w_type),
- space.w_str):
- space.warn("raising a string exception is deprecated",
- space.w_DeprecationWarning)
-
else:
# the only case left here is (inst, None), from a 'raise inst'.
w_inst = w_type
w_instclass = space.exception_getclass(w_inst)
if not space.exception_is_valid_class_w(w_instclass):
instclassname = w_instclass.getname(space, '?')
- msg = ("exceptions must be classes, or instances, "
- "or strings (deprecated), not %s")
+ msg = ("exceptions must be old-style classes or derived "
+ "from BaseException, not %s")
raise operationerrfmt(space.w_TypeError, msg, instclassname)
if not space.is_w(w_value, space.w_None):
Modified: pypy/branch/fast-forward/pypy/interpreter/test/test_generator.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/test/test_generator.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/test/test_generator.py Sun Oct 17 01:29:28 2010
@@ -89,9 +89,10 @@
pass
g = f()
g.next()
- # String exceptions are allowed (with DeprecationWarning)
- assert g.throw("Error") == 3
- raises(StopIteration, g.throw, "Error")
+ # String exceptions are not allowed anymore
+ raises(TypeError, g.throw, "Error")
+ assert g.throw(Exception) == 3
+ raises(StopIteration, g.throw, Exception)
def test_throw6(self):
def f():
From fijall at gmail.com Sun Oct 17 10:29:58 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 17 Oct 2010 10:29:58 +0200
Subject: [pypy-svn] r77933 - pypy/branch/jitffi/pypy/jit/metainterp/test
In-Reply-To: <20101014133223.A97D9282BEA@codespeak.net>
References: <20101014133223.A97D9282BEA@codespeak.net>
Message-ID:
The exception still can't happen though.
On Thu, Oct 14, 2010 at 3:32 PM, wrote:
> Author: antocuni
> Date: Thu Oct 14 15:32:22 2010
> New Revision: 77933
>
> Modified:
> ? pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
> Log:
> update this test too
>
>
> Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
> ==============================================================================
> --- pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py (original)
> +++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py Thu Oct 14 15:32:22 2010
> @@ -77,7 +77,9 @@
> ? ? ? ? call(0, p2, ? ? ? ? ? ? descr=libffi_prepare)
> ? ? ? ? call(0, p2, i0, ? ? ? ? descr=libffi_push_arg)
> ? ? ? ? call(0, p2, f1, ? ? ? ? descr=libffi_push_arg)
> - ? ? ? ?i3 = call(0, p2, 12345, descr=libffi_call)
> + ? ? ? ?i3 = call_may_force(0, p2, 12345, descr=libffi_call)
> + ? ? ? ?guard_not_forced() []
> + ? ? ? ?guard_no_exception() []
> ? ? ? ? jump(i3, f1, p2)
> ? ? ? ? """
> ? ? ? ? expected = ops
> _______________________________________________
> pypy-svn mailing list
> pypy-svn at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-svn
>
From afa at codespeak.net Mon Oct 18 10:00:22 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 18 Oct 2010 10:00:22 +0200 (CEST)
Subject: [pypy-svn] r78031 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101018080022.0359A282B90@codespeak.net>
Author: afa
Date: Mon Oct 18 10:00:19 2010
New Revision: 78031
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
Log:
Add FileIO.name, FileIO.mode
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Mon Oct 18 10:00:19 2010
@@ -1,5 +1,6 @@
from pypy.module._io.interp_iobase import W_RawIOBase
-from pypy.interpreter.typedef import TypeDef
+from pypy.interpreter.typedef import (
+ TypeDef, interp_attrproperty_w, GetSetProperty)
from pypy.interpreter.gateway import interp2app, unwrap_spec, Arguments
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
from pypy.interpreter.error import OperationError, wrap_oserror2
@@ -77,6 +78,7 @@
self.writable = False
self.seekable = -1
self.closefd = True
+ self.w_name = None
@unwrap_spec(ObjSpace, W_Root, Arguments)
def descr_new(space, w_subtype, __args__):
@@ -113,6 +115,19 @@
except OSError, e:
raise wrap_oserror2(space, e, w_name)
self.closefd = bool(closefd)
+ self.w_name = w_name
+
+ def _mode(self):
+ if self.readable:
+ if self.writable:
+ return 'rb+'
+ else:
+ return 'rb'
+ else:
+ return 'wb'
+
+ def descr_get_mode(space, self):
+ return space.wrap(self._mode())
def _check_closed(self, space):
if self.fd < 0:
@@ -157,5 +172,7 @@
readable = interp2app(W_FileIO.readable_w),
writable = interp2app(W_FileIO.writable_w),
seekable = interp2app(W_FileIO.seekable_w),
+ name = interp_attrproperty_w('w_name', cls=W_FileIO),
+ mode = GetSetProperty(W_FileIO.descr_get_mode),
)
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Mon Oct 18 10:00:19 2010
@@ -50,6 +50,8 @@
def test_open(self):
import io
f = io.open(self.tmpfile, "rb")
+ assert f.name.endswith('tmpfile')
+ assert f.mode == 'rb'
f.close()
def test_open_writable(self):
From afa at codespeak.net Mon Oct 18 10:04:15 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 18 Oct 2010 10:04:15 +0200 (CEST)
Subject: [pypy-svn] r78032 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101018080415.68403282B90@codespeak.net>
Author: afa
Date: Mon Oct 18 10:04:13 2010
New Revision: 78032
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
Log:
add FileIO.fileno()
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Mon Oct 18 10:04:13 2010
@@ -162,6 +162,11 @@
self.seekable = 1
return space.newbool(self.seekable == 1)
+ @unwrap_spec('self', ObjSpace)
+ def fileno_w(self, space):
+ self._check_closed(space)
+ return space.wrap(self.fd)
+
W_FileIO.typedef = TypeDef(
'FileIO', W_RawIOBase.typedef,
__new__ = interp2app(W_FileIO.descr_new.im_func),
@@ -172,6 +177,7 @@
readable = interp2app(W_FileIO.readable_w),
writable = interp2app(W_FileIO.writable_w),
seekable = interp2app(W_FileIO.seekable_w),
+ fileno = interp2app(W_FileIO.fileno_w),
name = interp_attrproperty_w('w_name', cls=W_FileIO),
mode = GetSetProperty(W_FileIO.descr_get_mode),
)
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Mon Oct 18 10:04:13 2010
@@ -65,4 +65,5 @@
os = self.posix
fd = os.open(self.tmpfile, os.O_RDONLY, 0666)
f = io.open(fd, "rb")
+ assert f.fileno() == fd
f.close()
From antocuni at codespeak.net Mon Oct 18 11:22:26 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 18 Oct 2010 11:22:26 +0200 (CEST)
Subject: [pypy-svn] r78033 - in pypy/branch/jitffi: .
lib-python/modified-2.5.2/ctypes lib_pypy/_ctypes pypy
pypy/config pypy/doc/config pypy/interpreter
pypy/interpreter/pyparser pypy/jit/backend/test
pypy/jit/backend/x86 pypy/jit/backend/x86/test
pypy/jit/codewriter pypy/jit/codewriter/test
pypy/jit/metainterp pypy/jit/metainterp/optimizeopt
pypy/jit/metainterp/test pypy/jit/tool
pypy/module/__builtin__ pypy/module/__builtin__/test
pypy/module/_rawffi pypy/module/_weakref
pypy/module/array/benchmark pypy/module/array/test
pypy/module/cpyext pypy/module/cpyext/test pypy/module/gc
pypy/module/pypyjit/test pypy/module/signal pypy/objspace/std
pypy/objspace/std/test pypy/rlib pypy/rlib/test pypy/rpython
pypy/rpython/lltypesystem pypy/rpython/module
pypy/rpython/test pypy/translator/c/gcc
pypy/translator/c/gcc/test/elf pypy/translator/c/src
Message-ID: <20101018092226.0B2EA282B90@codespeak.net>
Author: antocuni
Date: Mon Oct 18 11:22:20 2010
New Revision: 78033
Added:
pypy/branch/jitffi/pypy/doc/config/objspace.std.withmapdict.txt
- copied unchanged from r78032, pypy/trunk/pypy/doc/config/objspace.std.withmapdict.txt
pypy/branch/jitffi/pypy/doc/config/objspace.std.withstrbuf.txt
- copied unchanged from r78032, pypy/trunk/pypy/doc/config/objspace.std.withstrbuf.txt
pypy/branch/jitffi/pypy/objspace/std/mapdict.py
- copied unchanged from r78032, pypy/trunk/pypy/objspace/std/mapdict.py
pypy/branch/jitffi/pypy/objspace/std/strbufobject.py
- copied unchanged from r78032, pypy/trunk/pypy/objspace/std/strbufobject.py
pypy/branch/jitffi/pypy/objspace/std/test/test_mapdict.py
- copied unchanged from r78032, pypy/trunk/pypy/objspace/std/test/test_mapdict.py
pypy/branch/jitffi/pypy/objspace/std/test/test_strbufobject.py
- copied unchanged from r78032, pypy/trunk/pypy/objspace/std/test/test_strbufobject.py
pypy/branch/jitffi/pypy/rlib/rerased.py
- copied unchanged from r78032, pypy/trunk/pypy/rlib/rerased.py
pypy/branch/jitffi/pypy/rlib/test/test_rerased.py
- copied unchanged from r78032, pypy/trunk/pypy/rlib/test/test_rerased.py
pypy/branch/jitffi/pypy/translator/c/gcc/test/elf/track10.s
- copied unchanged from r78032, pypy/trunk/pypy/translator/c/gcc/test/elf/track10.s
pypy/branch/jitffi/pypy/translator/c/gcc/test/elf/track11.s
- copied unchanged from r78032, pypy/trunk/pypy/translator/c/gcc/test/elf/track11.s
Modified:
pypy/branch/jitffi/ (props changed)
pypy/branch/jitffi/lib-python/modified-2.5.2/ctypes/__init__.py
pypy/branch/jitffi/lib_pypy/_ctypes/function.py
pypy/branch/jitffi/pypy/ (props changed)
pypy/branch/jitffi/pypy/config/pypyoption.py
pypy/branch/jitffi/pypy/interpreter/baseobjspace.py
pypy/branch/jitffi/pypy/interpreter/pycode.py
pypy/branch/jitffi/pypy/interpreter/pyopcode.py
pypy/branch/jitffi/pypy/interpreter/pyparser/genpytokenize.py
pypy/branch/jitffi/pypy/interpreter/pyparser/pytokenize.py
pypy/branch/jitffi/pypy/interpreter/typedef.py
pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py
pypy/branch/jitffi/pypy/jit/backend/test/test_ll_random.py
pypy/branch/jitffi/pypy/jit/backend/x86/regalloc.py
pypy/branch/jitffi/pypy/jit/backend/x86/test/test_runner.py
pypy/branch/jitffi/pypy/jit/backend/x86/test/test_string.py
pypy/branch/jitffi/pypy/jit/codewriter/effectinfo.py
pypy/branch/jitffi/pypy/jit/codewriter/jtransform.py
pypy/branch/jitffi/pypy/jit/codewriter/test/test_jtransform.py
pypy/branch/jitffi/pypy/jit/metainterp/history.py
pypy/branch/jitffi/pypy/jit/metainterp/optimize_nopspec.py
pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/intbounds.py
pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/optimizer.py (props changed)
pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/rewrite.py
pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/string.py
pypy/branch/jitffi/pypy/jit/metainterp/resume.py
pypy/branch/jitffi/pypy/jit/metainterp/test/oparser.py
pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefindnode.py
pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py
pypy/branch/jitffi/pypy/jit/metainterp/test/test_resume.py
pypy/branch/jitffi/pypy/jit/metainterp/test/test_string.py
pypy/branch/jitffi/pypy/jit/tool/traceviewer.py
pypy/branch/jitffi/pypy/module/__builtin__/interp_classobj.py
pypy/branch/jitffi/pypy/module/__builtin__/test/test_classobj.py
pypy/branch/jitffi/pypy/module/_rawffi/array.py
pypy/branch/jitffi/pypy/module/_weakref/interp__weakref.py
pypy/branch/jitffi/pypy/module/array/benchmark/Makefile (props changed)
pypy/branch/jitffi/pypy/module/array/benchmark/intimg.c (props changed)
pypy/branch/jitffi/pypy/module/array/benchmark/intimgtst.c (props changed)
pypy/branch/jitffi/pypy/module/array/benchmark/intimgtst.py (props changed)
pypy/branch/jitffi/pypy/module/array/benchmark/loop.c (props changed)
pypy/branch/jitffi/pypy/module/array/benchmark/sum.c (props changed)
pypy/branch/jitffi/pypy/module/array/benchmark/sumtst.c (props changed)
pypy/branch/jitffi/pypy/module/array/benchmark/sumtst.py (props changed)
pypy/branch/jitffi/pypy/module/array/test/test_array_old.py (props changed)
pypy/branch/jitffi/pypy/module/cpyext/classobject.py
pypy/branch/jitffi/pypy/module/cpyext/test/test_unicodeobject.py
pypy/branch/jitffi/pypy/module/gc/referents.py
pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py
pypy/branch/jitffi/pypy/module/signal/interp_signal.py
pypy/branch/jitffi/pypy/objspace/std/celldict.py
pypy/branch/jitffi/pypy/objspace/std/dictmultiobject.py
pypy/branch/jitffi/pypy/objspace/std/model.py
pypy/branch/jitffi/pypy/objspace/std/objspace.py
pypy/branch/jitffi/pypy/objspace/std/sharingdict.py
pypy/branch/jitffi/pypy/objspace/std/stringobject.py
pypy/branch/jitffi/pypy/objspace/std/stringtype.py
pypy/branch/jitffi/pypy/objspace/std/test/test_dictmultiobject.py
pypy/branch/jitffi/pypy/objspace/std/test/test_shadowtracking.py
pypy/branch/jitffi/pypy/objspace/std/typeobject.py
pypy/branch/jitffi/pypy/rlib/_rsocket_rffi.py
pypy/branch/jitffi/pypy/rlib/clibffi.py
pypy/branch/jitffi/pypy/rlib/rmmap.py
pypy/branch/jitffi/pypy/rlib/rstring.py
pypy/branch/jitffi/pypy/rlib/test/test_rsocket.py
pypy/branch/jitffi/pypy/rlib/test/test_rstring.py
pypy/branch/jitffi/pypy/rlib/test/test_runicode.py
pypy/branch/jitffi/pypy/rpython/lltypesystem/ll2ctypes.py
pypy/branch/jitffi/pypy/rpython/lltypesystem/rbuilder.py
pypy/branch/jitffi/pypy/rpython/lltypesystem/rclass.py
pypy/branch/jitffi/pypy/rpython/module/ll_time.py
pypy/branch/jitffi/pypy/rpython/rbuilder.py
pypy/branch/jitffi/pypy/rpython/test/test_rbuilder.py
pypy/branch/jitffi/pypy/rpython/test/test_rclass.py
pypy/branch/jitffi/pypy/translator/c/gcc/test/elf/track5.s
pypy/branch/jitffi/pypy/translator/c/gcc/trackgcroot.py
pypy/branch/jitffi/pypy/translator/c/src/signals.h
Log:
merge from trunk: svn merge svn+ssh://codespeak.net/svn/pypy/trunk -r77720:78032
Modified: pypy/branch/jitffi/lib-python/modified-2.5.2/ctypes/__init__.py
==============================================================================
--- pypy/branch/jitffi/lib-python/modified-2.5.2/ctypes/__init__.py (original)
+++ pypy/branch/jitffi/lib-python/modified-2.5.2/ctypes/__init__.py Mon Oct 18 11:22:20 2010
@@ -345,9 +345,11 @@
_restype_ = c_int # default, can be overridden in instances
def __init__(self, name, mode=DEFAULT_MODE, handle=None):
+ import _ffi
self._name = name
if handle is None:
- self._handle = _dlopen(self._name, mode)
+ #self._handle = _dlopen(self._name, mode)
+ self._handle = _ffi.CDLL(name)
else:
self._handle = handle
Modified: pypy/branch/jitffi/lib_pypy/_ctypes/function.py
==============================================================================
--- pypy/branch/jitffi/lib_pypy/_ctypes/function.py (original)
+++ pypy/branch/jitffi/lib_pypy/_ctypes/function.py Mon Oct 18 11:22:20 2010
@@ -110,7 +110,8 @@
self.dll = ctypes.CDLL(self.dll)
# we need to check dll anyway
ptr = self._getfuncptr([], ctypes.c_int)
- self._buffer = ptr.byptr()
+ #self._buffer = ptr.byptr()
+ self._buffer = None
elif (sys.platform == 'win32' and
len(args) >= 2 and isinstance(args[0], (int, long))):
@@ -163,12 +164,25 @@
if argtypes is None:
argtypes = self._guess_argtypes(args)
- argtypes, argsandobjs = self._wrap_args(argtypes, args)
+
+ # XXX
+ #argtypes, argsandobjs = self._wrap_args(argtypes, args)
restype = self._restype_
funcptr = self._getfuncptr(argtypes, restype, thisarg)
- resbuffer = funcptr(*[arg._buffer for _, arg in argsandobjs])
- return self._build_result(restype, resbuffer, argtypes, argsandobjs)
+ return funcptr(*args)
+ #resbuffer = funcptr(*[arg._buffer for _, arg in argsandobjs])
+ #return self._build_result(restype, resbuffer, argtypes, argsandobjs)
+
+ def _shape_to_ffi_type(self, shape):
+ from _ffi import types
+ if shape == 'l':
+ return types.slong
+ elif shape == 'd':
+ return types.double
+ else:
+ print 'unknown shape %s' % shape
+ import pdb;pdb.set_trace()
def _getfuncptr(self, argtypes, restype, thisarg=None):
if self._ptr is not None and argtypes is self._argtypes_:
@@ -194,7 +208,11 @@
cdll = self.dll._handle
try:
- return cdll.ptr(self.name, argshapes, resshape, self._flags_)
+ #return cdll.ptr(self.name, argshapes, resshape, self._flags_)
+ ffi_argtypes = [self._shape_to_ffi_type(shape) for shape in argshapes]
+ ffi_restype = self._shape_to_ffi_type(resshape)
+ self._ptr = cdll.getfunc(self.name, ffi_argtypes, ffi_restype)
+ return self._ptr
except AttributeError:
if self._flags_ & _rawffi.FUNCFLAG_CDECL:
raise
Modified: pypy/branch/jitffi/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/jitffi/pypy/config/pypyoption.py (original)
+++ pypy/branch/jitffi/pypy/config/pypyoption.py Mon Oct 18 11:22:20 2010
@@ -198,6 +198,9 @@
BoolOption("withstrslice", "use strings optimized for slicing",
default=False),
+ BoolOption("withstrbuf", "use strings optimized for addition (ver 2)",
+ default=False),
+
BoolOption("withprebuiltchar",
"use prebuilt single-character string objects",
default=False),
@@ -210,7 +213,8 @@
BoolOption("withrope", "use ropes as the string implementation",
default=False,
requires=[("objspace.std.withstrslice", False),
- ("objspace.std.withstrjoin", False)],
+ ("objspace.std.withstrjoin", False),
+ ("objspace.std.withstrbuf", False)],
suggests=[("objspace.std.withprebuiltchar", True),
("objspace.std.sharesmallstr", True)]),
@@ -238,6 +242,16 @@
default=False,
requires=[("objspace.std.withshadowtracking", False)]),
+ BoolOption("withmapdict",
+ "make instances really small but slow without the JIT",
+ default=False,
+ requires=[("objspace.std.withshadowtracking", False),
+ ("objspace.std.withinlineddict", False),
+ ("objspace.std.withsharingdict", False),
+ ("objspace.std.getattributeshortcut", True),
+ ("objspace.std.withtypeversion", True),
+ ]),
+
BoolOption("withrangelist",
"enable special range list implementation that does not "
"actually create the full list until the resulting "
@@ -343,7 +357,7 @@
config.objspace.std.suggest(withprebuiltint=True)
config.objspace.std.suggest(withrangelist=True)
config.objspace.std.suggest(withprebuiltchar=True)
- config.objspace.std.suggest(withinlineddict=True)
+ config.objspace.std.suggest(withmapdict=True)
config.objspace.std.suggest(withstrslice=True)
config.objspace.std.suggest(withstrjoin=True)
# xxx other options? ropes maybe?
@@ -359,6 +373,7 @@
# extra optimizations with the JIT
if level == 'jit':
config.objspace.std.suggest(withcelldict=True)
+ #config.objspace.std.suggest(withmapdict=True)
def enable_allworkingmodules(config):
Modified: pypy/branch/jitffi/pypy/interpreter/baseobjspace.py
==============================================================================
--- pypy/branch/jitffi/pypy/interpreter/baseobjspace.py (original)
+++ pypy/branch/jitffi/pypy/interpreter/baseobjspace.py Mon Oct 18 11:22:20 2010
@@ -168,6 +168,20 @@
def _call_builtin_destructor(self):
pass # method overridden in typedef.py
+ # hooks that the mapdict implementations needs:
+ def _get_mapdict_map(self):
+ return None
+ def _set_mapdict_map(self, map):
+ raise NotImplementedError
+ def _mapdict_read_storage(self, index):
+ raise NotImplementedError
+ def _mapdict_write_storage(self, index, value):
+ raise NotImplementedError
+ def _mapdict_storage_length(self):
+ raise NotImplementedError
+ def _set_mapdict_storage_and_map(self, storage, map):
+ raise NotImplementedError
+
class Wrappable(W_Root):
"""A subclass of Wrappable is an internal, interpreter-level class
Modified: pypy/branch/jitffi/pypy/interpreter/pycode.py
==============================================================================
--- pypy/branch/jitffi/pypy/interpreter/pycode.py (original)
+++ pypy/branch/jitffi/pypy/interpreter/pycode.py Mon Oct 18 11:22:20 2010
@@ -117,6 +117,10 @@
self._compute_flatcall()
+ if self.space.config.objspace.std.withmapdict:
+ from pypy.objspace.std.mapdict import init_mapdict_cache
+ init_mapdict_cache(self)
+
def _freeze_(self):
if (self.magic == cpython_magic and
'__pypy__' not in sys.builtin_module_names):
Modified: pypy/branch/jitffi/pypy/interpreter/pyopcode.py
==============================================================================
--- pypy/branch/jitffi/pypy/interpreter/pyopcode.py (original)
+++ pypy/branch/jitffi/pypy/interpreter/pyopcode.py Mon Oct 18 11:22:20 2010
@@ -710,9 +710,14 @@
def LOAD_ATTR(self, nameindex, next_instr):
"obj.attributename"
- w_attributename = self.getname_w(nameindex)
w_obj = self.popvalue()
- w_value = self.space.getattr(w_obj, w_attributename)
+ if (self.space.config.objspace.std.withmapdict
+ and not jit.we_are_jitted()):
+ from pypy.objspace.std.mapdict import LOAD_ATTR_caching
+ w_value = LOAD_ATTR_caching(self.getcode(), w_obj, nameindex)
+ else:
+ w_attributename = self.getname_w(nameindex)
+ w_value = self.space.getattr(w_obj, w_attributename)
self.pushvalue(w_value)
LOAD_ATTR._always_inline_ = True
Modified: pypy/branch/jitffi/pypy/interpreter/pyparser/genpytokenize.py
==============================================================================
--- pypy/branch/jitffi/pypy/interpreter/pyparser/genpytokenize.py (original)
+++ pypy/branch/jitffi/pypy/interpreter/pyparser/genpytokenize.py Mon Oct 18 11:22:20 2010
@@ -176,6 +176,7 @@
pseudoToken = chain(states,
makeWhitespace(),
group(states,
+ newArcPair(states, EMPTY),
pseudoExtras, number, funny, contStr, name))
dfaStates, dfaAccepts = nfaToDfa(states, *pseudoToken)
return DFA(dfaStates, dfaAccepts)
Modified: pypy/branch/jitffi/pypy/interpreter/pyparser/pytokenize.py
==============================================================================
--- pypy/branch/jitffi/pypy/interpreter/pyparser/pytokenize.py (original)
+++ pypy/branch/jitffi/pypy/interpreter/pyparser/pytokenize.py Mon Oct 18 11:22:20 2010
@@ -23,7 +23,7 @@
# ______________________________________________________________________
# Automatically generated DFA's
-accepts = [False, True, True, True, True, True, True, True,
+accepts = [True, True, True, True, True, True, True, True,
True, True, False, True, True, True, False, False,
False, False, True, True, True, False, True,
False, True, False, True, False, False, True,
Modified: pypy/branch/jitffi/pypy/interpreter/typedef.py
==============================================================================
--- pypy/branch/jitffi/pypy/interpreter/typedef.py (original)
+++ pypy/branch/jitffi/pypy/interpreter/typedef.py Mon Oct 18 11:22:20 2010
@@ -133,6 +133,13 @@
typedef = cls.typedef
if wants_dict and typedef.hasdict:
wants_dict = False
+ if config.objspace.std.withmapdict and not typedef.hasdict:
+ # mapdict only works if the type does not already have a dict
+ if wants_del:
+ parentcls = get_unique_interplevel_subclass(config, cls, True, True,
+ False, True)
+ return _usersubclswithfeature(config, parentcls, "del")
+ return _usersubclswithfeature(config, cls, "user", "dict", "weakref", "slots")
# Forest of if's - see the comment above.
if wants_del:
if wants_dict:
@@ -186,10 +193,20 @@
def add(Proto):
for key, value in Proto.__dict__.items():
- if not key.startswith('__') or key == '__del__':
+ if (not key.startswith('__') and not key.startswith('_mixin_')
+ or key == '__del__'):
+ if hasattr(value, "func_name"):
+ value = func_with_new_name(value, value.func_name)
body[key] = value
+ if (config.objspace.std.withmapdict and "dict" in features):
+ from pypy.objspace.std.mapdict import BaseMapdictObject, ObjectMixin
+ add(BaseMapdictObject)
+ add(ObjectMixin)
+ features = ()
+
if "user" in features: # generic feature needed by all subcls
+
class Proto(object):
user_overridden_class = True
@@ -255,6 +272,9 @@
wantdict = False
if wantdict:
+ base_user_setup = supercls.user_setup.im_func
+ if "user_setup" in body:
+ base_user_setup = body["user_setup"]
class Proto(object):
def getdict(self):
return self.w__dict__
@@ -263,11 +283,9 @@
self.w__dict__ = check_new_dictionary(space, w_dict)
def user_setup(self, space, w_subtype):
- self.space = space
- self.w__class__ = w_subtype
self.w__dict__ = space.newdict(
instance=True, classofinstance=w_subtype)
- self.user_setup_slots(w_subtype.nslots)
+ base_user_setup(self, space, w_subtype)
def setclass(self, space, w_subtype):
# only used by descr_set___class__
Modified: pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/branch/jitffi/pypy/jit/backend/test/runner_test.py Mon Oct 18 11:22:20 2010
@@ -859,6 +859,23 @@
length_box], 'void')
assert self.look_string(r_box) == "!??cdef?!"
+ def test_copyunicodecontent(self):
+ s_box = self.alloc_unicode(u"abcdef")
+ for s_box in [s_box, s_box.constbox()]:
+ for srcstart_box in [BoxInt(2), ConstInt(2)]:
+ for dststart_box in [BoxInt(3), ConstInt(3)]:
+ for length_box in [BoxInt(4), ConstInt(4)]:
+ for r_box_is_const in [False, True]:
+ r_box = self.alloc_unicode(u"!???????!")
+ if r_box_is_const:
+ r_box = r_box.constbox()
+ self.execute_operation(rop.COPYUNICODECONTENT,
+ [s_box, r_box,
+ srcstart_box,
+ dststart_box,
+ length_box], 'void')
+ assert self.look_unicode(r_box) == u"!??cdef?!"
+
def test_do_unicode_basic(self):
u = self.cpu.bh_newunicode(5)
self.cpu.bh_unicodesetitem(u, 4, 123)
@@ -1253,6 +1270,10 @@
u_box = BoxPtr(lltype.cast_opaque_ptr(llmemory.GCREF, u))
return u_box
+ def look_unicode(self, unicode_box):
+ u = unicode_box.getref(lltype.Ptr(rstr.UNICODE))
+ return u''.join(u.chars)
+
def test_casts(self):
py.test.skip("xxx fix or kill")
Modified: pypy/branch/jitffi/pypy/jit/backend/test/test_ll_random.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/backend/test/test_ll_random.py (original)
+++ pypy/branch/jitffi/pypy/jit/backend/test/test_ll_random.py Mon Oct 18 11:22:20 2010
@@ -599,7 +599,7 @@
OPERATIONS.append(StrLenOperation(rop.STRLEN))
OPERATIONS.append(UnicodeLenOperation(rop.UNICODELEN))
OPERATIONS.append(CopyStrContentOperation(rop.COPYSTRCONTENT))
- #OPERATIONS.append(CopyUnicodeContentOperation(rop.COPYUNICODECONTENT))
+ OPERATIONS.append(CopyUnicodeContentOperation(rop.COPYUNICODECONTENT))
for i in range(2):
OPERATIONS.append(GuardClassOperation(rop.GUARD_CLASS))
Modified: pypy/branch/jitffi/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/branch/jitffi/pypy/jit/backend/x86/regalloc.py Mon Oct 18 11:22:20 2010
@@ -778,15 +778,11 @@
loc = self.loc(op.getarg(0))
return self._call(op, [loc])
# boehm GC (XXX kill the following code at some point)
- ofs_items, itemsize, ofs = symbolic.get_array_token(rstr.UNICODE, self.translate_support_code)
- if itemsize == 4:
- return self._malloc_varsize(ofs_items, ofs, 2, op.getarg(0),
- op.result)
- elif itemsize == 2:
- return self._malloc_varsize(ofs_items, ofs, 1, op.getarg(0),
- op.result)
- else:
- assert False, itemsize
+ ofs_items, _, ofs = symbolic.get_array_token(rstr.UNICODE,
+ self.translate_support_code)
+ scale = self._get_unicode_item_scale()
+ return self._malloc_varsize(ofs_items, ofs, scale, op.getarg(0),
+ op.result)
def _malloc_varsize(self, ofs_items, ofs_length, scale, v, res_v):
# XXX kill this function at some point
@@ -959,6 +955,12 @@
consider_unicodegetitem = consider_strgetitem
def consider_copystrcontent(self, op):
+ self._consider_copystrcontent(op, is_unicode=False)
+
+ def consider_copyunicodecontent(self, op):
+ self._consider_copystrcontent(op, is_unicode=True)
+
+ def _consider_copystrcontent(self, op, is_unicode):
# compute the source address
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(args[0], args)
@@ -970,7 +972,8 @@
srcaddr_box = TempBox()
forbidden_vars = [args[1], args[3], args[4], srcaddr_box]
srcaddr_loc = self.rm.force_allocate_reg(srcaddr_box, forbidden_vars)
- self._gen_address_inside_string(base_loc, ofs_loc, srcaddr_loc)
+ self._gen_address_inside_string(base_loc, ofs_loc, srcaddr_loc,
+ is_unicode=is_unicode)
# compute the destination address
base_loc = self.rm.make_sure_var_in_reg(args[1], forbidden_vars)
ofs_loc = self.rm.make_sure_var_in_reg(args[3], forbidden_vars)
@@ -980,25 +983,57 @@
forbidden_vars = [args[4], srcaddr_box]
dstaddr_box = TempBox()
dstaddr_loc = self.rm.force_allocate_reg(dstaddr_box, forbidden_vars)
- self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc)
+ self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc,
+ is_unicode=is_unicode)
+ # compute the length in bytes
+ length_box = args[4]
+ length_loc = self.loc(length_box)
+ if is_unicode:
+ self.rm.possibly_free_var(length_box)
+ forbidden_vars = [srcaddr_box, dstaddr_box]
+ bytes_box = TempBox()
+ bytes_loc = self.rm.force_allocate_reg(bytes_box, forbidden_vars)
+ scale = self._get_unicode_item_scale()
+ if not (isinstance(length_loc, ImmedLoc) or
+ isinstance(length_loc, RegLoc)):
+ self.assembler.mov(length_loc, bytes_loc)
+ length_loc = bytes_loc
+ self.assembler.load_effective_addr(length_loc, 0, scale, bytes_loc)
+ length_box = bytes_box
+ length_loc = bytes_loc
# call memcpy()
- length_loc = self.loc(args[4])
self.rm.before_call()
self.xrm.before_call()
self.assembler._emit_call(imm(self.assembler.memcpy_addr),
[dstaddr_loc, srcaddr_loc, length_loc])
- self.rm.possibly_free_var(args[4])
+ self.rm.possibly_free_var(length_box)
self.rm.possibly_free_var(dstaddr_box)
self.rm.possibly_free_var(srcaddr_box)
- def _gen_address_inside_string(self, baseloc, ofsloc, resloc):
+ def _gen_address_inside_string(self, baseloc, ofsloc, resloc, is_unicode):
cpu = self.assembler.cpu
- ofs_items, itemsize, _ = symbolic.get_array_token(rstr.STR,
+ if is_unicode:
+ ofs_items, _, _ = symbolic.get_array_token(rstr.UNICODE,
+ self.translate_support_code)
+ scale = self._get_unicode_item_scale()
+ else:
+ ofs_items, itemsize, _ = symbolic.get_array_token(rstr.STR,
self.translate_support_code)
- assert itemsize == 1
- self.assembler.load_effective_addr(ofsloc, ofs_items, 0,
+ assert itemsize == 1
+ scale = 0
+ self.assembler.load_effective_addr(ofsloc, ofs_items, scale,
resloc, baseloc)
+ def _get_unicode_item_scale(self):
+ _, itemsize, _ = symbolic.get_array_token(rstr.UNICODE,
+ self.translate_support_code)
+ if itemsize == 4:
+ return 2
+ elif itemsize == 2:
+ return 1
+ else:
+ raise AssertionError("bad unicode item size")
+
def consider_jump(self, op):
assembler = self.assembler
assert self.jump_target_descr is None
Modified: pypy/branch/jitffi/pypy/jit/backend/x86/test/test_runner.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/backend/x86/test/test_runner.py (original)
+++ pypy/branch/jitffi/pypy/jit/backend/x86/test/test_runner.py Mon Oct 18 11:22:20 2010
@@ -506,8 +506,8 @@
self.cpu.execute_token(ops.token)
# check debugging info
name, struct = self.cpu.assembler.loop_run_counters[0]
- assert name == 'xyz'
+ assert name == 0 # 'xyz'
assert struct.i == 10
self.cpu.finish_once()
lines = py.path.local(self.logfile + ".count").readlines()
- assert lines[0] == '10 xyz\n'
+ assert lines[0] == '0:10\n' # '10 xyz\n'
Modified: pypy/branch/jitffi/pypy/jit/backend/x86/test/test_string.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/backend/x86/test/test_string.py (original)
+++ pypy/branch/jitffi/pypy/jit/backend/x86/test/test_string.py Mon Oct 18 11:22:20 2010
@@ -2,8 +2,12 @@
from pypy.jit.metainterp.test import test_string
from pypy.jit.backend.x86.test.test_basic import Jit386Mixin
-class TestString(Jit386Mixin, test_string.StringTests):
+class TestString(Jit386Mixin, test_string.TestLLtype):
# for the individual tests see
# ====> ../../../metainterp/test/test_string.py
- CALL = 'call'
- CALL_PURE = 'call_pure'
+ pass
+
+class TestUnicode(Jit386Mixin, test_string.TestLLtypeUnicode):
+ # for the individual tests see
+ # ====> ../../../metainterp/test/test_string.py
+ pass
Modified: pypy/branch/jitffi/pypy/jit/codewriter/effectinfo.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/codewriter/effectinfo.py (original)
+++ pypy/branch/jitffi/pypy/jit/codewriter/effectinfo.py Mon Oct 18 11:22:20 2010
@@ -18,23 +18,34 @@
# the 'oopspecindex' field is one of the following values:
OS_NONE = 0 # normal case, no oopspec
OS_ARRAYCOPY = 1 # "list.ll_arraycopy"
- OS_STR_CONCAT = 2 # "stroruni.concat"
- OS_UNI_CONCAT = 3 # "stroruni.concat"
- OS_STR_SLICE = 4 # "stroruni.slice"
- OS_UNI_SLICE = 5 # "stroruni.slice"
- OS_STR_EQUAL = 6 # "stroruni.equal"
- OS_UNI_EQUAL = 7 # "stroruni.equal"
- OS_STREQ_SLICE_CHECKNULL = 8 # s2!=NULL and s1[x:x+length]==s2
- OS_STREQ_SLICE_NONNULL = 9 # s1[x:x+length]==s2 (assert s2!=NULL)
- OS_STREQ_SLICE_CHAR = 10 # s1[x:x+length]==char
- OS_STREQ_NONNULL = 11 # s1 == s2 (assert s1!=NULL,s2!=NULL)
- OS_STREQ_NONNULL_CHAR = 12 # s1 == char (assert s1!=NULL)
- OS_STREQ_CHECKNULL_CHAR = 13 # s1!=NULL and s1==char
- OS_STREQ_LENGTHOK = 14 # s1 == s2 (assert len(s1)==len(s2))
+ OS_STR2UNICODE = 2 # "str.str2unicode"
OS_LIBFFI_PREPARE = 15
OS_LIBFFI_PUSH_ARG = 16
OS_LIBFFI_CALL = 17
+ OS_STR_CONCAT = 22 # "stroruni.concat"
+ OS_STR_SLICE = 23 # "stroruni.slice"
+ OS_STR_EQUAL = 24 # "stroruni.equal"
+ OS_STREQ_SLICE_CHECKNULL = 25 # s2!=NULL and s1[x:x+length]==s2
+ OS_STREQ_SLICE_NONNULL = 26 # s1[x:x+length]==s2 (assert s2!=NULL)
+ OS_STREQ_SLICE_CHAR = 27 # s1[x:x+length]==char
+ OS_STREQ_NONNULL = 28 # s1 == s2 (assert s1!=NULL,s2!=NULL)
+ OS_STREQ_NONNULL_CHAR = 29 # s1 == char (assert s1!=NULL)
+ OS_STREQ_CHECKNULL_CHAR = 30 # s1!=NULL and s1==char
+ OS_STREQ_LENGTHOK = 31 # s1 == s2 (assert len(s1)==len(s2))
+
+ OS_UNI_CONCAT = 42 #
+ OS_UNI_SLICE = 43 #
+ OS_UNI_EQUAL = 44 #
+ OS_UNIEQ_SLICE_CHECKNULL = 45 #
+ OS_UNIEQ_SLICE_NONNULL = 46 #
+ OS_UNIEQ_SLICE_CHAR = 47 #
+ OS_UNIEQ_NONNULL = 48 # the same for unicode
+ OS_UNIEQ_NONNULL_CHAR = 49 # (must be the same amount as for
+ OS_UNIEQ_CHECKNULL_CHAR = 50 # STR, in the same order)
+ OS_UNIEQ_LENGTHOK = 51 #
+ _OS_offset_uni = OS_UNI_CONCAT - OS_STR_CONCAT
+
def __new__(cls, readonly_descrs_fields,
write_descrs_fields, write_descrs_arrays,
extraeffect=EF_CAN_RAISE,
Modified: pypy/branch/jitffi/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/branch/jitffi/pypy/jit/codewriter/jtransform.py Mon Oct 18 11:22:20 2010
@@ -316,6 +316,8 @@
prepare = self._handle_list_call
elif oopspec_name.startswith('stroruni.'):
prepare = self._handle_stroruni_call
+ elif oopspec_name == 'str.str2unicode':
+ prepare = self._handle_str2unicode_call
elif oopspec_name.startswith('virtual_ref'):
prepare = self._handle_virtual_ref_call
elif oopspec_name.startswith('libffi_'):
@@ -1059,28 +1061,32 @@
[c_func] + [varoftype(T) for T in argtypes],
varoftype(resulttype))
calldescr = self.callcontrol.getcalldescr(op, oopspecindex)
- func = heaptracker.adr2int(
- llmemory.cast_ptr_to_adr(c_func.value))
+ if isinstance(c_func.value, str): # in tests only
+ func = c_func.value
+ else:
+ func = heaptracker.adr2int(
+ llmemory.cast_ptr_to_adr(c_func.value))
_callinfo_for_oopspec[oopspecindex] = calldescr, func
def _handle_stroruni_call(self, op, oopspec_name, args):
- if args[0].concretetype.TO == rstr.STR:
+ SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE)
+ if SoU.TO == rstr.STR:
dict = {"stroruni.concat": EffectInfo.OS_STR_CONCAT,
"stroruni.slice": EffectInfo.OS_STR_SLICE,
"stroruni.equal": EffectInfo.OS_STR_EQUAL,
}
- elif args[0].concretetype.TO == rstr.UNICODE:
+ CHR = lltype.Char
+ elif SoU.TO == rstr.UNICODE:
dict = {"stroruni.concat": EffectInfo.OS_UNI_CONCAT,
"stroruni.slice": EffectInfo.OS_UNI_SLICE,
"stroruni.equal": EffectInfo.OS_UNI_EQUAL,
}
+ CHR = lltype.UniChar
else:
assert 0, "args[0].concretetype must be STR or UNICODE"
#
if oopspec_name == "stroruni.equal":
- SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE)
for otherindex, othername, argtypes, resulttype in [
-
(EffectInfo.OS_STREQ_SLICE_CHECKNULL,
"str.eq_slice_checknull",
[SoU, lltype.Signed, lltype.Signed, SoU],
@@ -1091,7 +1097,7 @@
lltype.Signed),
(EffectInfo.OS_STREQ_SLICE_CHAR,
"str.eq_slice_char",
- [SoU, lltype.Signed, lltype.Signed, lltype.Char],
+ [SoU, lltype.Signed, lltype.Signed, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_NONNULL,
"str.eq_nonnull",
@@ -1099,22 +1105,27 @@
lltype.Signed),
(EffectInfo.OS_STREQ_NONNULL_CHAR,
"str.eq_nonnull_char",
- [SoU, lltype.Char],
+ [SoU, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_CHECKNULL_CHAR,
"str.eq_checknull_char",
- [SoU, lltype.Char],
+ [SoU, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_LENGTHOK,
"str.eq_lengthok",
[SoU, SoU],
lltype.Signed),
]:
+ if args[0].concretetype.TO == rstr.UNICODE:
+ otherindex += EffectInfo._OS_offset_uni
self._register_extra_helper(otherindex, othername,
argtypes, resulttype)
#
return self._handle_oopspec_call(op, args, dict[oopspec_name])
+ def _handle_str2unicode_call(self, op, oopspec_name, args):
+ return self._handle_oopspec_call(op, args, EffectInfo.OS_STR2UNICODE)
+
# ----------
# VirtualRefs.
Modified: pypy/branch/jitffi/pypy/jit/codewriter/test/test_jtransform.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/codewriter/test/test_jtransform.py (original)
+++ pypy/branch/jitffi/pypy/jit/codewriter/test/test_jtransform.py Mon Oct 18 11:22:20 2010
@@ -77,7 +77,32 @@
class FakeBuiltinCallControl:
def guess_call_kind(self, op):
return 'builtin'
- def getcalldescr(self, op, oopspecindex):
+ def getcalldescr(self, op, oopspecindex=None):
+ assert oopspecindex is not None # in this test
+ EI = effectinfo.EffectInfo
+ if oopspecindex != EI.OS_ARRAYCOPY:
+ PSTR = lltype.Ptr(rstr.STR)
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ INT = lltype.Signed
+ UNICHAR = lltype.UniChar
+ argtypes = {
+ EI.OS_STR2UNICODE:([PSTR], PUNICODE),
+ EI.OS_STR_CONCAT: ([PSTR, PSTR], PSTR),
+ EI.OS_STR_SLICE: ([PSTR, INT, INT], PSTR),
+ EI.OS_UNI_CONCAT: ([PUNICODE, PUNICODE], PUNICODE),
+ EI.OS_UNI_SLICE: ([PUNICODE, INT, INT], PUNICODE),
+ EI.OS_UNI_EQUAL: ([PUNICODE, PUNICODE], lltype.Bool),
+ EI.OS_UNIEQ_SLICE_CHECKNULL:([PUNICODE, INT, INT, PUNICODE], INT),
+ EI.OS_UNIEQ_SLICE_NONNULL: ([PUNICODE, INT, INT, PUNICODE], INT),
+ EI.OS_UNIEQ_SLICE_CHAR: ([PUNICODE, INT, INT, UNICHAR], INT),
+ EI.OS_UNIEQ_NONNULL: ([PUNICODE, PUNICODE], INT),
+ EI.OS_UNIEQ_NONNULL_CHAR: ([PUNICODE, UNICHAR], INT),
+ EI.OS_UNIEQ_CHECKNULL_CHAR: ([PUNICODE, UNICHAR], INT),
+ EI.OS_UNIEQ_LENGTHOK: ([PUNICODE, PUNICODE], INT),
+ }
+ argtypes = argtypes[oopspecindex]
+ assert argtypes[0] == [v.concretetype for v in op.args[1:]]
+ assert argtypes[1] == op.result.concretetype
return 'calldescr-%d' % oopspecindex
def calldescr_canraise(self, calldescr):
return False
@@ -766,6 +791,46 @@
assert op1.args[3] == ListOfKind('ref', [v1])
assert op1.result == v4
+def test_str2unicode():
+ # test that the oopspec is present and correctly transformed
+ PSTR = lltype.Ptr(rstr.STR)
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ FUNC = lltype.FuncType([PSTR], PUNICODE)
+ func = lltype.functionptr(FUNC, 'll_str2unicode',
+ _callable=rstr.LLHelpers.ll_str2unicode)
+ v1 = varoftype(PSTR)
+ v2 = varoftype(PUNICODE)
+ op = SpaceOperation('direct_call', [const(func), v1], v2)
+ tr = Transformer(FakeCPU(), FakeBuiltinCallControl())
+ op1 = tr.rewrite_operation(op)
+ assert op1.opname == 'residual_call_r_r'
+ assert op1.args[0].value == func
+ assert op1.args[1] == 'calldescr-%d' % effectinfo.EffectInfo.OS_STR2UNICODE
+ assert op1.args[2] == ListOfKind('ref', [v1])
+ assert op1.result == v2
+
+def test_unicode_eq_checknull_char():
+ # test that the oopspec is present and correctly transformed
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ FUNC = lltype.FuncType([PUNICODE, PUNICODE], lltype.Bool)
+ func = lltype.functionptr(FUNC, 'll_streq',
+ _callable=rstr.LLHelpers.ll_streq)
+ v1 = varoftype(PUNICODE)
+ v2 = varoftype(PUNICODE)
+ v3 = varoftype(lltype.Bool)
+ op = SpaceOperation('direct_call', [const(func), v1, v2], v3)
+ tr = Transformer(FakeCPU(), FakeBuiltinCallControl())
+ op1 = tr.rewrite_operation(op)
+ assert op1.opname == 'residual_call_r_i'
+ assert op1.args[0].value == func
+ assert op1.args[1] == 'calldescr-%d' % effectinfo.EffectInfo.OS_UNI_EQUAL
+ assert op1.args[2] == ListOfKind('ref', [v1, v2])
+ assert op1.result == v3
+ # test that the OS_UNIEQ_* functions are registered
+ cifo = effectinfo._callinfo_for_oopspec
+ assert effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL in cifo
+ assert effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR in cifo
+
def test_list_ll_arraycopy():
from pypy.rlib.rgc import ll_arraycopy
LIST = lltype.GcArray(lltype.Signed)
Modified: pypy/branch/jitffi/pypy/jit/metainterp/history.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/history.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/history.py Mon Oct 18 11:22:20 2010
@@ -698,6 +698,21 @@
return result
_const_ptr_for_string = {}
+def get_const_ptr_for_unicode(s):
+ from pypy.rpython.annlowlevel import llunicode
+ if not we_are_translated():
+ try:
+ return _const_ptr_for_unicode[s]
+ except KeyError:
+ pass
+ if isinstance(s, str):
+ s = unicode(s)
+ result = ConstPtr(lltype.cast_opaque_ptr(llmemory.GCREF, llunicode(s)))
+ if not we_are_translated():
+ _const_ptr_for_unicode[s] = result
+ return result
+_const_ptr_for_unicode = {}
+
# ____________________________________________________________
# The TreeLoop class contains a loop or a generalized loop, i.e. a tree
Modified: pypy/branch/jitffi/pypy/jit/metainterp/optimize_nopspec.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/optimize_nopspec.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/optimize_nopspec.py Mon Oct 18 11:22:20 2010
@@ -14,6 +14,9 @@
def _optimize_loop(metainterp_sd, old_loop_tokens, loop):
cpu = metainterp_sd.cpu
metainterp_sd.logger_noopt.log_loop(loop.inputargs, loop.operations)
+ # XXX the following lines are probably still needed, to discard invalid
+ # loops. bit silly to run a full perfect specialization and throw the
+ # result away.
finder = PerfectSpecializationFinder(cpu)
finder.find_nodes_loop(loop, False)
if old_loop_tokens:
@@ -31,6 +34,7 @@
def _optimize_bridge(metainterp_sd, old_loop_tokens, bridge):
cpu = metainterp_sd.cpu
metainterp_sd.logger_noopt.log_loop(bridge.inputargs, bridge.operations)
+ # XXX same comment as above applies
finder = BridgeSpecializationFinder(cpu)
finder.find_nodes_bridge(bridge)
if old_loop_tokens:
Modified: pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/intbounds.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/intbounds.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/intbounds.py Mon Oct 18 11:22:20 2010
@@ -191,6 +191,7 @@
v1.intbound.make_ge(IntLowerBound(0))
optimize_STRLEN = optimize_ARRAYLEN_GC
+ optimize_UNICODELEN = optimize_ARRAYLEN_GC
def make_int_lt(self, box1, box2):
v1 = self.getvalue(box1)
Modified: pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/rewrite.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/rewrite.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/rewrite.py Mon Oct 18 11:22:20 2010
@@ -3,6 +3,7 @@
from pypy.jit.metainterp.history import ConstInt
from pypy.jit.metainterp.optimizeutil import _findall
from pypy.jit.metainterp.resoperation import rop, ResOperation
+from pypy.jit.codewriter.effectinfo import EffectInfo
class OptRewrite(Optimization):
"""Rewrite operations into equivalent, cheaper operations.
@@ -326,8 +327,37 @@
## return
## self.emit_operation(op)
-optimize_ops = _findall(OptRewrite, 'optimize_')
-
+ def optimize_CALL(self, op):
+ # dispatch based on 'oopspecindex' to a method that handles
+ # specifically the given oopspec call. For non-oopspec calls,
+ # oopspecindex is just zero.
+ effectinfo = op.getdescr().get_extra_info()
+ if effectinfo is not None:
+ oopspecindex = effectinfo.oopspecindex
+ if oopspecindex == EffectInfo.OS_ARRAYCOPY:
+ if self._optimize_CALL_ARRAYCOPY(op):
+ return
+ self.emit_operation(op)
+ def _optimize_CALL_ARRAYCOPY(self, op):
+ source_value = self.getvalue(op.getarg(1))
+ dest_value = self.getvalue(op.getarg(2))
+ source_start_box = self.get_constant_box(op.getarg(3))
+ dest_start_box = self.get_constant_box(op.getarg(4))
+ length = self.get_constant_box(op.getarg(5))
+ if (source_value.is_virtual() and source_start_box and dest_start_box
+ and length and dest_value.is_virtual()):
+ # XXX optimize the case where dest value is not virtual,
+ # but we still can avoid a mess
+ source_start = source_start_box.getint()
+ dest_start = dest_start_box.getint()
+ for index in range(length.getint()):
+ val = source_value.getitem(index + source_start)
+ dest_value.setitem(index + dest_start, val)
+ return True
+ if length and length.getint() == 0:
+ return True # 0-length arraycopy
+ return False
+
+optimize_ops = _findall(OptRewrite, 'optimize_')
-
Modified: pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/string.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/string.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/string.py Mon Oct 18 11:22:20 2010
@@ -3,6 +3,7 @@
from pypy.jit.metainterp.history import Box, BoxInt, BoxPtr
from pypy.jit.metainterp.history import Const, ConstInt, ConstPtr
from pypy.jit.metainterp.history import get_const_ptr_for_string
+from pypy.jit.metainterp.history import get_const_ptr_for_unicode
from pypy.jit.metainterp.resoperation import rop, ResOperation
from pypy.jit.metainterp.optimizeopt import optimizer, virtualize
from pypy.jit.metainterp.optimizeopt.optimizer import CONST_0, CONST_1
@@ -11,56 +12,103 @@
from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec
from pypy.jit.codewriter import heaptracker
from pypy.rlib.unroll import unrolling_iterable
+from pypy.rlib.objectmodel import specialize
+
+
+class StrOrUnicode(object):
+ def __init__(self, LLTYPE, hlstr, emptystr, chr,
+ NEWSTR, STRLEN, STRGETITEM, STRSETITEM, COPYSTRCONTENT,
+ OS_offset):
+ self.LLTYPE = LLTYPE
+ self.hlstr = hlstr
+ self.emptystr = emptystr
+ self.chr = chr
+ self.NEWSTR = NEWSTR
+ self.STRLEN = STRLEN
+ self.STRGETITEM = STRGETITEM
+ self.STRSETITEM = STRSETITEM
+ self.COPYSTRCONTENT = COPYSTRCONTENT
+ self.OS_offset = OS_offset
+
+ def _freeze_(self):
+ return True
+
+mode_string = StrOrUnicode(rstr.STR, annlowlevel.hlstr, '', chr,
+ rop.NEWSTR, rop.STRLEN, rop.STRGETITEM,
+ rop.STRSETITEM, rop.COPYSTRCONTENT, 0)
+mode_unicode = StrOrUnicode(rstr.UNICODE, annlowlevel.hlunicode, u'', unichr,
+ rop.NEWUNICODE, rop.UNICODELEN, rop.UNICODEGETITEM,
+ rop.UNICODESETITEM, rop.COPYUNICODECONTENT,
+ EffectInfo._OS_offset_uni)
+
+# ____________________________________________________________
class __extend__(optimizer.OptValue):
"""New methods added to the base class OptValue for this file."""
- def getstrlen(self, newoperations):
- s = self.get_constant_string()
- if s is not None:
- return ConstInt(len(s))
+ def getstrlen(self, newoperations, mode):
+ if mode is mode_string:
+ s = self.get_constant_string_spec(mode_string)
+ if s is not None:
+ return ConstInt(len(s))
else:
- if newoperations is None:
- return None
- self.ensure_nonnull()
- box = self.force_box()
- lengthbox = BoxInt()
- newoperations.append(ResOperation(rop.STRLEN, [box], lengthbox))
- return lengthbox
+ s = self.get_constant_string_spec(mode_unicode)
+ if s is not None:
+ return ConstInt(len(s))
+ if newoperations is None:
+ return None
+ self.ensure_nonnull()
+ box = self.force_box()
+ lengthbox = BoxInt()
+ newoperations.append(ResOperation(mode.STRLEN, [box], lengthbox))
+ return lengthbox
- def get_constant_string(self):
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
if self.is_constant():
- s = self.box.getref(lltype.Ptr(rstr.STR))
- return annlowlevel.hlstr(s)
+ s = self.box.getref(lltype.Ptr(mode.LLTYPE))
+ return mode.hlstr(s)
else:
return None
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
# Copies the pointer-to-string 'self' into the target string
# given by 'targetbox', at the specified offset. Returns the offset
# at the end of the copy.
- lengthbox = self.getstrlen(newoperations)
+ lengthbox = self.getstrlen(newoperations, mode)
srcbox = self.force_box()
return copy_str_content(newoperations, srcbox, targetbox,
- CONST_0, offsetbox, lengthbox)
+ CONST_0, offsetbox, lengthbox, mode)
class VAbstractStringValue(virtualize.AbstractVirtualValue):
- _attrs_ = ()
+ _attrs_ = ('mode',)
+
+ def __init__(self, optimizer, keybox, source_op, mode):
+ virtualize.AbstractVirtualValue.__init__(self, optimizer, keybox,
+ source_op)
+ self.mode = mode
def _really_force(self):
- s = self.get_constant_string()
- if s is not None:
- c_s = get_const_ptr_for_string(s)
- self.make_constant(c_s)
- return
+ if self.mode is mode_string:
+ s = self.get_constant_string_spec(mode_string)
+ if s is not None:
+ c_s = get_const_ptr_for_string(s)
+ self.make_constant(c_s)
+ return
+ else:
+ s = self.get_constant_string_spec(mode_unicode)
+ if s is not None:
+ c_s = get_const_ptr_for_unicode(s)
+ self.make_constant(c_s)
+ return
assert self.source_op is not None
self.box = box = self.source_op.result
newoperations = self.optimizer.newoperations
- lengthbox = self.getstrlen(newoperations)
- newoperations.append(ResOperation(rop.NEWSTR, [lengthbox], box))
- self.string_copy_parts(newoperations, box, CONST_0)
+ lengthbox = self.getstrlen(newoperations, self.mode)
+ newoperations.append(ResOperation(self.mode.NEWSTR, [lengthbox], box))
+ self.string_copy_parts(newoperations, box, CONST_0, self.mode)
class VStringPlainValue(VAbstractStringValue):
@@ -74,7 +122,7 @@
assert 0 <= start <= stop <= len(longerlist)
self._chars = longerlist[start:stop]
- def getstrlen(self, _):
+ def getstrlen(self, _, mode):
if self._lengthbox is None:
self._lengthbox = ConstInt(len(self._chars))
return self._lengthbox
@@ -86,18 +134,21 @@
assert isinstance(charvalue, optimizer.OptValue)
self._chars[index] = charvalue
- def get_constant_string(self):
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
for c in self._chars:
if c is optimizer.CVAL_UNINITIALIZED_ZERO or not c.is_constant():
return None
- return ''.join([chr(c.box.getint()) for c in self._chars])
+ return mode.emptystr.join([mode.chr(c.box.getint())
+ for c in self._chars])
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
for i in range(len(self._chars)):
charbox = self._chars[i].force_box()
- newoperations.append(ResOperation(rop.STRSETITEM, [targetbox,
- offsetbox,
- charbox], None))
+ newoperations.append(ResOperation(mode.STRSETITEM, [targetbox,
+ offsetbox,
+ charbox],
+ None))
offsetbox = _int_add(newoperations, offsetbox, CONST_1)
return offsetbox
@@ -109,7 +160,7 @@
value.get_args_for_fail(modifier)
def _make_virtual(self, modifier):
- return modifier.make_vstrplain()
+ return modifier.make_vstrplain(self.mode is mode_unicode)
class VStringConcatValue(VAbstractStringValue):
@@ -120,23 +171,24 @@
self.right = right
self.lengthbox = lengthbox
- def getstrlen(self, _):
+ def getstrlen(self, _, mode):
return self.lengthbox
- def get_constant_string(self):
- s1 = self.left.get_constant_string()
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
+ s1 = self.left.get_constant_string_spec(mode)
if s1 is None:
return None
- s2 = self.right.get_constant_string()
+ s2 = self.right.get_constant_string_spec(mode)
if s2 is None:
return None
return s1 + s2
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
offsetbox = self.left.string_copy_parts(newoperations, targetbox,
- offsetbox)
+ offsetbox, mode)
offsetbox = self.right.string_copy_parts(newoperations, targetbox,
- offsetbox)
+ offsetbox, mode)
return offsetbox
def get_args_for_fail(self, modifier):
@@ -150,7 +202,7 @@
self.right.get_args_for_fail(modifier)
def _make_virtual(self, modifier):
- return modifier.make_vstrconcat()
+ return modifier.make_vstrconcat(self.mode is mode_unicode)
class VStringSliceValue(VAbstractStringValue):
@@ -162,12 +214,13 @@
self.vstart = vstart
self.vlength = vlength
- def getstrlen(self, _):
+ def getstrlen(self, _, mode):
return self.vlength.force_box()
- def get_constant_string(self):
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
if self.vstart.is_constant() and self.vlength.is_constant():
- s1 = self.vstr.get_constant_string()
+ s1 = self.vstr.get_constant_string_spec(mode)
if s1 is None:
return None
start = self.vstart.box.getint()
@@ -177,12 +230,12 @@
return s1[start : start + length]
return None
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
- lengthbox = self.getstrlen(newoperations)
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
+ lengthbox = self.getstrlen(newoperations, mode)
return copy_str_content(newoperations,
self.vstr.force_box(), targetbox,
self.vstart.force_box(), offsetbox,
- lengthbox)
+ lengthbox, mode)
def get_args_for_fail(self, modifier):
if self.box is None and not modifier.already_seen_virtual(self.keybox):
@@ -195,11 +248,11 @@
self.vlength.get_args_for_fail(modifier)
def _make_virtual(self, modifier):
- return modifier.make_vstrslice()
+ return modifier.make_vstrslice(self.mode is mode_unicode)
def copy_str_content(newoperations, srcbox, targetbox,
- srcoffsetbox, offsetbox, lengthbox):
+ srcoffsetbox, offsetbox, lengthbox, mode):
if isinstance(srcbox, ConstPtr) and isinstance(srcoffsetbox, Const):
M = 5
else:
@@ -208,17 +261,18 @@
# up to M characters are done "inline", i.e. with STRGETITEM/STRSETITEM
# instead of just a COPYSTRCONTENT.
for i in range(lengthbox.value):
- charbox = _strgetitem(newoperations, srcbox, srcoffsetbox)
+ charbox = _strgetitem(newoperations, srcbox, srcoffsetbox, mode)
srcoffsetbox = _int_add(newoperations, srcoffsetbox, CONST_1)
- newoperations.append(ResOperation(rop.STRSETITEM, [targetbox,
- offsetbox,
- charbox], None))
+ newoperations.append(ResOperation(mode.STRSETITEM, [targetbox,
+ offsetbox,
+ charbox],
+ None))
offsetbox = _int_add(newoperations, offsetbox, CONST_1)
else:
nextoffsetbox = _int_add(newoperations, offsetbox, lengthbox)
- op = ResOperation(rop.COPYSTRCONTENT, [srcbox, targetbox,
- srcoffsetbox, offsetbox,
- lengthbox], None)
+ op = ResOperation(mode.COPYSTRCONTENT, [srcbox, targetbox,
+ srcoffsetbox, offsetbox,
+ lengthbox], None)
newoperations.append(op)
offsetbox = nextoffsetbox
return offsetbox
@@ -245,12 +299,16 @@
newoperations.append(ResOperation(rop.INT_SUB, [box1, box2], resbox))
return resbox
-def _strgetitem(newoperations, strbox, indexbox):
+def _strgetitem(newoperations, strbox, indexbox, mode):
if isinstance(strbox, ConstPtr) and isinstance(indexbox, ConstInt):
- s = strbox.getref(lltype.Ptr(rstr.STR))
- return ConstInt(ord(s.chars[indexbox.getint()]))
+ if mode is mode_string:
+ s = strbox.getref(lltype.Ptr(rstr.STR))
+ return ConstInt(ord(s.chars[indexbox.getint()]))
+ else:
+ s = strbox.getref(lltype.Ptr(rstr.UNICODE))
+ return ConstInt(ord(s.chars[indexbox.getint()]))
resbox = BoxInt()
- newoperations.append(ResOperation(rop.STRGETITEM, [strbox, indexbox],
+ newoperations.append(ResOperation(mode.STRGETITEM, [strbox, indexbox],
resbox))
return resbox
@@ -258,62 +316,34 @@
class OptString(optimizer.Optimization):
"Handling of strings and unicodes."
- def make_vstring_plain(self, box, source_op=None):
- vvalue = VStringPlainValue(self.optimizer, box, source_op)
+ def make_vstring_plain(self, box, source_op, mode):
+ vvalue = VStringPlainValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
return vvalue
- def make_vstring_concat(self, box, source_op=None):
- vvalue = VStringConcatValue(self.optimizer, box, source_op)
+ def make_vstring_concat(self, box, source_op, mode):
+ vvalue = VStringConcatValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
return vvalue
- def make_vstring_slice(self, box, source_op=None):
- vvalue = VStringSliceValue(self.optimizer, box, source_op)
+ def make_vstring_slice(self, box, source_op, mode):
+ vvalue = VStringSliceValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
return vvalue
- def optimize_CALL(self, op):
- # dispatch based on 'oopspecindex' to a method that handles
- # specifically the given oopspec call. For non-oopspec calls,
- # oopspecindex is just zero.
- effectinfo = op.getdescr().get_extra_info()
- if effectinfo is not None:
- oopspecindex = effectinfo.oopspecindex
- for value, meth in opt_call_oopspec_ops:
- if oopspecindex == value:
- if meth(self, op):
- return
- self.emit_operation(op)
-
- def opt_call_oopspec_ARRAYCOPY(self, op):
- source_value = self.getvalue(op.getarg(1))
- dest_value = self.getvalue(op.getarg(2))
- source_start_box = self.get_constant_box(op.getarg(3))
- dest_start_box = self.get_constant_box(op.getarg(4))
- length = self.get_constant_box(op.getarg(5))
- if (source_value.is_virtual() and source_start_box and dest_start_box
- and length and dest_value.is_virtual()):
- # XXX optimize the case where dest value is not virtual,
- # but we still can avoid a mess
- source_start = source_start_box.getint()
- dest_start = dest_start_box.getint()
- for index in range(length.getint()):
- val = source_value.getitem(index + source_start)
- dest_value.setitem(index + dest_start, val)
- return True
- if length and length.getint() == 0:
- return True # 0-length arraycopy
- return False
-
def optimize_NEWSTR(self, op):
+ self._optimize_NEWSTR(op, mode_string)
+ def optimize_NEWUNICODE(self, op):
+ self._optimize_NEWSTR(op, mode_unicode)
+
+ def _optimize_NEWSTR(self, op, mode):
length_box = self.get_constant_box(op.getarg(0))
if length_box:
# if the original 'op' did not have a ConstInt as argument,
# build a new one with the ConstInt argument
if not isinstance(op.getarg(0), ConstInt):
- op = ResOperation(rop.NEWSTR, [length_box], op.result)
- vvalue = self.make_vstring_plain(op.result, op)
+ op = ResOperation(mode.NEWSTR, [length_box], op.result)
+ vvalue = self.make_vstring_plain(op.result, op, mode)
vvalue.setup(length_box.getint())
else:
self.getvalue(op.result).ensure_nonnull()
@@ -329,13 +359,20 @@
value.ensure_nonnull()
self.emit_operation(op)
+ optimize_UNICODESETITEM = optimize_STRSETITEM
+
def optimize_STRGETITEM(self, op):
+ self._optimize_STRGETITEM(op, mode_string)
+ def optimize_UNICODEGETITEM(self, op):
+ self._optimize_STRGETITEM(op, mode_unicode)
+
+ def _optimize_STRGETITEM(self, op, mode):
value = self.getvalue(op.getarg(0))
vindex = self.getvalue(op.getarg(1))
- vresult = self.strgetitem(value, vindex)
+ vresult = self.strgetitem(value, vindex, mode)
self.make_equal_to(op.result, vresult)
- def strgetitem(self, value, vindex):
+ def strgetitem(self, value, vindex, mode):
value.ensure_nonnull()
#
if value.is_virtual() and isinstance(value, VStringSliceValue):
@@ -350,28 +387,71 @@
return value.getitem(vindex.box.getint())
#
resbox = _strgetitem(self.optimizer.newoperations,
- value.force_box(),vindex.force_box())
+ value.force_box(),vindex.force_box(), mode)
return self.getvalue(resbox)
def optimize_STRLEN(self, op):
+ self._optimize_STRLEN(op, mode_string)
+ def optimize_UNICODELEN(self, op):
+ self._optimize_STRLEN(op, mode_unicode)
+
+ def _optimize_STRLEN(self, op, mode):
value = self.getvalue(op.getarg(0))
- lengthbox = value.getstrlen(self.optimizer.newoperations)
+ lengthbox = value.getstrlen(self.optimizer.newoperations, mode)
self.make_equal_to(op.result, self.getvalue(lengthbox))
- def opt_call_oopspec_STR_CONCAT(self, op):
+ def optimize_CALL(self, op):
+ # dispatch based on 'oopspecindex' to a method that handles
+ # specifically the given oopspec call. For non-oopspec calls,
+ # oopspecindex is just zero.
+ effectinfo = op.getdescr().get_extra_info()
+ if effectinfo is not None:
+ oopspecindex = effectinfo.oopspecindex
+ for value, meth in opt_call_oopspec_ops:
+ if oopspecindex == value: # a match with the OS_STR_xxx
+ if meth(self, op, mode_string):
+ return
+ break
+ if oopspecindex == value + EffectInfo._OS_offset_uni:
+ # a match with the OS_UNI_xxx
+ if meth(self, op, mode_unicode):
+ return
+ break
+ if oopspecindex == EffectInfo.OS_STR2UNICODE:
+ if self.opt_call_str_STR2UNICODE(op):
+ return
+ self.emit_operation(op)
+
+ def opt_call_str_STR2UNICODE(self, op):
+ # Constant-fold unicode("constant string").
+ # More generally, supporting non-constant but virtual cases is
+ # not obvious, because of the exception UnicodeDecodeError that
+ # can be raised by ll_str2unicode()
+ varg = self.getvalue(op.getarg(1))
+ s = varg.get_constant_string_spec(mode_string)
+ if s is None:
+ return False
+ try:
+ u = unicode(s)
+ except UnicodeDecodeError:
+ return False
+ self.make_constant(op.result, get_const_ptr_for_unicode(u))
+ return True
+
+ def opt_call_stroruni_STR_CONCAT(self, op, mode):
vleft = self.getvalue(op.getarg(1))
vright = self.getvalue(op.getarg(2))
vleft.ensure_nonnull()
vright.ensure_nonnull()
newoperations = self.optimizer.newoperations
- len1box = vleft.getstrlen(newoperations)
- len2box = vright.getstrlen(newoperations)
+ len1box = vleft.getstrlen(newoperations, mode)
+ len2box = vright.getstrlen(newoperations, mode)
lengthbox = _int_add(newoperations, len1box, len2box)
- value = self.make_vstring_concat(op.result, op)
+ value = self.make_vstring_concat(op.result, op, mode)
value.setup(vleft, vright, lengthbox)
return True
- def opt_call_oopspec_STR_SLICE(self, op):
+ def opt_call_stroruni_STR_SLICE(self, op, mode):
newoperations = self.optimizer.newoperations
vstr = self.getvalue(op.getarg(1))
vstart = self.getvalue(op.getarg(2))
@@ -380,7 +460,7 @@
if (isinstance(vstr, VStringPlainValue) and vstart.is_constant()
and vstop.is_constant()):
# slicing with constant bounds of a VStringPlainValue
- value = self.make_vstring_plain(op.result, op)
+ value = self.make_vstring_plain(op.result, op, mode)
value.setup_slice(vstr._chars, vstart.box.getint(),
vstop.box.getint())
return True
@@ -398,16 +478,16 @@
vstart.force_box())
vstart = self.getvalue(startbox)
#
- value = self.make_vstring_slice(op.result, op)
+ value = self.make_vstring_slice(op.result, op, mode)
value.setup(vstr, vstart, self.getvalue(lengthbox))
return True
- def opt_call_oopspec_STR_EQUAL(self, op):
+ def opt_call_stroruni_STR_EQUAL(self, op, mode):
v1 = self.getvalue(op.getarg(1))
v2 = self.getvalue(op.getarg(2))
#
- l1box = v1.getstrlen(None)
- l2box = v2.getstrlen(None)
+ l1box = v1.getstrlen(None, mode)
+ l2box = v2.getstrlen(None, mode)
if (l1box is not None and l2box is not None and
isinstance(l1box, ConstInt) and
isinstance(l2box, ConstInt) and
@@ -416,13 +496,13 @@
self.make_constant(op.result, CONST_0)
return True
#
- if self.handle_str_equal_level1(v1, v2, op.result):
+ if self.handle_str_equal_level1(v1, v2, op.result, mode):
return True
- if self.handle_str_equal_level1(v2, v1, op.result):
+ if self.handle_str_equal_level1(v2, v1, op.result, mode):
return True
- if self.handle_str_equal_level2(v1, v2, op.result):
+ if self.handle_str_equal_level2(v1, v2, op.result, mode):
return True
- if self.handle_str_equal_level2(v2, v1, op.result):
+ if self.handle_str_equal_level2(v2, v1, op.result, mode):
return True
#
if v1.is_nonnull() and v2.is_nonnull():
@@ -434,37 +514,37 @@
else:
do = EffectInfo.OS_STREQ_NONNULL
self.generate_modified_call(do, [v1.force_box(),
- v2.force_box()], op.result)
+ v2.force_box()], op.result, mode)
return True
return False
- def handle_str_equal_level1(self, v1, v2, resultbox):
- l2box = v2.getstrlen(None)
+ def handle_str_equal_level1(self, v1, v2, resultbox, mode):
+ l2box = v2.getstrlen(None, mode)
if isinstance(l2box, ConstInt):
if l2box.value == 0:
- lengthbox = v1.getstrlen(self.optimizer.newoperations)
+ lengthbox = v1.getstrlen(self.optimizer.newoperations, mode)
seo = self.optimizer.send_extra_operation
seo(ResOperation(rop.INT_EQ, [lengthbox, CONST_0], resultbox))
return True
if l2box.value == 1:
- l1box = v1.getstrlen(None)
+ l1box = v1.getstrlen(None, mode)
if isinstance(l1box, ConstInt) and l1box.value == 1:
# comparing two single chars
- vchar1 = self.strgetitem(v1, optimizer.CVAL_ZERO)
- vchar2 = self.strgetitem(v2, optimizer.CVAL_ZERO)
+ vchar1 = self.strgetitem(v1, optimizer.CVAL_ZERO, mode)
+ vchar2 = self.strgetitem(v2, optimizer.CVAL_ZERO, mode)
seo = self.optimizer.send_extra_operation
seo(ResOperation(rop.INT_EQ, [vchar1.force_box(),
vchar2.force_box()],
resultbox))
return True
if isinstance(v1, VStringSliceValue):
- vchar = self.strgetitem(v2, optimizer.CVAL_ZERO)
+ vchar = self.strgetitem(v2, optimizer.CVAL_ZERO, mode)
do = EffectInfo.OS_STREQ_SLICE_CHAR
self.generate_modified_call(do, [v1.vstr.force_box(),
v1.vstart.force_box(),
v1.vlength.force_box(),
vchar.force_box()],
- resultbox)
+ resultbox, mode)
return True
#
if v2.is_null():
@@ -482,17 +562,18 @@
#
return False
- def handle_str_equal_level2(self, v1, v2, resultbox):
- l2box = v2.getstrlen(None)
+ def handle_str_equal_level2(self, v1, v2, resultbox, mode):
+ l2box = v2.getstrlen(None, mode)
if isinstance(l2box, ConstInt):
if l2box.value == 1:
- vchar = self.strgetitem(v2, optimizer.CVAL_ZERO)
+ vchar = self.strgetitem(v2, optimizer.CVAL_ZERO, mode)
if v1.is_nonnull():
do = EffectInfo.OS_STREQ_NONNULL_CHAR
else:
do = EffectInfo.OS_STREQ_CHECKNULL_CHAR
self.generate_modified_call(do, [v1.force_box(),
- vchar.force_box()], resultbox)
+ vchar.force_box()], resultbox,
+ mode)
return True
#
if v1.is_virtual() and isinstance(v1, VStringSliceValue):
@@ -503,11 +584,12 @@
self.generate_modified_call(do, [v1.vstr.force_box(),
v1.vstart.force_box(),
v1.vlength.force_box(),
- v2.force_box()], resultbox)
+ v2.force_box()], resultbox, mode)
return True
return False
- def generate_modified_call(self, oopspecindex, args, result):
+ def generate_modified_call(self, oopspecindex, args, result, mode):
+ oopspecindex += mode.OS_offset
calldescr, func = callinfo_for_oopspec(oopspecindex)
op = ResOperation(rop.CALL, [ConstInt(func)] + args, result,
descr=calldescr)
@@ -525,7 +607,7 @@
optimize_ops = _findall(OptString, 'optimize_')
def _findall_call_oopspec():
- prefix = 'opt_call_oopspec_'
+ prefix = 'opt_call_stroruni_'
result = []
for name in dir(OptString):
if name.startswith(prefix):
Modified: pypy/branch/jitffi/pypy/jit/metainterp/resume.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/resume.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/resume.py Mon Oct 18 11:22:20 2010
@@ -255,13 +255,19 @@
def make_varray(self, arraydescr):
return VArrayInfo(arraydescr)
- def make_vstrplain(self):
+ def make_vstrplain(self, is_unicode=False):
+ if is_unicode:
+ return VUniPlainInfo()
return VStrPlainInfo()
- def make_vstrconcat(self):
+ def make_vstrconcat(self, is_unicode=False):
+ if is_unicode:
+ return VUniConcatInfo()
return VStrConcatInfo()
- def make_vstrslice(self):
+ def make_vstrslice(self, is_unicode=False):
+ if is_unicode:
+ return VUniSliceInfo()
return VStrSliceInfo()
def register_virtual_fields(self, virtualbox, fieldboxes):
@@ -550,6 +556,60 @@
for i in self.fieldnums:
debug_print("\t\t", str(untag(i)))
+
+class VUniPlainInfo(AbstractVirtualInfo):
+ """Stands for the unicode string made out of the characters of all
+ fieldnums."""
+
+ @specialize.argtype(1)
+ def allocate(self, decoder, index):
+ length = len(self.fieldnums)
+ string = decoder.allocate_unicode(length)
+ decoder.virtuals_cache[index] = string
+ for i in range(length):
+ decoder.unicode_setitem(string, i, self.fieldnums[i])
+ return string
+
+ def debug_prints(self):
+ debug_print("\tvuniplaininfo length", len(self.fieldnums))
+
+
+class VUniConcatInfo(AbstractVirtualInfo):
+ """Stands for the unicode string made out of the concatenation of two
+ other unicode strings."""
+
+ @specialize.argtype(1)
+ def allocate(self, decoder, index):
+ # xxx for blackhole resuming, this will build all intermediate
+ # strings and throw them away immediately, which is a bit sub-
+ # efficient. Not sure we care.
+ left, right = self.fieldnums
+ string = decoder.concat_unicodes(left, right)
+ decoder.virtuals_cache[index] = string
+ return string
+
+ def debug_prints(self):
+ debug_print("\tvuniconcatinfo")
+ for i in self.fieldnums:
+ debug_print("\t\t", str(untag(i)))
+
+
+class VUniSliceInfo(AbstractVirtualInfo):
+ """Stands for the unicode string made out of slicing another
+ unicode string."""
+
+ @specialize.argtype(1)
+ def allocate(self, decoder, index):
+ largerstr, start, length = self.fieldnums
+ string = decoder.slice_unicode(largerstr, start, length)
+ decoder.virtuals_cache[index] = string
+ return string
+
+ def debug_prints(self):
+ debug_print("\tvunisliceinfo")
+ for i in self.fieldnums:
+ debug_print("\t\t", str(untag(i)))
+
# ____________________________________________________________
class AbstractResumeDataReader(object):
@@ -725,6 +785,32 @@
return self.metainterp.execute_and_record_varargs(
rop.CALL, [ConstInt(func), strbox, startbox, stopbox], calldescr)
+ def allocate_unicode(self, length):
+ return self.metainterp.execute_and_record(rop.NEWUNICODE,
+ None, ConstInt(length))
+
+ def unicode_setitem(self, strbox, index, charnum):
+ charbox = self.decode_box(charnum, INT)
+ self.metainterp.execute_and_record(rop.UNICODESETITEM, None,
+ strbox, ConstInt(index), charbox)
+
+ def concat_unicodes(self, str1num, str2num):
+ calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT)
+ str1box = self.decode_box(str1num, REF)
+ str2box = self.decode_box(str2num, REF)
+ return self.metainterp.execute_and_record_varargs(
+ rop.CALL, [ConstInt(func), str1box, str2box], calldescr)
+
+ def slice_unicode(self, strnum, startnum, lengthnum):
+ calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE)
+ strbox = self.decode_box(strnum, REF)
+ startbox = self.decode_box(startnum, INT)
+ lengthbox = self.decode_box(lengthnum, INT)
+ stopbox = self.metainterp.execute_and_record(rop.INT_ADD, None,
+ startbox, lengthbox)
+ return self.metainterp.execute_and_record_varargs(
+ rop.CALL, [ConstInt(func), strbox, startbox, stopbox], calldescr)
+
def setfield(self, descr, structbox, fieldnum):
if descr.is_pointer_field():
kind = REF
@@ -967,6 +1053,31 @@
result = funcptr(str, start, start + length)
return lltype.cast_opaque_ptr(llmemory.GCREF, result)
+ def allocate_unicode(self, length):
+ return self.cpu.bh_newunicode(length)
+
+ def unicode_setitem(self, str, index, charnum):
+ char = self.decode_int(charnum)
+ self.cpu.bh_unicodesetitem(str, index, char)
+
+ def concat_unicodes(self, str1num, str2num):
+ str1 = self.decode_ref(str1num)
+ str2 = self.decode_ref(str2num)
+ str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str1)
+ str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str2)
+ funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT)
+ result = funcptr(str1, str2)
+ return lltype.cast_opaque_ptr(llmemory.GCREF, result)
+
+ def slice_unicode(self, strnum, startnum, lengthnum):
+ str = self.decode_ref(strnum)
+ start = self.decode_int(startnum)
+ length = self.decode_int(lengthnum)
+ str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str)
+ funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE)
+ result = funcptr(str, start, start + length)
+ return lltype.cast_opaque_ptr(llmemory.GCREF, result)
+
def setfield(self, descr, struct, fieldnum):
if descr.is_pointer_field():
newvalue = self.decode_ref(fieldnum)
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/oparser.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/oparser.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/oparser.py Mon Oct 18 11:22:20 2010
@@ -5,7 +5,7 @@
from pypy.jit.metainterp.history import TreeLoop, BoxInt, ConstInt,\
ConstObj, ConstPtr, Box, BasicFailDescr, BoxFloat, ConstFloat,\
- LoopToken, get_const_ptr_for_string
+ LoopToken, get_const_ptr_for_string, get_const_ptr_for_unicode
from pypy.jit.metainterp.resoperation import rop, ResOperation, ResOpWithDescr, N_aryOp
from pypy.jit.metainterp.typesystem import llhelper
from pypy.jit.codewriter.heaptracker import adr2int
@@ -158,10 +158,15 @@
except ValueError:
if self.is_float(arg):
return ConstFloat(float(arg))
- if arg.startswith('"') or arg.startswith("'"):
+ if (arg.startswith('"') or arg.startswith("'") or
+ arg.startswith('s"')):
# XXX ootype
- info = arg.strip("'\"")
+ info = arg[1:].strip("'\"")
return get_const_ptr_for_string(info)
+ if arg.startswith('u"'):
+ # XXX ootype
+ info = arg[1:].strip("'\"")
+ return get_const_ptr_for_unicode(info)
if arg.startswith('ConstClass('):
name = arg[len('ConstClass('):-1]
return self.get_const(name, 'class')
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefindnode.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefindnode.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefindnode.py Mon Oct 18 11:22:20 2010
@@ -117,33 +117,32 @@
EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE))
arraycopydescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
EffectInfo([], [], [], oopspecindex=EffectInfo.OS_ARRAYCOPY))
- strconcatdescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_CONCAT))
- slicedescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_SLICE))
- strequaldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_EQUAL))
- streq_slice_checknull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_CHECKNULL))
- streq_slice_nonnull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_NONNULL))
- streq_slice_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_CHAR))
- streq_nonnull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_NONNULL))
- streq_nonnull_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_NONNULL_CHAR))
- streq_checknull_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_CHECKNULL_CHAR))
- streq_lengthok_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_LENGTHOK))
+
+ for _name, _os in [
+ ('strconcatdescr', 'OS_STR_CONCAT'),
+ ('strslicedescr', 'OS_STR_SLICE'),
+ ('strequaldescr', 'OS_STR_EQUAL'),
+ ('streq_slice_checknull_descr', 'OS_STREQ_SLICE_CHECKNULL'),
+ ('streq_slice_nonnull_descr', 'OS_STREQ_SLICE_NONNULL'),
+ ('streq_slice_char_descr', 'OS_STREQ_SLICE_CHAR'),
+ ('streq_nonnull_descr', 'OS_STREQ_NONNULL'),
+ ('streq_nonnull_char_descr', 'OS_STREQ_NONNULL_CHAR'),
+ ('streq_checknull_char_descr', 'OS_STREQ_CHECKNULL_CHAR'),
+ ('streq_lengthok_descr', 'OS_STREQ_LENGTHOK'),
+ ]:
+ _oopspecindex = getattr(EffectInfo, _os)
+ locals()[_name] = \
+ cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=_oopspecindex))
+ #
+ _oopspecindex = getattr(EffectInfo, _os.replace('STR', 'UNI'))
+ locals()[_name.replace('str', 'unicode')] = \
+ cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=_oopspecindex))
+
+ s2u_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR2UNICODE))
+ #
class LoopToken(AbstractDescr):
pass
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizeopt.py Mon Oct 18 11:22:20 2010
@@ -3498,7 +3498,7 @@
i0 = strlen(p0)
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not', expected)
def test_addsub_const(self):
ops = """
@@ -3904,6 +3904,15 @@
"""
self.optimize_loop(ops, 'Not, Not', expected)
+ # ----------
+ def optimize_strunicode_loop(self, ops, spectext, optops):
+ # check with the arguments passed in
+ self.optimize_loop(ops, spectext, optops)
+ # check with replacing 'str' with 'unicode' everywhere
+ self.optimize_loop(ops.replace('str','unicode').replace('s"', 'u"'),
+ spectext,
+ optops.replace('str','unicode').replace('s"', 'u"'))
+
def test_newstr_1(self):
ops = """
[i0]
@@ -3916,7 +3925,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not', expected)
def test_newstr_2(self):
ops = """
@@ -3932,7 +3941,7 @@
[i0, i1]
jump(i1, i0)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_concat_1(self):
ops = """
@@ -3953,7 +3962,7 @@
copystrcontent(p2, p3, 0, i4, i5)
jump(p2, p3)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_concat_vstr2_str(self):
ops = """
@@ -3976,7 +3985,7 @@
copystrcontent(p2, p3, 0, 2, i4)
jump(i1, i0, p3)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_concat_str_vstr2(self):
ops = """
@@ -4000,7 +4009,7 @@
i6 = int_add(i5, 1) # will be killed by the backend
jump(i1, i0, p3)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_concat_str_str_str(self):
ops = """
@@ -4027,12 +4036,12 @@
copystrcontent(p3, p5, 0, i12b, i3b)
jump(p2, p3, p5)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_concat_str_cstr1(self):
ops = """
[p2]
- p3 = call(0, p2, "x", descr=strconcatdescr)
+ p3 = call(0, p2, s"x", descr=strconcatdescr)
jump(p3)
"""
expected = """
@@ -4046,28 +4055,28 @@
i5 = int_add(i4, 1) # will be killed by the backend
jump(p3)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not', expected)
def test_str_concat_consts(self):
ops = """
[]
- p1 = same_as("ab")
- p2 = same_as("cde")
+ p1 = same_as(s"ab")
+ p2 = same_as(s"cde")
p3 = call(0, p1, p2, descr=strconcatdescr)
escape(p3)
jump()
"""
expected = """
[]
- escape("abcde")
+ escape(s"abcde")
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_strunicode_loop(ops, '', expected)
def test_str_slice_1(self):
ops = """
[p1, i1, i2]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
jump(p2, i1, i2)
"""
expected = """
@@ -4077,12 +4086,12 @@
copystrcontent(p1, p2, i1, 0, i3)
jump(p2, i1, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_slice_2(self):
ops = """
[p1, i2]
- p2 = call(0, p1, 0, i2, descr=slicedescr)
+ p2 = call(0, p1, 0, i2, descr=strslicedescr)
jump(p2, i2)
"""
expected = """
@@ -4091,13 +4100,13 @@
copystrcontent(p1, p2, 0, 0, i2)
jump(p2, i2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_slice_3(self):
ops = """
[p1, i1, i2, i3, i4]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
- p3 = call(0, p2, i3, i4, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
+ p3 = call(0, p2, i3, i4, descr=strslicedescr)
jump(p3, i1, i2, i3, i4)
"""
expected = """
@@ -4109,12 +4118,12 @@
copystrcontent(p1, p3, i6, 0, i5)
jump(p3, i1, i2, i3, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not, Not', expected)
def test_str_slice_getitem1(self):
ops = """
[p1, i1, i2, i3]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
i4 = strgetitem(p2, i3)
escape(i4)
jump(p1, i1, i2, i3)
@@ -4127,7 +4136,7 @@
escape(i4)
jump(p1, i1, i2, i3)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not', expected)
def test_str_slice_plain(self):
ops = """
@@ -4135,7 +4144,7 @@
p1 = newstr(2)
strsetitem(p1, 0, i3)
strsetitem(p1, 1, i4)
- p2 = call(0, p1, 1, 2, descr=slicedescr)
+ p2 = call(0, p1, 1, 2, descr=strslicedescr)
i5 = strgetitem(p2, 0)
escape(i5)
jump(i3, i4)
@@ -4145,12 +4154,12 @@
escape(i4)
jump(i3, i4)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_slice_concat(self):
ops = """
[p1, i1, i2, p2]
- p3 = call(0, p1, i1, i2, descr=slicedescr)
+ p3 = call(0, p1, i1, i2, descr=strslicedescr)
p4 = call(0, p3, p2, descr=strconcatdescr)
jump(p4, i1, i2, p2)
"""
@@ -4166,10 +4175,10 @@
copystrcontent(p2, p4, 0, i3, i4b)
jump(p4, i1, i2, p2)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not', expected)
# ----------
- def optimize_loop_extradescrs(self, ops, spectext, optops):
+ def optimize_strunicode_loop_extradescrs(self, ops, spectext, optops):
from pypy.jit.metainterp.optimizeopt import string
def my_callinfo_for_oopspec(oopspecindex):
calldescrtype = type(LLtypeMixin.strequaldescr)
@@ -4184,7 +4193,7 @@
saved = string.callinfo_for_oopspec
try:
string.callinfo_for_oopspec = my_callinfo_for_oopspec
- self.optimize_loop(ops, spectext, optops)
+ self.optimize_strunicode_loop(ops, spectext, optops)
finally:
string.callinfo_for_oopspec = saved
@@ -4195,7 +4204,7 @@
escape(i0)
jump(p1, p2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', ops)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', ops)
def test_str_equal_noop2(self):
ops = """
@@ -4220,12 +4229,13 @@
escape(i0)
jump(p1, p2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not',
+ expected)
def test_str_equal_slice1(self):
ops = """
[p1, i1, i2, p3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p4, p3, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4237,12 +4247,13 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_slice2(self):
ops = """
[p1, i1, i2, p3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p3, p4, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4254,13 +4265,14 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_slice3(self):
ops = """
[p1, i1, i2, p3]
guard_nonnull(p3) []
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p3, p4, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4273,13 +4285,14 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_slice4(self):
ops = """
[p1, i1, i2]
- p3 = call(0, p1, i1, i2, descr=slicedescr)
- i0 = call(0, p3, "x", descr=strequaldescr)
+ p3 = call(0, p1, i1, i2, descr=strslicedescr)
+ i0 = call(0, p3, s"x", descr=strequaldescr)
escape(i0)
jump(p1, i1, i2)
"""
@@ -4290,12 +4303,13 @@
escape(i0)
jump(p1, i1, i2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not',
+ expected)
def test_str_equal_slice5(self):
ops = """
[p1, i1, i2, i3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
p5 = newstr(1)
strsetitem(p5, 0, i3)
i0 = call(0, p5, p4, descr=strequaldescr)
@@ -4309,7 +4323,8 @@
escape(i0)
jump(p1, i1, i2, i3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_none1(self):
ops = """
@@ -4324,7 +4339,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_none2(self):
ops = """
@@ -4339,30 +4354,30 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull1(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "hello world", descr=strequaldescr)
+ i0 = call(0, p1, s"hello world", descr=strequaldescr)
escape(i0)
jump(p1)
"""
expected = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "hello world", descr=streq_nonnull_descr)
+ i0 = call(0, p1, s"hello world", descr=streq_nonnull_descr)
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull2(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "", descr=strequaldescr)
+ i0 = call(0, p1, s"", descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4374,13 +4389,13 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull3(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "x", descr=strequaldescr)
+ i0 = call(0, p1, s"x", descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4391,13 +4406,13 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull4(self):
ops = """
[p1, p2]
p4 = call(0, p1, p2, descr=strconcatdescr)
- i0 = call(0, "hello world", p4, descr=strequaldescr)
+ i0 = call(0, s"hello world", p4, descr=strequaldescr)
escape(i0)
jump(p1, p2)
"""
@@ -4412,17 +4427,17 @@
i5 = strlen(p2)
i6 = int_add(i4, i5) # will be killed by the backend
copystrcontent(p2, p4, 0, i4, i5)
- i0 = call(0, "hello world", p4, descr=streq_nonnull_descr)
+ i0 = call(0, s"hello world", p4, descr=streq_nonnull_descr)
escape(i0)
jump(p1, p2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', expected)
def test_str_equal_chars0(self):
ops = """
[i1]
p1 = newstr(0)
- i0 = call(0, p1, "", descr=strequaldescr)
+ i0 = call(0, p1, s"", descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4431,14 +4446,14 @@
escape(1)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_chars1(self):
ops = """
[i1]
p1 = newstr(1)
strsetitem(p1, 0, i1)
- i0 = call(0, p1, "x", descr=strequaldescr)
+ i0 = call(0, p1, s"x", descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4448,7 +4463,7 @@
escape(i0)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_chars2(self):
ops = """
@@ -4456,7 +4471,7 @@
p1 = newstr(2)
strsetitem(p1, 0, i1)
strsetitem(p1, 1, i2)
- i0 = call(0, p1, "xy", descr=strequaldescr)
+ i0 = call(0, p1, s"xy", descr=strequaldescr)
escape(i0)
jump(i1, i2)
"""
@@ -4465,16 +4480,16 @@
p1 = newstr(2)
strsetitem(p1, 0, i1)
strsetitem(p1, 1, i2)
- i0 = call(0, p1, "xy", descr=streq_lengthok_descr)
+ i0 = call(0, p1, s"xy", descr=streq_lengthok_descr)
escape(i0)
jump(i1, i2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', expected)
def test_str_equal_chars3(self):
ops = """
[p1]
- i0 = call(0, "x", p1, descr=strequaldescr)
+ i0 = call(0, s"x", p1, descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4484,14 +4499,14 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_lengthmismatch1(self):
ops = """
[i1]
p1 = newstr(1)
strsetitem(p1, 0, i1)
- i0 = call(0, "xy", p1, descr=strequaldescr)
+ i0 = call(0, s"xy", p1, descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4500,12 +4515,33 @@
escape(0)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
-
- # XXX unicode operations
- # XXX str2unicode
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
+ def test_str2unicode_constant(self):
+ ops = """
+ []
+ p0 = call(0, "xy", descr=s2u_descr) # string -> unicode
+ escape(p0)
+ jump()
+ """
+ expected = """
+ []
+ escape(u"xy")
+ jump()
+ """
+ self.optimize_strunicode_loop_extradescrs(ops, '', expected)
+ def test_str2unicode_nonconstant(self):
+ ops = """
+ [p0]
+ p1 = call(0, p0, descr=s2u_descr) # string -> unicode
+ escape(p1)
+ jump(p1)
+ """
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', ops)
+ # more generally, supporting non-constant but virtual cases is
+ # not obvious, because of the exception UnicodeDecodeError that
+ # can be raised by ll_str2unicode()
##class TestOOtype(OptimizeOptTest, OOtypeMixin):
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_resume.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_resume.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_resume.py Mon Oct 18 11:22:20 2010
@@ -240,6 +240,17 @@
return FakeBuiltObject(strconcat=[left, right])
def slice_string(self, str, start, length):
return FakeBuiltObject(strslice=[str, start, length])
+ def allocate_unicode(self, length):
+ return FakeBuiltObject(unistring=[None]*length)
+ def unicode_setitem(self, unistring, i, fieldnum):
+ value, tag = untag(fieldnum)
+ assert tag == TAGINT
+ assert 0 <= i < len(unistring.unistring)
+ unistring.unistring[i] = value
+ def concat_unicodes(self, left, right):
+ return FakeBuiltObject(uniconcat=[left, right])
+ def slice_unicode(self, str, start, length):
+ return FakeBuiltObject(unislice=[str, start, length])
class FakeBuiltObject(object):
def __init__(self, **kwds):
@@ -304,6 +315,30 @@
assert reader.force_all_virtuals() == [
FakeBuiltObject(strslice=info.fieldnums)]
+def test_vuniplaininfo():
+ info = VUniPlainInfo()
+ info.fieldnums = [tag(60, TAGINT)]
+ reader = FakeResumeDataReader()
+ reader._prepare_virtuals([info])
+ assert reader.force_all_virtuals() == [
+ FakeBuiltObject(unistring=[60])]
+
+def test_vuniconcatinfo():
+ info = VUniConcatInfo()
+ info.fieldnums = [tag(10, TAGBOX), tag(20, TAGBOX)]
+ reader = FakeResumeDataReader()
+ reader._prepare_virtuals([info])
+ assert reader.force_all_virtuals() == [
+ FakeBuiltObject(uniconcat=info.fieldnums)]
+
+def test_vunisliceinfo():
+ info = VUniSliceInfo()
+ info.fieldnums = [tag(10, TAGBOX), tag(20, TAGBOX), tag(30, TAGBOX)]
+ reader = FakeResumeDataReader()
+ reader._prepare_virtuals([info])
+ assert reader.force_all_virtuals() == [
+ FakeBuiltObject(unislice=info.fieldnums)]
+
# ____________________________________________________________
Modified: pypy/branch/jitffi/pypy/jit/metainterp/test/test_string.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/test/test_string.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/test/test_string.py Mon Oct 18 11:22:20 2010
@@ -6,14 +6,17 @@
class StringTests:
+ _str, _chr = str, chr
+
def test_eq_residual(self):
+ _str = self._str
jitdriver = JitDriver(greens = [], reds = ['n', 'i', 's'])
- global_s = "hello"
+ global_s = _str("hello")
def f(n, b, s):
if b:
- s += "ello"
+ s += _str("ello")
else:
- s += "allo"
+ s += _str("allo")
i = 0
while n > 0:
jitdriver.can_enter_jit(s=s, n=n, i=i)
@@ -21,18 +24,19 @@
n -= 1 + (s == global_s)
i += 1
return i
- res = self.meta_interp(f, [10, True, 'h'], listops=True)
+ res = self.meta_interp(f, [10, True, _str('h')], listops=True)
assert res == 5
self.check_loops(**{self.CALL: 1, self.CALL_PURE: 0})
def test_eq_folded(self):
+ _str = self._str
jitdriver = JitDriver(greens = ['s'], reds = ['n', 'i'])
- global_s = "hello"
+ global_s = _str("hello")
def f(n, b, s):
if b:
- s += "ello"
+ s += _str("ello")
else:
- s += "allo"
+ s += _str("allo")
i = 0
while n > 0:
jitdriver.can_enter_jit(s=s, n=n, i=i)
@@ -40,31 +44,18 @@
n -= 1 + (s == global_s)
i += 1
return i
- res = self.meta_interp(f, [10, True, 'h'], listops=True)
+ res = self.meta_interp(f, [10, True, _str('h')], listops=True)
assert res == 5
self.check_loops(**{self.CALL: 0, self.CALL_PURE: 0})
def test_newstr(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['n', 'm'])
def f(n, m):
while True:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- bytecode = 'adlfkj' + chr(n)
- res = bytecode[n]
- m -= 1
- if m < 0:
- return ord(res)
- res = self.meta_interp(f, [6, 10])
- assert res == 6
-
- def test_newunicode(self):
- jitdriver = JitDriver(greens = [], reds = ['n', 'm'])
- def f(n, m):
- while True:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- bytecode = u'adlfkj' + unichr(n)
+ bytecode = _str('adlfkj') + _chr(n)
res = bytecode[n]
m -= 1
if m < 0:
@@ -73,95 +64,96 @@
assert res == 6
def test_char2string_pure(self):
- for dochr in [chr, ]: #unichr]:
- jitdriver = JitDriver(greens = [], reds = ['n'])
- @dont_look_inside
- def escape(x):
- pass
- def f(n):
- while n > 0:
- jitdriver.can_enter_jit(n=n)
- jitdriver.jit_merge_point(n=n)
- s = dochr(n)
- if not we_are_jitted():
- s += s # forces to be a string
- if n > 100:
- escape(s)
- n -= 1
- return 42
- self.meta_interp(f, [6])
- self.check_loops(newstr=0, strsetitem=0, strlen=0,
- newunicode=0, unicodesetitem=0, unicodelen=0)
+ _str, _chr = self._str, self._chr
+ jitdriver = JitDriver(greens = [], reds = ['n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ def f(n):
+ while n > 0:
+ jitdriver.can_enter_jit(n=n)
+ jitdriver.jit_merge_point(n=n)
+ s = _chr(n)
+ if not we_are_jitted():
+ s += s # forces to be a string
+ if n > 100:
+ escape(s)
+ n -= 1
+ return 42
+ self.meta_interp(f, [6])
+ self.check_loops(newstr=0, strsetitem=0, strlen=0,
+ newunicode=0, unicodesetitem=0, unicodelen=0)
def test_char2string_escape(self):
- for dochr in [chr, ]: #unichr]:
- jitdriver = JitDriver(greens = [], reds = ['n', 'total'])
- @dont_look_inside
- def escape(x):
- return ord(x[0])
- def f(n):
- total = 0
- while n > 0:
- jitdriver.can_enter_jit(n=n, total=total)
- jitdriver.jit_merge_point(n=n, total=total)
- s = dochr(n)
- if not we_are_jitted():
- s += s # forces to be a string
- total += escape(s)
- n -= 1
- return total
- res = self.meta_interp(f, [6])
- assert res == 21
+ _str, _chr = self._str, self._chr
+ jitdriver = JitDriver(greens = [], reds = ['n', 'total'])
+ @dont_look_inside
+ def escape(x):
+ return ord(x[0])
+ def f(n):
+ total = 0
+ while n > 0:
+ jitdriver.can_enter_jit(n=n, total=total)
+ jitdriver.jit_merge_point(n=n, total=total)
+ s = _chr(n)
+ if not we_are_jitted():
+ s += s # forces to be a string
+ total += escape(s)
+ n -= 1
+ return total
+ res = self.meta_interp(f, [6])
+ assert res == 21
def test_char2string2char(self):
- for dochr in [chr, ]: #unichr]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'total'])
- def f(m):
- total = 0
- while m > 0:
- jitdriver.can_enter_jit(m=m, total=total)
- jitdriver.jit_merge_point(m=m, total=total)
- string = dochr(m)
- if m > 100:
- string += string # forces to be a string
- # read back the character
- c = string[0]
- total += ord(c)
- m -= 1
- return total
- res = self.meta_interp(f, [6])
- assert res == 21
- self.check_loops(newstr=0, strgetitem=0, strsetitem=0, strlen=0,
- newunicode=0, unicodegetitem=0, unicodesetitem=0,
- unicodelen=0)
+ _str, _chr = self._str, self._chr
+ jitdriver = JitDriver(greens = [], reds = ['m', 'total'])
+ def f(m):
+ total = 0
+ while m > 0:
+ jitdriver.can_enter_jit(m=m, total=total)
+ jitdriver.jit_merge_point(m=m, total=total)
+ string = _chr(m)
+ if m > 100:
+ string += string # forces to be a string
+ # read back the character
+ c = string[0]
+ total += ord(c)
+ m -= 1
+ return total
+ res = self.meta_interp(f, [6])
+ assert res == 21
+ self.check_loops(newstr=0, strgetitem=0, strsetitem=0, strlen=0,
+ newunicode=0, unicodegetitem=0, unicodesetitem=0,
+ unicodelen=0)
def test_strconcat_pure(self):
- for somestr in ["abc", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- mylist = [somestr+str(i) for i in range(10)]
- def f(n, m):
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + mylist[m]
- if m > 100:
- escape(s)
- m -= 1
- return 42
- self.meta_interp(f, [6, 7])
- self.check_loops(newstr=0, strsetitem=0,
- newunicode=0, unicodesetitem=0,
- call=0, call_pure=0)
+ _str = self._str
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ mylist = [_str("abc") + _str(i) for i in range(10)]
+ def f(n, m):
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = mylist[n] + mylist[m]
+ if m > 100:
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 7])
+ self.check_loops(newstr=0, strsetitem=0,
+ newunicode=0, unicodesetitem=0,
+ call=0, call_pure=0)
def test_strconcat_escape_str_str(self):
+ _str = self._str
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
@@ -171,46 +163,64 @@
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=0, copystrcontent=2,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=0, copystrcontent=2,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=0,
+ copyunicodecontent=2,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_str_char(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + chr(m)
+ s = mylist[n] + _chr(m)
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=1,
+ copyunicodecontent=1,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_char_str(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = chr(n) + mylist[m]
+ s = _chr(n) + mylist[m]
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=1,
+ copyunicodecontent=1,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_char_char(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
@@ -219,91 +229,132 @@
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = chr(n) + chr(m)
+ s = _chr(n) + _chr(m)
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=2, copystrcontent=0,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=2, copystrcontent=0,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=2,
+ copyunicodecontent=0,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_str_char_str(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + chr(n) + mylist[m]
+ s = mylist[n] + _chr(n) + mylist[m]
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=1, copystrcontent=2,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=1, copystrcontent=2,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=1,
+ copyunicodecontent=2,
+ call=1, call_pure=0) # escape
def test_strconcat_guard_fail(self):
- for somestr in ["abc", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- mylist = [somestr+str(i) for i in range(12)]
- def f(n, m):
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + mylist[m]
- if m & 1:
- escape(s)
- m -= 1
- return 42
- self.meta_interp(f, [6, 10])
+ _str = self._str
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ mylist = [_str("abc") + _str(i) for i in range(12)]
+ def f(n, m):
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = mylist[n] + mylist[m]
+ if m & 1:
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 10])
def test_strslice(self):
- for somestr in ["abc", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- def f(n, m):
- assert n >= 0
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = "foobarbazetc"[m:n]
- if m <= 5:
- escape(s)
- m -= 1
- return 42
- self.meta_interp(f, [10, 10])
+ _str = self._str
+ longstring = _str("foobarbazetc")
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ def f(n, m):
+ assert n >= 0
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = longstring[m:n]
+ if m <= 5:
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [10, 10])
def test_streq_char(self):
- for somestr in ["?abcdefg", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- def f(n, m):
- assert n >= 0
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = somestr[:m]
- escape(s == "?")
- m -= 1
- return 42
- self.meta_interp(f, [6, 7])
- self.check_loops(newstr=0, newunicode=0)
-
-
-class TestOOtype(StringTests, OOJitMixin):
- CALL = "oosend"
- CALL_PURE = "oosend_pure"
+ _str = self._str
+ longstring = _str("?abcdefg")
+ somechar = _str("?")
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ def f(n, m):
+ assert n >= 0
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = longstring[:m]
+ escape(s == somechar)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 7])
+ self.check_loops(newstr=0, newunicode=0)
+
+
+#class TestOOtype(StringTests, OOJitMixin):
+# CALL = "oosend"
+# CALL_PURE = "oosend_pure"
class TestLLtype(StringTests, LLJitMixin):
CALL = "call"
CALL_PURE = "call_pure"
+
+class TestLLtypeUnicode(TestLLtype):
+ _str, _chr = unicode, unichr
+
+ def test_str2unicode(self):
+ _str = self._str
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ class Foo:
+ pass
+ @dont_look_inside
+ def escape(x):
+ assert x == _str("6y")
+ def f(n, m):
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ foo = Foo()
+ foo.y = chr(m)
+ foo.y = "y"
+ s = _str(str(n)) + _str(foo.y)
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 7])
+ self.check_loops(call=3, # str(), _str(), escape()
+ newunicode=1, unicodegetitem=0,
+ unicodesetitem=1, copyunicodecontent=1)
Modified: pypy/branch/jitffi/pypy/jit/tool/traceviewer.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/tool/traceviewer.py (original)
+++ pypy/branch/jitffi/pypy/jit/tool/traceviewer.py Mon Oct 18 11:22:20 2010
@@ -253,10 +253,10 @@
def main(loopfile, use_threshold, view=True):
countname = py.path.local(loopfile + '.count')
if countname.check():
- counts = [re.split('( 20 and use_threshold:
counts.threshold = l[-20]
Modified: pypy/branch/jitffi/pypy/module/__builtin__/interp_classobj.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/__builtin__/interp_classobj.py (original)
+++ pypy/branch/jitffi/pypy/module/__builtin__/interp_classobj.py Mon Oct 18 11:22:20 2010
@@ -2,9 +2,11 @@
from pypy.interpreter.error import OperationError, operationerrfmt
from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped, applevel
from pypy.interpreter.gateway import interp2app, ObjSpace
-from pypy.interpreter.typedef import TypeDef, make_weakref_descr
+from pypy.interpreter.typedef import TypeDef
from pypy.interpreter.argument import Arguments
from pypy.interpreter.baseobjspace import Wrappable
+from pypy.interpreter.typedef import GetSetProperty, descr_get_dict
+from pypy.interpreter.typedef import descr_set_dict
from pypy.rlib.rarithmetic import r_uint, intmask
from pypy.rlib.objectmodel import compute_identity_hash
from pypy.rlib.debug import make_sure_not_resized
@@ -57,6 +59,14 @@
self.bases_w = bases
self.w_dict = w_dict
+ def instantiate(self, space):
+ cache = space.fromcache(Cache)
+ if self.lookup(space, '__del__') is not None:
+ w_inst = cache.cls_with_del(space, self)
+ else:
+ w_inst = cache.cls_without_del(space, self)
+ return w_inst
+
def getdict(self):
return self.w_dict
@@ -100,15 +110,15 @@
return False
@jit.unroll_safe
- def lookup(self, space, w_attr):
+ def lookup(self, space, attr):
# returns w_value or interplevel None
- w_result = space.finditem(self.w_dict, w_attr)
+ w_result = space.finditem_str(self.w_dict, attr)
if w_result is not None:
return w_result
for base in self.bases_w:
# XXX fix annotation of bases_w to be a list of W_ClassObjects
assert isinstance(base, W_ClassObject)
- w_result = base.lookup(space, w_attr)
+ w_result = base.lookup(space, attr)
if w_result is not None:
return w_result
return None
@@ -122,7 +132,7 @@
return space.wrap(self.name)
elif name == "__bases__":
return space.newtuple(self.bases_w)
- w_value = self.lookup(space, w_attr)
+ w_value = self.lookup(space, name)
if w_value is None:
raise operationerrfmt(
space.w_AttributeError,
@@ -147,7 +157,7 @@
self.setbases(space, w_value)
return
elif name == "__del__":
- if self.lookup(space, w_attr) is None:
+ if self.lookup(space, name) is None:
msg = ("a __del__ method added to an existing class "
"will not be called")
space.warn(msg, space.w_RuntimeWarning)
@@ -195,13 +205,20 @@
# NOT_RPYTHON
return '' % self.name
+class Cache:
+ def __init__(self, space):
+ from pypy.interpreter.typedef import _usersubclswithfeature
+ # evil
+ self.cls_without_del = _usersubclswithfeature(
+ space.config, W_InstanceObject, "dict", "weakref")
+ self.cls_with_del = _usersubclswithfeature(
+ space.config, self.cls_without_del, "del")
+
+
def class_descr_call(space, w_self, __args__):
self = space.interp_w(W_ClassObject, w_self)
- if self.lookup(space, space.wrap('__del__')) is not None:
- w_inst = W_InstanceObjectWithDel(space, self)
- else:
- w_inst = W_InstanceObject(space, self)
- w_init = w_inst.getattr_from_class(space, space.wrap('__init__'))
+ w_inst = self.instantiate(space)
+ w_init = w_inst.getattr_from_class(space, '__init__')
if w_init is not None:
w_result = space.call_args(w_init, __args__)
if not space.is_w(w_result, space.w_None):
@@ -234,7 +251,7 @@
def make_unary_instance_method(name):
def unaryop(self, space):
- w_meth = self.getattr(space, space.wrap(name), True)
+ w_meth = self.getattr(space, name, True)
return space.call_function(w_meth)
unaryop.func_name = name
return unaryop
@@ -242,7 +259,7 @@
def make_binary_returning_notimplemented_instance_method(name):
def binaryop(self, space, w_other):
try:
- w_meth = self.getattr(space, space.wrap(name), False)
+ w_meth = self.getattr(space, name, False)
except OperationError, e:
if e.match(space, space.w_AttributeError):
return space.w_NotImplemented
@@ -267,7 +284,7 @@
w_a = self
w_b = w_other
if w_a is self:
- w_meth = self.getattr(space, space.wrap(specialname), False)
+ w_meth = self.getattr(space, specialname, False)
if w_meth is None:
return space.w_NotImplemented
return space.call_function(w_meth, w_b)
@@ -278,7 +295,7 @@
def rbinaryop(self, space, w_other):
w_a, w_b = _coerce_helper(space, self, w_other)
if w_a is None or w_a is self:
- w_meth = self.getattr(space, space.wrap(rspecialname), False)
+ w_meth = self.getattr(space, rspecialname, False)
if w_meth is None:
return space.w_NotImplemented
return space.call_function(w_meth, w_other)
@@ -302,46 +319,34 @@
raise OperationError(
space.w_TypeError,
space.wrap("instance() first arg must be class"))
- if space.is_w(w_dict, space.w_None):
- w_dict = None
- elif not space.is_true(space.isinstance(w_dict, space.w_dict)):
- raise OperationError(
- space.w_TypeError,
- space.wrap("instance() second arg must be dictionary or None"))
- return W_InstanceObject(space, w_class, w_dict)
+ w_result = w_class.instantiate(space)
+ if not space.is_w(w_dict, space.w_None):
+ w_result.setdict(space, w_dict)
+ return w_result
class W_InstanceObject(Wrappable):
- def __init__(self, space, w_class, w_dict=None):
- if w_dict is None:
- w_dict = space.newdict(instance=True)
+ def __init__(self, space, w_class):
+ # note that user_setup is overridden by the typedef.py machinery
+ self.user_setup(space, space.gettypeobject(self.typedef))
assert isinstance(w_class, W_ClassObject)
self.w_class = w_class
- self.w_dict = w_dict
- self.space = space
-
- def getdict(self):
- return self.w_dict
- def setdict(self, space, w_dict):
- if (w_dict is None or
- not space.is_true(space.isinstance(w_dict, space.w_dict))):
- raise OperationError(
- space.w_TypeError,
- space.wrap("__dict__ must be a dictionary object"))
- self.w_dict = w_dict
+ def user_setup(self, space, w_subtype):
+ self.space = space
- def setclass(self, space, w_class):
+ def set_oldstyle_class(self, space, w_class):
if w_class is None or not isinstance(w_class, W_ClassObject):
raise OperationError(
space.w_TypeError,
space.wrap("__class__ must be set to a class"))
self.w_class = w_class
- def getattr_from_class(self, space, w_name):
+ def getattr_from_class(self, space, name):
# Look up w_name in the class dict, and call its __get__.
# This method ignores the instance dict and the __getattr__.
# Returns None if not found.
- w_value = self.w_class.lookup(space, w_name)
+ assert isinstance(name, str)
+ w_value = self.w_class.lookup(space, name)
if w_value is None:
return None
w_descr_get = space.lookup(w_value, '__get__')
@@ -349,19 +354,20 @@
return w_value
return space.call_function(w_descr_get, w_value, self, self.w_class)
- def getattr(self, space, w_name, exc=True):
+ def getattr(self, space, name, exc=True):
# Normal getattr rules: look up w_name in the instance dict,
# in the class dict, and then via a call to __getatttr__.
- w_result = space.finditem(self.w_dict, w_name)
+ assert isinstance(name, str)
+ w_result = self.getdictvalue(space, name)
if w_result is not None:
return w_result
- w_result = self.getattr_from_class(space, w_name)
+ w_result = self.getattr_from_class(space, name)
if w_result is not None:
return w_result
- w_meth = self.getattr_from_class(space, space.wrap('__getattr__'))
+ w_meth = self.getattr_from_class(space, '__getattr__')
if w_meth is not None:
try:
- return space.call_function(w_meth, w_name)
+ return space.call_function(w_meth, space.wrap(name))
except OperationError, e:
if not exc and e.match(space, space.w_AttributeError):
return None # eat the AttributeError
@@ -371,7 +377,7 @@
raise operationerrfmt(
space.w_AttributeError,
"%s instance has no attribute '%s'",
- self.w_class.name, space.str_w(w_name))
+ self.w_class.name, name)
else:
return None
@@ -379,44 +385,46 @@
name = space.str_w(w_attr)
if len(name) >= 8 and name[0] == '_':
if name == "__dict__":
- return self.w_dict
+ return self.getdict()
elif name == "__class__":
return self.w_class
- return self.getattr(space, w_attr)
+ return self.getattr(space, name)
def descr_setattr(self, space, w_name, w_value):
name = unwrap_attr(space, w_name)
- w_meth = self.getattr_from_class(space, space.wrap('__setattr__'))
+ w_meth = self.getattr_from_class(space, '__setattr__')
if name and name[0] == "_":
if name == '__dict__':
self.setdict(space, w_value)
return
if name == '__class__':
- self.setclass(space, w_value)
+ self.set_oldstyle_class(space, w_value)
return
if name == '__del__' and w_meth is None:
- if (not isinstance(self, W_InstanceObjectWithDel)
- and space.finditem(self.w_dict, w_name) is None):
+ cache = space.fromcache(Cache)
+ if (not isinstance(self, cache.cls_with_del)
+ and self.getdictvalue(space, '__del__') is None):
msg = ("a __del__ method added to an instance "
"with no __del__ in the class will not be called")
space.warn(msg, space.w_RuntimeWarning)
if w_meth is not None:
space.call_function(w_meth, w_name, w_value)
else:
- self.setdictvalue(space, name, w_value)
+ # bit obscure: appease normalization
+ self.setdictvalue(space, name, w_value, True)
def descr_delattr(self, space, w_name):
name = unwrap_attr(space, w_name)
if name and name[0] == "_":
if name == '__dict__':
# use setdict to raise the error
- self.setdict(space, None)
+ self.setdict(space, space.w_None)
return
elif name == '__class__':
- # use setclass to raise the error
- self.setclass(space, None)
+ # use set_oldstyle_class to raise the error
+ self.set_oldstyle_class(space, None)
return
- w_meth = self.getattr_from_class(space, space.wrap('__delattr__'))
+ w_meth = self.getattr_from_class(space, '__delattr__')
if w_meth is not None:
space.call_function(w_meth, w_name)
else:
@@ -427,7 +435,7 @@
self.w_class.name, name)
def descr_repr(self, space):
- w_meth = self.getattr(space, space.wrap('__repr__'), False)
+ w_meth = self.getattr(space, '__repr__', False)
if w_meth is None:
w_class = self.w_class
mod = w_class.get_module_string(space)
@@ -435,19 +443,19 @@
return space.call_function(w_meth)
def descr_str(self, space):
- w_meth = self.getattr(space, space.wrap('__str__'), False)
+ w_meth = self.getattr(space, '__str__', False)
if w_meth is None:
return self.descr_repr(space)
return space.call_function(w_meth)
def descr_unicode(self, space):
- w_meth = self.getattr(space, space.wrap('__unicode__'), False)
+ w_meth = self.getattr(space, '__unicode__', False)
if w_meth is None:
return self.descr_str(space)
return space.call_function(w_meth)
def descr_len(self, space):
- w_meth = self.getattr(space, space.wrap('__len__'))
+ w_meth = self.getattr(space, '__len__')
w_result = space.call_function(w_meth)
if space.is_true(space.isinstance(w_result, space.w_int)):
if space.is_true(space.lt(w_result, space.wrap(0))):
@@ -460,22 +468,22 @@
space.wrap("__len__() should return an int"))
def descr_getitem(self, space, w_key):
- w_meth = self.getattr(space, space.wrap('__getitem__'))
+ w_meth = self.getattr(space, '__getitem__')
return space.call_function(w_meth, w_key)
def descr_setitem(self, space, w_key, w_value):
- w_meth = self.getattr(space, space.wrap('__setitem__'))
+ w_meth = self.getattr(space, '__setitem__')
space.call_function(w_meth, w_key, w_value)
def descr_delitem(self, space, w_key):
- w_meth = self.getattr(space, space.wrap('__delitem__'))
+ w_meth = self.getattr(space, '__delitem__')
space.call_function(w_meth, w_key)
def descr_iter(self, space):
- w_meth = self.getattr(space, space.wrap('__iter__'), False)
+ w_meth = self.getattr(space, '__iter__', False)
if w_meth is not None:
return space.call_function(w_meth)
- w_meth = self.getattr(space, space.wrap('__getitem__'), False)
+ w_meth = self.getattr(space, '__getitem__', False)
if w_meth is None:
raise OperationError(
space.w_TypeError,
@@ -485,14 +493,14 @@
# don't see the point
def descr_getslice(self, space, w_i, w_j):
- w_meth = self.getattr(space, space.wrap('__getslice__'), False)
+ w_meth = self.getattr(space, '__getslice__', False)
if w_meth is not None:
return space.call_function(w_meth, w_i, w_j)
else:
return space.getitem(self, space.newslice(w_i, w_j, space.w_None))
def descr_setslice(self, space, w_i, w_j, w_sequence):
- w_meth = self.getattr(space, space.wrap('__setslice__'), False)
+ w_meth = self.getattr(space, '__setslice__', False)
if w_meth is not None:
space.call_function(w_meth, w_i, w_j, w_sequence)
else:
@@ -500,20 +508,20 @@
w_sequence)
def descr_delslice(self, space, w_i, w_j):
- w_meth = self.getattr(space, space.wrap('__delslice__'), False)
+ w_meth = self.getattr(space, '__delslice__', False)
if w_meth is not None:
space.call_function(w_meth, w_i, w_j)
else:
return space.delitem(self, space.newslice(w_i, w_j, space.w_None))
def descr_call(self, space, __args__):
- w_meth = self.getattr(space, space.wrap('__call__'))
+ w_meth = self.getattr(space, '__call__')
return space.call_args(w_meth, __args__)
def descr_nonzero(self, space):
- w_func = self.getattr(space, space.wrap('__nonzero__'), False)
+ w_func = self.getattr(space, '__nonzero__', False)
if w_func is None:
- w_func = self.getattr(space, space.wrap('__len__'), False)
+ w_func = self.getattr(space, '__len__', False)
if w_func is None:
return space.w_True
w_result = space.call_function(w_func)
@@ -537,7 +545,7 @@
not isinstance(w_b, W_InstanceObject)):
return space.cmp(w_a, w_b)
if isinstance(w_a, W_InstanceObject):
- w_func = w_a.getattr(space, space.wrap('__cmp__'), False)
+ w_func = w_a.getattr(space, '__cmp__', False)
if w_func is not None:
w_res = space.call_function(w_func, w_b)
if space.is_w(w_res, space.w_NotImplemented):
@@ -556,7 +564,7 @@
return space.wrap(-1)
return space.wrap(0)
if isinstance(w_b, W_InstanceObject):
- w_func = w_b.getattr(space, space.wrap('__cmp__'), False)
+ w_func = w_b.getattr(space, '__cmp__', False)
if w_func is not None:
w_res = space.call_function(w_func, w_a)
if space.is_w(w_res, space.w_NotImplemented):
@@ -577,10 +585,10 @@
return space.w_NotImplemented
def descr_hash(self, space):
- w_func = self.getattr(space, space.wrap('__hash__'), False)
+ w_func = self.getattr(space, '__hash__', False)
if w_func is None:
- w_eq = self.getattr(space, space.wrap('__eq__'), False)
- w_cmp = self.getattr(space, space.wrap('__cmp__'), False)
+ w_eq = self.getattr(space, '__eq__', False)
+ w_cmp = self.getattr(space, '__cmp__', False)
if w_eq is not None or w_cmp is not None:
raise OperationError(space.w_TypeError,
space.wrap("unhashable instance"))
@@ -595,7 +603,7 @@
return w_ret
def descr_index(self, space):
- w_func = self.getattr(space, space.wrap('__index__'), False)
+ w_func = self.getattr(space, '__index__', False)
if w_func is not None:
return space.call_function(w_func)
raise OperationError(
@@ -603,7 +611,7 @@
space.wrap("object cannot be interpreted as an index"))
def descr_contains(self, space, w_obj):
- w_func = self.getattr(space, space.wrap('__contains__'), False)
+ w_func = self.getattr(space, '__contains__', False)
if w_func is not None:
return space.wrap(space.is_true(space.call_function(w_func, w_obj)))
# now do it ourselves
@@ -626,7 +634,7 @@
w_a = self
w_b = w_other
if w_a is self:
- w_func = self.getattr(space, space.wrap('__pow__'), False)
+ w_func = self.getattr(space, '__pow__', False)
if w_func is not None:
return space.call_function(w_func, w_other)
return space.w_NotImplemented
@@ -634,7 +642,7 @@
return space.pow(w_a, w_b, space.w_None)
else:
# CPython also doesn't try coercion in this case
- w_func = self.getattr(space, space.wrap('__pow__'), False)
+ w_func = self.getattr(space, '__pow__', False)
if w_func is not None:
return space.call_function(w_func, w_other, w_modulo)
return space.w_NotImplemented
@@ -646,7 +654,7 @@
w_a = self
w_b = w_other
if w_a is self:
- w_func = self.getattr(space, space.wrap('__rpow__'), False)
+ w_func = self.getattr(space, '__rpow__', False)
if w_func is not None:
return space.call_function(w_func, w_other)
return space.w_NotImplemented
@@ -654,13 +662,13 @@
return space.pow(w_b, w_a, space.w_None)
else:
# CPython also doesn't try coercion in this case
- w_func = self.getattr(space, space.wrap('__rpow__'), False)
+ w_func = self.getattr(space, '__rpow__', False)
if w_func is not None:
return space.call_function(w_func, w_other, w_modulo)
return space.w_NotImplemented
def descr_next(self, space):
- w_func = self.getattr(space, space.wrap('next'), False)
+ w_func = self.getattr(space, 'next', False)
if w_func is None:
raise OperationError(space.w_TypeError,
space.wrap("instance has no next() method"))
@@ -669,10 +677,9 @@
def descr_del(self, space):
# Note that this is called from executioncontext.UserDelAction
# via the space.userdel() method.
- w_name = space.wrap('__del__')
- w_func = space.finditem(self.w_dict, w_name)
+ w_func = self.getdictvalue(space, '__del__')
if w_func is None:
- w_func = self.getattr_from_class(space, w_name)
+ w_func = self.getattr_from_class(space, '__del__')
if w_func is not None:
space.call_function(w_func)
@@ -717,6 +724,14 @@
rmeth,
unwrap_spec=["self", ObjSpace, W_Root])
+
+def descr_del_dict(space, w_inst):
+ # use setdict to raise the error
+ w_inst.setdict(space, space.w_None)
+
+dict_descr = GetSetProperty(descr_get_dict, descr_set_dict, descr_del_dict)
+dict_descr.name = '__dict__'
+
W_InstanceObject.typedef = TypeDef("instance",
__new__ = interp2app(descr_instance_new),
__getattribute__ = interp2app(W_InstanceObject.descr_getattribute,
@@ -766,12 +781,9 @@
unwrap_spec=['self', ObjSpace, W_Root, W_Root]),
next = interp2app(W_InstanceObject.descr_next,
unwrap_spec=['self', ObjSpace]),
- __weakref__ = make_weakref_descr(W_InstanceObject),
__del__ = interp2app(W_InstanceObject.descr_del,
unwrap_spec=['self', ObjSpace]),
+ __dict__ = dict_descr,
**rawdict
)
-
-class W_InstanceObjectWithDel(W_InstanceObject):
- def __del__(self):
- self._enqueue_for_destruction(self.space)
+W_InstanceObject.typedef.acceptable_as_base_class = False
Modified: pypy/branch/jitffi/pypy/module/__builtin__/test/test_classobj.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/__builtin__/test/test_classobj.py (original)
+++ pypy/branch/jitffi/pypy/module/__builtin__/test/test_classobj.py Mon Oct 18 11:22:20 2010
@@ -928,6 +928,31 @@
assert x is b
assert y == 5
+ def test_cant_subclass_instance(self):
+ class A:
+ pass
+ try:
+ class B(type(A())):
+ pass
+ except TypeError:
+ pass
+ else:
+ assert 0, "should have raised"
+
+ def test_dict_descriptor(self):
+ import sys
+ if not hasattr(sys, 'pypy_objspaceclass'):
+ skip("on CPython old-style instances don't have a __dict__ descriptor")
+ class A:
+ pass
+ a = A()
+ a.x = 1
+ descr = type(a).__dict__['__dict__']
+ assert descr.__get__(a) == {'x': 1}
+ descr.__set__(a, {'x': 2})
+ assert a.x == 2
+ raises(TypeError, descr.__delete__, a)
+
class AppTestOldStyleSharing(AppTestOldstyle):
def setup_class(cls):
@@ -966,3 +991,22 @@
a = 1
b = 2
assert self.is_strdict(A)
+
+class AppTestOldStyleMapDict(AppTestOldstyle):
+ def setup_class(cls):
+ cls.space = gettestobjspace(**{"objspace.std.withmapdict": True})
+ if option.runappdirect:
+ py.test.skip("can only be run on py.py")
+ def has_mapdict(space, w_inst):
+ return space.wrap(w_inst._get_mapdict_map() is not None)
+ cls.w_has_mapdict = cls.space.wrap(gateway.interp2app(has_mapdict))
+
+
+ def test_has_mapdict(self):
+ class A:
+ def __init__(self):
+ self.x = 42
+ a = A()
+ assert a.x == 42
+ assert self.has_mapdict(a)
+
Modified: pypy/branch/jitffi/pypy/module/_rawffi/array.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/_rawffi/array.py (original)
+++ pypy/branch/jitffi/pypy/module/_rawffi/array.py Mon Oct 18 11:22:20 2010
@@ -97,7 +97,15 @@
class W_ArrayInstance(W_DataInstance):
def __init__(self, space, shape, length, address=r_uint(0)):
- W_DataInstance.__init__(self, space, shape.size * length, address)
+ # Workaround for a strange behavior of libffi: make sure that
+ # we always have at least 8 bytes. For W_ArrayInstances that are
+ # used as the result value of a function call, ffi_call() writes
+ # 8 bytes into it even if the function's result type asks for less.
+ # This strange behavior is documented.
+ memsize = shape.size * length
+ if memsize < 8:
+ memsize = 8
+ W_DataInstance.__init__(self, space, memsize, address)
self.length = length
self.shape = shape
Modified: pypy/branch/jitffi/pypy/module/_weakref/interp__weakref.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/_weakref/interp__weakref.py (original)
+++ pypy/branch/jitffi/pypy/module/_weakref/interp__weakref.py Mon Oct 18 11:22:20 2010
@@ -7,7 +7,7 @@
import weakref
-class WeakrefLifeline(object):
+class WeakrefLifeline(W_Root):
def __init__(self, space):
self.space = space # this is here for W_Root.clear_all_weakrefs()
self.refs_weak = []
Modified: pypy/branch/jitffi/pypy/module/cpyext/classobject.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/cpyext/classobject.py (original)
+++ pypy/branch/jitffi/pypy/module/cpyext/classobject.py Mon Oct 18 11:22:20 2010
@@ -15,16 +15,20 @@
class is the class of new object. The dict parameter will be used as the
object's __dict__; if NULL, a new dictionary will be created for the
instance."""
- if not PyClass_Check(space, w_class):
+ if not isinstance(w_class, W_ClassObject):
return PyErr_BadInternalCall(space)
- return W_InstanceObject(space, w_class, w_dict)
+ w_result = w_class.instantiate(space)
+ if w_dict is not None:
+ w_result.setdict(space, w_dict)
+ return w_result
@cpython_api([PyObject, PyObject], PyObject, error=CANNOT_FAIL)
def _PyInstance_Lookup(space, w_instance, w_name):
+ name = space.str_w(w_name)
assert isinstance(w_instance, W_InstanceObject)
- w_result = space.finditem(w_instance.w_dict, w_name)
+ w_result = w_instance.getdictvalue(space, name)
if w_result is not None:
return w_result
- return w_instance.w_class.lookup(space, w_name)
+ return w_instance.w_class.lookup(space, name)
Modified: pypy/branch/jitffi/pypy/module/cpyext/test/test_unicodeobject.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/cpyext/test/test_unicodeobject.py (original)
+++ pypy/branch/jitffi/pypy/module/cpyext/test/test_unicodeobject.py Mon Oct 18 11:22:20 2010
@@ -177,13 +177,14 @@
encoded_charp = rffi.str2charp(encoded)
strict_charp = rffi.str2charp("strict")
if endian is not None:
- pendian = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
if endian < 0:
- pendian[0] = -1
+ value = -1
elif endian > 0:
- pendian[0] = 1
+ value = 1
else:
- pendian[0] = 0
+ value = 0
+ pendian = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
+ pendian[0] = rffi.cast(rffi.INT, value)
else:
pendian = None
Modified: pypy/branch/jitffi/pypy/module/gc/referents.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/gc/referents.py (original)
+++ pypy/branch/jitffi/pypy/module/gc/referents.py Mon Oct 18 11:22:20 2010
@@ -15,8 +15,16 @@
def try_cast_gcref_to_w_root(gcref):
w_obj = rgc.try_cast_gcref_to_instance(W_Root, gcref)
- if not we_are_translated() and not hasattr(w_obj, 'typedef'):
- w_obj = None
+ # Ignore the instances of W_Root that are not really valid as Python
+ # objects. There is e.g. WeakrefLifeline in module/_weakref that
+ # inherits from W_Root for internal reasons. Such instances don't
+ # have a typedef at all (or have a null typedef after translation).
+ if not we_are_translated():
+ if not hasattr(w_obj, 'typedef'):
+ return None
+ else:
+ if w_obj is None or not w_obj.typedef:
+ return None
return w_obj
def wrap(space, gcref):
Modified: pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/branch/jitffi/pypy/module/pypyjit/test/test_pypy_c.py Mon Oct 18 11:22:20 2010
@@ -276,7 +276,7 @@
assert len(ops) == 2
assert not ops[0].get_opnames("call")
assert not ops[0].get_opnames("new")
- assert len(ops[0].get_opnames("guard")) <= 7
+ assert len(ops[0].get_opnames("guard")) <= 3 # we get 2 withmapdict
assert not ops[1] # second LOOKUP_METHOD folded away
ops = self.get_by_bytecode("CALL_METHOD")
@@ -287,7 +287,7 @@
else:
assert not bytecode.get_opnames("call")
assert not bytecode.get_opnames("new")
- assert len(bytecode.get_opnames("guard")) <= 9
+ assert len(bytecode.get_opnames("guard")) <= 6
assert len(ops[1]) < len(ops[0])
ops = self.get_by_bytecode("LOAD_ATTR")
@@ -321,8 +321,8 @@
assert len(ops) == 2
assert not ops[0].get_opnames("call")
assert not ops[0].get_opnames("new")
- assert len(ops[0].get_opnames("guard")) <= 7
- assert len(ops[0].get_opnames("getfield")) < 6
+ assert len(ops[0].get_opnames("guard")) <= 3 # we get 2 withmapdict
+ assert len(ops[0].get_opnames("getfield")) <= 5 # we get <5 withmapdict
assert not ops[1] # second LOOKUP_METHOD folded away
def test_default_and_kw(self):
@@ -386,7 +386,7 @@
a.x = 2
i = i + a.x
return i
- ''', 67,
+ ''', 69,
([20], 20),
([31], 32))
@@ -394,7 +394,7 @@
self.get_by_bytecode("CALL_FUNCTION"))
assert not callA.get_opnames("call")
assert not callA.get_opnames("new")
- assert len(callA.get_opnames("guard")) <= 8
+ assert len(callA.get_opnames("guard")) <= 2
assert not callisinstance1.get_opnames("call")
assert not callisinstance1.get_opnames("new")
assert len(callisinstance1.get_opnames("guard")) <= 2
@@ -746,6 +746,8 @@
'''%(op1, float(a)/4.0, float(b)/4.0, op2), 109, ([], res))
def test_boolrewrite_ptr(self):
+ # XXX this test is way too imprecise in what it is actually testing
+ # it should count the number of guards instead
compares = ('a == b', 'b == a', 'a != b', 'b != a', 'a == c', 'c != b')
for e1 in compares:
for e2 in compares:
@@ -769,7 +771,7 @@
print
print 'Test:', e1, e2, n, res
self.run_source('''
- class tst:
+ class tst(object):
pass
def main():
a = tst()
@@ -851,6 +853,8 @@
''', 65, ([], 122880))
def test_array_intimg(self):
+ # XXX this test is way too imprecise in what it is actually testing
+ # it should count the number of guards instead
for tc, maxops in zip('ilILd', (67, 67, 69, 69, 61)):
res = 73574560
if tc in 'IL':
Modified: pypy/branch/jitffi/pypy/module/signal/interp_signal.py
==============================================================================
--- pypy/branch/jitffi/pypy/module/signal/interp_signal.py (original)
+++ pypy/branch/jitffi/pypy/module/signal/interp_signal.py Mon Oct 18 11:22:20 2010
@@ -1,7 +1,6 @@
from pypy.interpreter.error import OperationError
from pypy.interpreter.baseobjspace import W_Root, ObjSpace
from pypy.interpreter.executioncontext import AsyncAction, AbstractActionFlag
-from pypy.rlib.rarithmetic import LONG_BIT, intmask
import signal as cpy_signal
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.translator.tool.cbuild import ExternalCompilationInfo
@@ -64,8 +63,8 @@
class CheckSignalAction(AsyncAction):
"""An action that is automatically invoked when a signal is received."""
- # The C-level signal handler sets the highest bit of pypysig_occurred:
- bitmask = intmask(1 << (LONG_BIT-1))
+ # The C-level signal handler sets the bit 30 of pypysig_occurred:
+ bitmask = 1 << 30
def __init__(self, space):
AsyncAction.__init__(self, space)
Modified: pypy/branch/jitffi/pypy/objspace/std/celldict.py
==============================================================================
--- pypy/branch/jitffi/pypy/objspace/std/celldict.py (original)
+++ pypy/branch/jitffi/pypy/objspace/std/celldict.py Mon Oct 18 11:22:20 2010
@@ -45,7 +45,7 @@
if space.is_w(space.type(w_key), space.w_str):
self.impl_setitem_str(self.space.str_w(w_key), w_value)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
def impl_setitem_str(self, name, w_value, shadows_type=True):
self.getcell(name, True).w_value = w_value
@@ -66,7 +66,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_length(self):
# inefficient, but do we care?
@@ -85,7 +85,7 @@
elif _is_sane_hash(space, w_lookup_type):
return None
else:
- return self._as_rdict().getitem(w_lookup)
+ return self._as_rdict().impl_fallback_getitem(w_lookup)
def impl_getitem_str(self, lookup):
res = self.getcell(lookup, False)
Modified: pypy/branch/jitffi/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/branch/jitffi/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/branch/jitffi/pypy/objspace/std/dictmultiobject.py Mon Oct 18 11:22:20 2010
@@ -102,17 +102,17 @@
else:
return None
- # _________________________________________________________________
+ # _________________________________________________________________
# implementation methods
def impl_getitem(self, w_key):
#return w_value or None
raise NotImplementedError("abstract base class")
- def impl_getitem_str(self, w_key):
+ def impl_getitem_str(self, key):
#return w_value or None
raise NotImplementedError("abstract base class")
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value, shadows_type=True):
raise NotImplementedError("abstract base class")
def impl_setitem(self, w_key, w_value):
@@ -120,7 +120,7 @@
def impl_delitem(self, w_key):
raise NotImplementedError("abstract base class")
-
+
def impl_length(self):
raise NotImplementedError("abstract base class")
@@ -310,7 +310,7 @@
if space.is_w(space.type(w_key), space.w_str):
self.impl_setitem_str(self.space.str_w(w_key), w_value)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
def impl_setitem_str(self, key, w_value, shadows_type=True):
self.content[key] = w_value
@@ -324,7 +324,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_length(self):
return len(self.content)
@@ -344,7 +344,7 @@
elif _is_sane_hash(space, w_lookup_type):
return None
else:
- return self._as_rdict().getitem(w_key)
+ return self._as_rdict().impl_fallback_getitem(w_key)
def impl_iter(self):
return StrIteratorImplementation(self.space, self)
@@ -414,7 +414,7 @@
StrDictImplementation.impl_setitem_str(
self, self.space.str_w(w_key), w_value, False)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
def impl_shadows_anything(self):
return (self._shadows_anything or
@@ -446,7 +446,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_get_builtin_indexed(self, i):
return self.shadowed[i]
Modified: pypy/branch/jitffi/pypy/objspace/std/model.py
==============================================================================
--- pypy/branch/jitffi/pypy/objspace/std/model.py (original)
+++ pypy/branch/jitffi/pypy/objspace/std/model.py Mon Oct 18 11:22:20 2010
@@ -18,6 +18,7 @@
"withsmallint" : ["smallintobject.W_SmallIntObject"],
"withstrslice" : ["strsliceobject.W_StringSliceObject"],
"withstrjoin" : ["strjoinobject.W_StringJoinObject"],
+ "withstrbuf" : ["strbufobject.W_StringBufferObject"],
"withrope" : ["ropeobject.W_RopeObject",
"ropeobject.W_RopeIterObject"],
"withropeunicode": ["ropeunicodeobject.W_RopeUnicodeObject",
@@ -75,6 +76,7 @@
from pypy.objspace.std import ropeunicodeobject
from pypy.objspace.std import strsliceobject
from pypy.objspace.std import strjoinobject
+ from pypy.objspace.std import strbufobject
from pypy.objspace.std import typeobject
from pypy.objspace.std import sliceobject
from pypy.objspace.std import longobject
@@ -222,6 +224,13 @@
(unicodeobject.W_UnicodeObject,
strjoinobject.delegate_join2unicode)
]
+ elif config.objspace.std.withstrbuf:
+ self.typeorder[strbufobject.W_StringBufferObject] += [
+ (stringobject.W_StringObject,
+ strbufobject.delegate_buf2str),
+ (unicodeobject.W_UnicodeObject,
+ strbufobject.delegate_buf2unicode)
+ ]
if config.objspace.std.withrangelist:
self.typeorder[rangeobject.W_RangeListObject] += [
(listobject.W_ListObject,
Modified: pypy/branch/jitffi/pypy/objspace/std/objspace.py
==============================================================================
--- pypy/branch/jitffi/pypy/objspace/std/objspace.py (original)
+++ pypy/branch/jitffi/pypy/objspace/std/objspace.py Mon Oct 18 11:22:20 2010
@@ -23,6 +23,7 @@
from pypy.objspace.std.listobject import W_ListObject
from pypy.objspace.std.longobject import W_LongObject
from pypy.objspace.std.noneobject import W_NoneObject
+from pypy.objspace.std.objectobject import W_ObjectObject
from pypy.objspace.std.ropeobject import W_RopeObject
from pypy.objspace.std.iterobject import W_SeqIterObject
from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject
@@ -317,9 +318,14 @@
w_subtype = w_type.check_user_subclass(w_subtype)
if cls.typedef.applevel_subclasses_base is not None:
cls = cls.typedef.applevel_subclasses_base
- subcls = get_unique_interplevel_subclass(
- self.config, cls, w_subtype.hasdict, w_subtype.nslots != 0,
- w_subtype.needsdel, w_subtype.weakrefable)
+ if (self.config.objspace.std.withmapdict and cls is W_ObjectObject
+ and not w_subtype.needsdel):
+ from pypy.objspace.std.mapdict import get_subclass_of_correct_size
+ subcls = get_subclass_of_correct_size(self, cls, w_subtype)
+ else:
+ subcls = get_unique_interplevel_subclass(
+ self.config, cls, w_subtype.hasdict, w_subtype.nslots != 0,
+ w_subtype.needsdel, w_subtype.weakrefable)
instance = instantiate(subcls)
assert isinstance(instance, cls)
instance.user_setup(self, w_subtype)
Modified: pypy/branch/jitffi/pypy/objspace/std/sharingdict.py
==============================================================================
--- pypy/branch/jitffi/pypy/objspace/std/sharingdict.py (original)
+++ pypy/branch/jitffi/pypy/objspace/std/sharingdict.py Mon Oct 18 11:22:20 2010
@@ -71,7 +71,7 @@
elif _is_sane_hash(space, w_lookup_type):
return None
else:
- return self._as_rdict().getitem(w_lookup)
+ return self._as_rdict().impl_fallback_getitem(w_lookup)
def impl_getitem_str(self, lookup):
i = self.structure.lookup_position(lookup)
@@ -84,7 +84,7 @@
if space.is_w(space.type(w_key), space.w_str):
self.impl_setitem_str(self.space.str_w(w_key), w_value)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
@unroll_safe
def impl_setitem_str(self, key, w_value, shadows_type=True):
@@ -132,7 +132,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_length(self):
return self.structure.length
Modified: pypy/branch/jitffi/pypy/objspace/std/stringobject.py
==============================================================================
--- pypy/branch/jitffi/pypy/objspace/std/stringobject.py (original)
+++ pypy/branch/jitffi/pypy/objspace/std/stringobject.py Mon Oct 18 11:22:20 2010
@@ -14,7 +14,7 @@
from pypy.rlib.rstring import StringBuilder, string_repeat
from pypy.interpreter.buffer import StringBuffer
-from pypy.objspace.std.stringtype import sliced, joined, wrapstr, wrapchar, \
+from pypy.objspace.std.stringtype import sliced, wrapstr, wrapchar, \
stringendswith, stringstartswith, joined2
from pypy.objspace.std.formatting import mod_format
Modified: pypy/branch/jitffi/pypy/objspace/std/stringtype.py
==============================================================================
--- pypy/branch/jitffi/pypy/objspace/std/stringtype.py (original)
+++ pypy/branch/jitffi/pypy/objspace/std/stringtype.py Mon Oct 18 11:22:20 2010
@@ -55,19 +55,14 @@
return W_StringSliceObject(s, start, stop)
return wrapstr(space, s[start:stop])
-def joined(space, strlist):
- assert not space.config.objspace.std.withrope
- if space.config.objspace.std.withstrjoin:
- from pypy.objspace.std.strjoinobject import W_StringJoinObject
- return W_StringJoinObject(strlist)
- else:
- return wrapstr(space, "".join(strlist))
-
def joined2(space, str1, str2):
assert not space.config.objspace.std.withrope
if space.config.objspace.std.withstrjoin:
from pypy.objspace.std.strjoinobject import W_StringJoinObject
return W_StringJoinObject([str1, str2])
+ elif space.config.objspace.std.withstrbuf:
+ from pypy.objspace.std.strbufobject import joined2
+ return joined2(str1, str2)
else:
return wrapstr(space, str1 + str2)
Modified: pypy/branch/jitffi/pypy/objspace/std/test/test_dictmultiobject.py
==============================================================================
--- pypy/branch/jitffi/pypy/objspace/std/test/test_dictmultiobject.py (original)
+++ pypy/branch/jitffi/pypy/objspace/std/test/test_dictmultiobject.py Mon Oct 18 11:22:20 2010
@@ -602,6 +602,15 @@
classofinstance=classofinstance,
from_strdict_shared=from_strdict_shared)
+ def finditem_str(self, w_dict, s):
+ return w_dict.getitem_str(s) # assume it's a multidict
+
+ def setitem_str(self, w_dict, s, w_value):
+ return w_dict.setitem_str(s, w_value) # assume it's a multidict
+
+ def delitem(self, w_dict, w_s):
+ return w_dict.delitem(w_s) # assume it's a multidict
+
def allocate_instance(self, cls, type):
return object.__new__(cls)
@@ -611,7 +620,7 @@
w_StopIteration = StopIteration
w_None = None
StringObjectCls = FakeString
- w_dict = None
+ w_dict = W_DictMultiObject
iter = iter
fixedview = list
listview = list
@@ -687,6 +696,14 @@
assert self.impl.length() == 0
self.check_not_devolved()
+ def test_clear(self):
+ self.fill_impl()
+ assert self.impl.length() == 2
+ self.impl.clear()
+ assert self.impl.length() == 0
+ self.check_not_devolved()
+
+
def test_keys(self):
self.fill_impl()
keys = self.impl.keys()
Modified: pypy/branch/jitffi/pypy/objspace/std/test/test_shadowtracking.py
==============================================================================
--- pypy/branch/jitffi/pypy/objspace/std/test/test_shadowtracking.py (original)
+++ pypy/branch/jitffi/pypy/objspace/std/test/test_shadowtracking.py Mon Oct 18 11:22:20 2010
@@ -3,7 +3,8 @@
class TestShadowTracking(object):
def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withshadowtracking": True})
+ cls.space = gettestobjspace(**{"objspace.std.withshadowtracking": True,
+ "objspace.std.withmapdict": False})
def test_simple_shadowing(self):
space = self.space
Modified: pypy/branch/jitffi/pypy/objspace/std/typeobject.py
==============================================================================
--- pypy/branch/jitffi/pypy/objspace/std/typeobject.py (original)
+++ pypy/branch/jitffi/pypy/objspace/std/typeobject.py Mon Oct 18 11:22:20 2010
@@ -75,7 +75,9 @@
'weakrefable',
'hasdict',
'nslots',
- 'instancetypedef']
+ 'instancetypedef',
+ 'terminator',
+ ]
# for config.objspace.std.getattributeshortcut
# (False is a conservative default, fixed during real usage)
@@ -116,6 +118,12 @@
# dict_w of any of the types in the mro changes, or if the mro
# itself changes
w_self._version_tag = VersionTag()
+ if space.config.objspace.std.withmapdict:
+ from pypy.objspace.std.mapdict import DictTerminator, NoDictTerminator
+ if w_self.hasdict:
+ w_self.terminator = DictTerminator(space, w_self)
+ else:
+ w_self.terminator = NoDictTerminator(space, w_self)
def mutated(w_self):
space = w_self.space
Modified: pypy/branch/jitffi/pypy/rlib/_rsocket_rffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/_rsocket_rffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/_rsocket_rffi.py Mon Oct 18 11:22:20 2010
@@ -324,10 +324,11 @@
('sll_hatype', rffi.INT),
('sll_addr', rffi.CFixedArray(rffi.CHAR, 8)),
('sll_halen', rffi.INT)],
- )
+ ifdef='AF_PACKET')
CConfig.ifreq = platform.Struct('struct ifreq', [('ifr_ifindex', rffi.INT),
- ('ifr_name', rffi.CFixedArray(rffi.CHAR, 8))])
+ ('ifr_name', rffi.CFixedArray(rffi.CHAR, 8))],
+ ifdef='AF_PACKET')
if _WIN32:
CConfig.WSAEVENT = platform.SimpleType('WSAEVENT', rffi.VOIDP)
@@ -532,8 +533,9 @@
socketpair_t = rffi.CArray(socketfd_type)
socketpair = external('socketpair', [rffi.INT, rffi.INT, rffi.INT,
lltype.Ptr(socketpair_t)], rffi.INT)
- ioctl = external('ioctl', [socketfd_type, rffi.INT, lltype.Ptr(ifreq)],
- rffi.INT)
+ if ifreq is not None:
+ ioctl = external('ioctl', [socketfd_type, rffi.INT, lltype.Ptr(ifreq)],
+ rffi.INT)
if _WIN32:
ioctlsocket = external('ioctlsocket',
Modified: pypy/branch/jitffi/pypy/rlib/clibffi.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/clibffi.py (original)
+++ pypy/branch/jitffi/pypy/rlib/clibffi.py Mon Oct 18 11:22:20 2010
@@ -17,7 +17,6 @@
import sys
import ctypes.util
-
# maaaybe isinstance here would be better. Think
_MSVC = platform.name == "msvc"
_MINGW = platform.name == "mingw32"
Modified: pypy/branch/jitffi/pypy/rlib/rmmap.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/rmmap.py (original)
+++ pypy/branch/jitffi/pypy/rlib/rmmap.py Mon Oct 18 11:22:20 2010
@@ -50,7 +50,7 @@
constant_names = ['MAP_SHARED', 'MAP_PRIVATE',
'PROT_READ', 'PROT_WRITE',
'MS_SYNC']
- opt_constant_names = ['MAP_ANON', 'MAP_ANONYMOUS',
+ opt_constant_names = ['MAP_ANON', 'MAP_ANONYMOUS', 'MAP_NORESERVE',
'PROT_EXEC',
'MAP_DENYWRITE', 'MAP_EXECUTABLE']
for name in constant_names:
Modified: pypy/branch/jitffi/pypy/rlib/rstring.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/rstring.py (original)
+++ pypy/branch/jitffi/pypy/rlib/rstring.py Mon Oct 18 11:22:20 2010
@@ -54,6 +54,7 @@
self.l = []
def append(self, s):
+ assert isinstance(s, self._type)
self.l.append(s)
def append_slice(self, s, start, end):
@@ -63,11 +64,16 @@
def append_multiple_char(self, c, times):
self.l.append(c * times)
+ def getlength(self):
+ return len(self.build())
+
class StringBuilder(AbstractStringBuilder):
+ _type = str
def build(self):
return "".join(self.l)
class UnicodeBuilder(AbstractStringBuilder):
+ _type = unicode
def build(self):
return u''.join(self.l)
@@ -121,9 +127,12 @@
assert s_times.nonneg
return s_None
+ def method_getlength(self):
+ return SomeInteger(nonneg=True)
+
def method_build(self):
return SomeString()
-
+
def rtyper_makerepr(self, rtyper):
return rtyper.type_system.rbuilder.stringbuilder_repr
@@ -146,6 +155,9 @@
assert s_times.nonneg
return s_None
+ def method_getlength(self):
+ return SomeInteger(nonneg=True)
+
def method_build(self):
return SomeUnicodeString()
Modified: pypy/branch/jitffi/pypy/rlib/test/test_rsocket.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_rsocket.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_rsocket.py Mon Oct 18 11:22:20 2010
@@ -437,3 +437,31 @@
foo = self.serv.accept()
py.test.raises(SocketError, raise_error)
+def _test_cond_include(cond):
+ # Test that _rsocket_rffi is importable even on platforms where
+ # AF_PACKET or AF_NETLINK is not defined.
+ import re
+ from pypy.rlib import _rsocket_rffi
+ srcfile = _rsocket_rffi.__file__
+ if srcfile.lower().endswith('c') or srcfile.lower().endswith('o'):
+ srcfile = srcfile[:-1] # .pyc => .py
+ assert srcfile.lower().endswith('.py')
+ sourcelines = open(srcfile, 'rb').read().splitlines()
+ found = False
+ for i, line in enumerate(sourcelines):
+ line2 = re.sub(r"(\s*COND_HEADER\s*=)",
+ r"\1'#undef %s\\n'+" % cond,
+ line)
+ if line2 != line:
+ found = True
+ sourcelines[i] = line2
+ assert found
+ d = {}
+ sourcelines.append('')
+ exec '\n'.join(sourcelines) in d
+
+def test_no_AF_PACKET():
+ _test_cond_include('AF_PACKET')
+
+def test_no_AF_NETLINK():
+ _test_cond_include('AF_NETLINK')
Modified: pypy/branch/jitffi/pypy/rlib/test/test_rstring.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_rstring.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_rstring.py Mon Oct 18 11:22:20 2010
@@ -29,6 +29,7 @@
s = StringBuilder()
s.append("a")
s.append("abc")
+ assert s.getlength() == len('aabc')
s.append("a")
s.append_slice("abc", 1, 2)
s.append_multiple_char('d', 4)
@@ -39,6 +40,7 @@
s.append(u'a')
s.append(u'abc')
s.append_slice(u'abcdef', 1, 2)
+ assert s.getlength() == len('aabcb')
s.append_multiple_char('d', 4)
assert s.build() == 'aabcbdddd'
assert isinstance(s.build(), unicode)
Modified: pypy/branch/jitffi/pypy/rlib/test/test_runicode.py
==============================================================================
--- pypy/branch/jitffi/pypy/rlib/test/test_runicode.py (original)
+++ pypy/branch/jitffi/pypy/rlib/test/test_runicode.py Mon Oct 18 11:22:20 2010
@@ -76,7 +76,7 @@
assert start == startingpos
assert stop == endingpos
return u"42424242", stop
- return "", endingpos
+ return u"", endingpos
decoder = self.getdecoder(encoding)
if addstuff:
s += "some rest in ascii"
Modified: pypy/branch/jitffi/pypy/rpython/lltypesystem/ll2ctypes.py
==============================================================================
--- pypy/branch/jitffi/pypy/rpython/lltypesystem/ll2ctypes.py (original)
+++ pypy/branch/jitffi/pypy/rpython/lltypesystem/ll2ctypes.py Mon Oct 18 11:22:20 2010
@@ -69,7 +69,7 @@
PIECESIZE = 0x08000000
PIECES = 10
m = rmmap.mmap(-1, PIECES * PIECESIZE,
- rmmap.MAP_PRIVATE|rmmap.MAP_ANONYMOUS,
+ rmmap.MAP_PRIVATE|rmmap.MAP_ANONYMOUS|rmmap.MAP_NORESERVE,
rmmap.PROT_READ|rmmap.PROT_WRITE)
m.close = lambda : None # leak instead of giving a spurious
# error at CPython's shutdown
Modified: pypy/branch/jitffi/pypy/rpython/lltypesystem/rbuilder.py
==============================================================================
--- pypy/branch/jitffi/pypy/rpython/lltypesystem/rbuilder.py (original)
+++ pypy/branch/jitffi/pypy/rpython/lltypesystem/rbuilder.py Mon Oct 18 11:22:20 2010
@@ -100,6 +100,10 @@
ll_builder.used = used
@staticmethod
+ def ll_getlength(ll_builder):
+ return ll_builder.used
+
+ @staticmethod
def ll_build(ll_builder):
final_size = ll_builder.used
assert final_size >= 0
Modified: pypy/branch/jitffi/pypy/rpython/lltypesystem/rclass.py
==============================================================================
--- pypy/branch/jitffi/pypy/rpython/lltypesystem/rclass.py (original)
+++ pypy/branch/jitffi/pypy/rpython/lltypesystem/rclass.py Mon Oct 18 11:22:20 2010
@@ -329,16 +329,33 @@
fields['__class__'] = 'typeptr', get_type_repr(self.rtyper)
else:
# instance attributes
- if llfields is None:
- llfields = []
attrs = self.classdef.attrs.items()
attrs.sort()
+ myllfields = []
for name, attrdef in attrs:
if not attrdef.readonly:
r = self.rtyper.getrepr(attrdef.s_value)
mangled_name = 'inst_' + name
fields[name] = mangled_name, r
- llfields.append((mangled_name, r.lowleveltype))
+ myllfields.append((mangled_name, r.lowleveltype))
+
+ # Sort the instance attributes by decreasing "likely size",
+ # as reported by rffi.sizeof(), to minimize padding holes in C.
+ # Fields of the same size are sorted by name (by attrs.sort()
+ # above) just to minimize randomness.
+ def keysize((_, T)):
+ if T is lltype.Void:
+ return None
+ from pypy.rpython.lltypesystem.rffi import sizeof
+ try:
+ return -sizeof(T)
+ except StandardError:
+ return None
+ myllfields.sort(key = keysize)
+ if llfields is None:
+ llfields = myllfields
+ else:
+ llfields = llfields + myllfields
self.rbase = getinstancerepr(self.rtyper, self.classdef.basedef,
self.gcflavor)
Modified: pypy/branch/jitffi/pypy/rpython/module/ll_time.py
==============================================================================
--- pypy/branch/jitffi/pypy/rpython/module/ll_time.py (original)
+++ pypy/branch/jitffi/pypy/rpython/module/ll_time.py Mon Oct 18 11:22:20 2010
@@ -108,7 +108,7 @@
errcode = -1
if self.GETTIMEOFDAY_NO_TZ:
- errcode = g_gettimeofday(t)
+ errcode = c_gettimeofday(t)
else:
errcode = c_gettimeofday(t, void)
Modified: pypy/branch/jitffi/pypy/rpython/rbuilder.py
==============================================================================
--- pypy/branch/jitffi/pypy/rpython/rbuilder.py (original)
+++ pypy/branch/jitffi/pypy/rpython/rbuilder.py Mon Oct 18 11:22:20 2010
@@ -36,8 +36,12 @@
hop.exception_cannot_occur()
return hop.gendirectcall(self.ll_append_multiple_char, *vlist)
+ def rtype_method_getlength(self, hop):
+ vlist = hop.inputargs(self)
+ hop.exception_cannot_occur()
+ return hop.gendirectcall(self.ll_getlength, *vlist)
+
def rtype_method_build(self, hop):
vlist = hop.inputargs(self)
hop.exception_cannot_occur()
return hop.gendirectcall(self.ll_build, *vlist)
-
Modified: pypy/branch/jitffi/pypy/rpython/test/test_rbuilder.py
==============================================================================
--- pypy/branch/jitffi/pypy/rpython/test/test_rbuilder.py (original)
+++ pypy/branch/jitffi/pypy/rpython/test/test_rbuilder.py Mon Oct 18 11:22:20 2010
@@ -1,4 +1,4 @@
-
+import py
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.rpython.lltypesystem.rbuilder import *
from pypy.rpython.annlowlevel import llstr, hlstr
@@ -55,8 +55,29 @@
assert res == 'aabcabcdefbuuuu'
assert isinstance(res, unicode)
+ def test_string_getlength(self):
+ def func():
+ s = StringBuilder()
+ s.append("a")
+ s.append("abc")
+ return s.getlength()
+ res = self.interpret(func, [])
+ assert res == 4
+
+ def test_unicode_getlength(self):
+ def func():
+ s = UnicodeBuilder()
+ s.append(u"a")
+ s.append(u"abc")
+ return s.getlength()
+ res = self.interpret(func, [])
+ assert res == 4
+
class TestLLtype(BaseTestStringBuilder, LLRtypeMixin):
pass
class TestOOtype(BaseTestStringBuilder, OORtypeMixin):
- pass
+ def test_string_getlength(self):
+ py.test.skip("getlength(): not implemented on ootype")
+ def test_unicode_getlength(self):
+ py.test.skip("getlength(): not implemented on ootype")
Modified: pypy/branch/jitffi/pypy/rpython/test/test_rclass.py
==============================================================================
--- pypy/branch/jitffi/pypy/rpython/test/test_rclass.py (original)
+++ pypy/branch/jitffi/pypy/rpython/test/test_rclass.py Mon Oct 18 11:22:20 2010
@@ -3,7 +3,7 @@
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.lltypesystem.lltype import *
from pypy.rpython.ootypesystem import ootype
-from pypy.rlib.rarithmetic import intmask
+from pypy.rlib.rarithmetic import intmask, r_longlong
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.objspace.flow.model import summary
@@ -1012,6 +1012,40 @@
res = self.interpret(f, [5])
assert res == 0
+ def test_order_of_fields(self):
+ class A(object):
+ pass
+ def f(n):
+ a = A()
+ a.as_int = n
+ a.as_char = chr(n)
+ a.as_unichar = unichr(n)
+ a.as_double = n + 0.5
+ a.as_bool = bool(n)
+ a.as_void = None
+ a.as_longlong = r_longlong(n)
+ a.as_reference = A()
+ return a
+
+ res = self.interpret(f, [5])
+ names = list(typeOf(res).TO._names)
+ i = names.index('inst_as_int')
+ c = names.index('inst_as_char')
+ u = names.index('inst_as_unichar')
+ d = names.index('inst_as_double')
+ b = names.index('inst_as_bool')
+ v = names.index('inst_as_void')
+ l = names.index('inst_as_longlong')
+ r = names.index('inst_as_reference')
+ assert v == 1 # void fields are first
+ assert sorted([c, b]) == [7, 8]
+ if sys.maxint == 2147483647:
+ assert sorted([u, i, r]) == [4, 5, 6] # 32-bit types
+ assert sorted([d, l]) == [2, 3] # 64-bit types
+ else:
+ assert sorted([u]) == [6] # 32-bit types
+ assert sorted([i, r, d, l]) == [2, 3, 4, 5] # 64-bit types
+
class TestOOtype(BaseTestRclass, OORtypeMixin):
Modified: pypy/branch/jitffi/pypy/translator/c/gcc/test/elf/track5.s
==============================================================================
--- pypy/branch/jitffi/pypy/translator/c/gcc/test/elf/track5.s (original)
+++ pypy/branch/jitffi/pypy/translator/c/gcc/test/elf/track5.s Mon Oct 18 11:22:20 2010
@@ -44,7 +44,7 @@
addl %eax, %ebx
jmp .L1221
.L1227:
- call RPyAbort
+ ;;call RPyAbort
cmpl 12(%esi), %ebx
jb .L1229
addl $20, %esp
Modified: pypy/branch/jitffi/pypy/translator/c/gcc/trackgcroot.py
==============================================================================
--- pypy/branch/jitffi/pypy/translator/c/gcc/trackgcroot.py (original)
+++ pypy/branch/jitffi/pypy/translator/c/gcc/trackgcroot.py Mon Oct 18 11:22:20 2010
@@ -46,6 +46,7 @@
self.findlabels()
self.parse_instructions()
try:
+ self.trim_unreachable_instructions()
self.find_noncollecting_calls()
if not self.list_collecting_call_insns():
return []
@@ -122,6 +123,26 @@
assert label not in self.labels, "duplicate label: %s" % label
self.labels[label] = Label(label, lineno)
+ def trim_unreachable_instructions(self):
+ reached = set([self.insns[0]])
+ prevlen = 0
+ while len(reached) > prevlen:
+ prevlen = len(reached)
+ for insn in self.insns:
+ if insn not in reached:
+ for previnsn in insn.previous_insns:
+ if previnsn in reached:
+ # this instruction is reachable too
+ reached.add(insn)
+ break
+ # now kill all unreachable instructions
+ i = 0
+ while i < len(self.insns):
+ if self.insns[i] in reached:
+ i += 1
+ else:
+ del self.insns[i]
+
def find_noncollecting_calls(self):
cannot_collect = {}
for line in self.lines:
@@ -752,7 +773,7 @@
target, = sources
if target in self.FUNCTIONS_NOT_RETURNING:
- return [InsnStop(target), InsnCannotFollowEsp()]
+ return [InsnStop(target)]
if self.format == 'mingw32' and target == '__alloca':
# in functions with large stack requirements, windows
# needs a call to _alloca(), to turn reserved pages
Modified: pypy/branch/jitffi/pypy/translator/c/src/signals.h
==============================================================================
--- pypy/branch/jitffi/pypy/translator/c/src/signals.h (original)
+++ pypy/branch/jitffi/pypy/translator/c/src/signals.h Mon Oct 18 11:22:20 2010
@@ -6,20 +6,6 @@
#include
-#ifndef LONG_MAX
-#if SIZEOF_LONG == 4
-#define LONG_MAX 0X7FFFFFFFL
-#elif SIZEOF_LONG == 8
-#define LONG_MAX 0X7FFFFFFFFFFFFFFFL
-#else
-#error "could not set LONG_MAX in pyport.h"
-#endif
-#endif
-
-#ifndef LONG_MIN
-#define LONG_MIN (-LONG_MAX-1)
-#endif
-
#include
#ifdef MS_WINDOWS
@@ -28,10 +14,6 @@
#include
-#ifndef SIG_ERR
-#define SIG_ERR ((PyOS_sighandler_t)(-1))
-#endif
-
#if defined(PYOS_OS2) && !defined(PYCC_GCC)
#define NSIG 12
#include
@@ -65,11 +47,11 @@
/* utility to poll for signals that arrived */
int pypysig_poll(void); /* => signum or -1 */
-/* When a signal is received, the high bit of pypysig_occurred is set.
- After all signals are processed by pypysig_poll(), the high bit is
+/* When a signal is received, the bit 30 of pypysig_occurred is set.
+ After all signals are processed by pypysig_poll(), the bit 30 is
cleared again. The variable is exposed and RPython code is free to
use the other bits in any way. */
-#define PENDING_SIGNAL_BIT (LONG_MIN) /* high bit */
+#define PENDING_SIGNAL_BIT (1 << 30)
/* This is a struct for the JIT. See interp_signal.py. */
struct pypysig_long_struct {
long value;
From antocuni at codespeak.net Mon Oct 18 11:27:55 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 18 Oct 2010 11:27:55 +0200 (CEST)
Subject: [pypy-svn] r78034 - in pypy/branch/jitffi/pypy/jit/codewriter: .
test
Message-ID: <20101018092755.C1037282B90@codespeak.net>
Author: antocuni
Date: Mon Oct 18 11:27:54 2010
New Revision: 78034
Modified:
pypy/branch/jitffi/pypy/jit/codewriter/effectinfo.py
pypy/branch/jitffi/pypy/jit/codewriter/test/test_effectinfo.py
Log:
add a test to check that we don't have any duplicate in EffectInfo.OS_*. Also, move libffi stuff at the bottom
Modified: pypy/branch/jitffi/pypy/jit/codewriter/effectinfo.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/codewriter/effectinfo.py (original)
+++ pypy/branch/jitffi/pypy/jit/codewriter/effectinfo.py Mon Oct 18 11:27:54 2010
@@ -19,10 +19,7 @@
OS_NONE = 0 # normal case, no oopspec
OS_ARRAYCOPY = 1 # "list.ll_arraycopy"
OS_STR2UNICODE = 2 # "str.str2unicode"
- OS_LIBFFI_PREPARE = 15
- OS_LIBFFI_PUSH_ARG = 16
- OS_LIBFFI_CALL = 17
-
+ #
OS_STR_CONCAT = 22 # "stroruni.concat"
OS_STR_SLICE = 23 # "stroruni.slice"
OS_STR_EQUAL = 24 # "stroruni.equal"
@@ -33,7 +30,7 @@
OS_STREQ_NONNULL_CHAR = 29 # s1 == char (assert s1!=NULL)
OS_STREQ_CHECKNULL_CHAR = 30 # s1!=NULL and s1==char
OS_STREQ_LENGTHOK = 31 # s1 == s2 (assert len(s1)==len(s2))
-
+ #
OS_UNI_CONCAT = 42 #
OS_UNI_SLICE = 43 #
OS_UNI_EQUAL = 44 #
@@ -45,6 +42,10 @@
OS_UNIEQ_CHECKNULL_CHAR = 50 # STR, in the same order)
OS_UNIEQ_LENGTHOK = 51 #
_OS_offset_uni = OS_UNI_CONCAT - OS_STR_CONCAT
+ #
+ OS_LIBFFI_PREPARE = 60
+ OS_LIBFFI_PUSH_ARG = 61
+ OS_LIBFFI_CALL = 62
def __new__(cls, readonly_descrs_fields,
write_descrs_fields, write_descrs_arrays,
Modified: pypy/branch/jitffi/pypy/jit/codewriter/test/test_effectinfo.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/codewriter/test/test_effectinfo.py (original)
+++ pypy/branch/jitffi/pypy/jit/codewriter/test/test_effectinfo.py Mon Oct 18 11:27:54 2010
@@ -1,7 +1,8 @@
from pypy.rpython.lltypesystem.rclass import OBJECT
from pypy.rpython.lltypesystem import lltype
from pypy.rpython.ootypesystem import ootype
-from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze
+from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze,\
+ EffectInfo
class FakeCPU:
def fielddescrof(self, T, fieldname):
@@ -9,6 +10,14 @@
def arraydescrof(self, A):
return ('arraydescr', A)
+def test_no_oopspec_duplicate():
+ # check that all the various EffectInfo.OS_* have unique values
+ oopspecs = set()
+ for name, value in EffectInfo.__dict__.iteritems():
+ if name.startswith('OS_'):
+ assert value not in oopspecs
+ oopspecs.add(value)
+
def test_include_read_field():
S = lltype.GcStruct("S", ("a", lltype.Signed))
effects = frozenset([("readstruct", lltype.Ptr(S), "a")])
From antocuni at codespeak.net Mon Oct 18 11:32:29 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 18 Oct 2010 11:32:29 +0200 (CEST)
Subject: [pypy-svn] r78035 -
pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt
Message-ID: <20101018093229.55E75282B90@codespeak.net>
Author: antocuni
Date: Mon Oct 18 11:32:27 2010
New Revision: 78035
Modified:
pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
Log:
try to make the annotator happier (should not be necessary, but tests misteriously fail with buildbot :-( )
Modified: pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py (original)
+++ pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py Mon Oct 18 11:32:27 2010
@@ -11,6 +11,8 @@
argtypes = None
restype = None
descr = None
+ prepare_op = None
+ force_token_op = None
def __init__(self, funcval, cpu, prepare_op):
self.funcval = funcval
@@ -18,7 +20,6 @@
argtypes, restype = self._get_signature(funcval)
self.descr = cpu.calldescrof_dynamic(argtypes, restype)
self.prepare_op = prepare_op
- self.force_token_op = None
def _get_signature(self, funcval):
"""
From antocuni at codespeak.net Mon Oct 18 13:18:15 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 18 Oct 2010 13:18:15 +0200 (CEST)
Subject: [pypy-svn] r78036 - pypy/branch/jitffi/pypy/doc/config
Message-ID: <20101018111815.EA178282B90@codespeak.net>
Author: antocuni
Date: Mon Oct 18 13:18:13 2010
New Revision: 78036
Added:
pypy/branch/jitffi/pypy/doc/config/objspace.usemodules._ffi.txt (contents, props changed)
Log:
add doc for _ffi
Added: pypy/branch/jitffi/pypy/doc/config/objspace.usemodules._ffi.txt
==============================================================================
--- (empty file)
+++ pypy/branch/jitffi/pypy/doc/config/objspace.usemodules._ffi.txt Mon Oct 18 13:18:13 2010
@@ -0,0 +1 @@
+Applevel interface to libffi. It is more high level than _rawffi, and most importantly it is JIT friendly
From afa at codespeak.net Mon Oct 18 14:54:19 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 18 Oct 2010 14:54:19 +0200 (CEST)
Subject: [pypy-svn] r78038 - in pypy/branch/fast-forward/pypy/objspace/std:
. test
Message-ID: <20101018125419.BE6CF282B90@codespeak.net>
Author: afa
Date: Mon Oct 18 14:54:17 2010
New Revision: 78038
Modified:
pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py
pypy/branch/fast-forward/pypy/objspace/std/typeobject.py
Log:
Better test for the "safe subclass" case:
allow subclasses with a TypeDef, as long as they don't redefine the __new__ slot.
(like CPython which compares the tp_new slots)
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py Mon Oct 18 14:54:17 2010
@@ -28,6 +28,39 @@
raises(TypeError, type, 'sub', (stufftype,), {})
""")
+ def test_safe_subclass(self):
+ space = self.space
+ class W_Base(W_Object):
+ pass
+ def descr__new__(space, w_subtype):
+ return space.allocate_instance(W_Base, w_subtype)
+ W_Base.typedef = StdTypeDef("base",
+ __new__ = interp2app(descr__new__))
+ w_base = space.gettypeobject(W_Base.typedef)
+
+ # Cannot be created with base.__new__(derived)
+ class W_Derived(W_Base):
+ pass
+ def descr__new__(space, w_subtype):
+ return space.allocate_instance(W_Derived, w_subtype)
+ W_Derived.typedef = StdTypeDef("derived", W_Base.typedef,
+ __new__ = interp2app(descr__new__))
+ w_derived = space.gettypeobject(W_Derived.typedef)
+
+ # Can be created with base.__new__(derived2)
+ class W_Derived2(W_Base):
+ pass
+ W_Derived2.typedef = StdTypeDef("derived2", W_Base.typedef,
+ )
+ w_derived2 = space.gettypeobject(W_Derived2.typedef)
+
+ space.appexec([w_base, w_derived, w_derived2],
+ """(base, derived, derived2):
+ raises(TypeError, base.__new__, derived)
+ x = base.__new__(derived2)
+ assert isinstance(x, derived2)
+ """)
+
def test_del_warning(self):
warnings = []
def my_warn(msg, warningscls):
Modified: pypy/branch/fast-forward/pypy/objspace/std/typeobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/typeobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/typeobject.py Mon Oct 18 14:54:17 2010
@@ -326,7 +326,8 @@
raise operationerrfmt(space.w_TypeError,
"%s.__new__(%s): %s is not a subtype of %s",
w_self.name, w_subtype.name, w_subtype.name, w_self.name)
- if w_self.instancetypedef is not w_subtype.instancetypedef:
+ if (w_self.instancetypedef is not w_subtype.instancetypedef and
+ w_self.lookup('__new__') is not w_subtype.lookup('__new__')):
raise operationerrfmt(space.w_TypeError,
"%s.__new__(%s) is not safe, use %s.__new__()",
w_self.name, w_subtype.name, w_subtype.name)
From antocuni at codespeak.net Mon Oct 18 15:26:46 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 18 Oct 2010 15:26:46 +0200 (CEST)
Subject: [pypy-svn] r78039 - pypy/branch/jitffi/pypy/jit/backend/x86/test
Message-ID: <20101018132646.0E629282B90@codespeak.net>
Author: antocuni
Date: Mon Oct 18 15:26:45 2010
New Revision: 78039
Modified:
pypy/branch/jitffi/pypy/jit/backend/x86/test/test_zrpy_gc.py
Log:
a workaround for these failing tests, and a huge comment explaining it
Modified: pypy/branch/jitffi/pypy/jit/backend/x86/test/test_zrpy_gc.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/backend/x86/test/test_zrpy_gc.py (original)
+++ pypy/branch/jitffi/pypy/jit/backend/x86/test/test_zrpy_gc.py Mon Oct 18 15:26:45 2010
@@ -191,6 +191,32 @@
def run_orig(self, name, n, x):
self.main_allfuncs(name, n, x)
+ def define_libffi_workaround(cls):
+ # This is a workaround for a bug in database.py. It seems that the
+ # problem is triggered by optimizeopt/fficall.py, and in particular by
+ # the ``cast_base_ptr_to_instance(Func, llfunc)``: in these tests,
+ # that line is the only place where libffi.Func is referenced.
+ #
+ # The problem occurs because the gctransformer tries to annotate a
+ # low-level helper to call the __del__ of libffi.Func when it's too
+ # late.
+ #
+ # This workaround works by forcing the annotator (and all the rest of
+ # the toolchain) to see libffi.Func in a "proper" context, not just as
+ # the target of cast_base_ptr_to_instance. Note that the function
+ # below is *never* called by any actual test, it's just annotated.
+ #
+ from pypy.rlib.libffi import get_libc_name, CDLL, types, ArgChain
+ libc_name = get_libc_name()
+ def f(n, x, *args):
+ libc = CDLL(libc_name)
+ ptr = libc.getpointer('labs', [types.slong], types.slong)
+ chain = ArgChain()
+ chain.arg(n)
+ n = ptr.call(chain, lltype.Signed)
+ return (n, x) + args
+ return None, f, None
+
def define_compile_framework_1(cls):
# a moving GC. Supports malloc_varsize_nonmovable. Simple test, works
# without write_barriers and root stack enumeration.
From antocuni at codespeak.net Mon Oct 18 15:42:04 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 18 Oct 2010 15:42:04 +0200 (CEST)
Subject: [pypy-svn] r78040 - pypy/branch/jitffi/pypy/jit/backend/x86/test
Message-ID: <20101018134204.BBFC2282B90@codespeak.net>
Author: antocuni
Date: Mon Oct 18 15:42:03 2010
New Revision: 78040
Modified:
pypy/branch/jitffi/pypy/jit/backend/x86/test/test_zrpy_gc.py
Log:
add an XXX to the comment, to make armin happy :-)
Modified: pypy/branch/jitffi/pypy/jit/backend/x86/test/test_zrpy_gc.py
==============================================================================
--- pypy/branch/jitffi/pypy/jit/backend/x86/test/test_zrpy_gc.py (original)
+++ pypy/branch/jitffi/pypy/jit/backend/x86/test/test_zrpy_gc.py Mon Oct 18 15:42:03 2010
@@ -192,10 +192,11 @@
self.main_allfuncs(name, n, x)
def define_libffi_workaround(cls):
- # This is a workaround for a bug in database.py. It seems that the
- # problem is triggered by optimizeopt/fficall.py, and in particular by
- # the ``cast_base_ptr_to_instance(Func, llfunc)``: in these tests,
- # that line is the only place where libffi.Func is referenced.
+ # XXX: this is a workaround for a bug in database.py. It seems that
+ # the problem is triggered by optimizeopt/fficall.py, and in
+ # particular by the ``cast_base_ptr_to_instance(Func, llfunc)``: in
+ # these tests, that line is the only place where libffi.Func is
+ # referenced.
#
# The problem occurs because the gctransformer tries to annotate a
# low-level helper to call the __del__ of libffi.Func when it's too
From arigo at codespeak.net Mon Oct 18 15:43:19 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 15:43:19 +0200 (CEST)
Subject: [pypy-svn] r78041 - pypy/branch/rsre-jit/pypy/jit/codewriter
Message-ID: <20101018134319.CACEF36E3D6@codespeak.net>
Author: arigo
Date: Mon Oct 18 15:43:18 2010
New Revision: 78041
Modified:
pypy/branch/rsre-jit/pypy/jit/codewriter/codewriter.py
Log:
Write the jitcode into files, even if it is also printed.
Modified: pypy/branch/rsre-jit/pypy/jit/codewriter/codewriter.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/codewriter/codewriter.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/codewriter/codewriter.py Mon Oct 18 15:43:18 2010
@@ -95,18 +95,18 @@
print '%s:' % (ssarepr.name,)
print format_assembler(ssarepr)
else:
- dir = udir.ensure("jitcodes", dir=1)
- if portal_jitdriver:
- name = "%02d_portal_runner" % (portal_jitdriver.index,)
- elif ssarepr.name and ssarepr.name != '?':
- name = ssarepr.name
- else:
- name = 'unnamed' % id(ssarepr)
- i = 1
- extra = ''
- while name+extra in self._seen_files:
- i += 1
- extra = '.%d' % i
- self._seen_files.add(name+extra)
- dir.join(name+extra).write(format_assembler(ssarepr))
log.dot()
+ dir = udir.ensure("jitcodes", dir=1)
+ if portal_jitdriver:
+ name = "%02d_portal_runner" % (portal_jitdriver.index,)
+ elif ssarepr.name and ssarepr.name != '?':
+ name = ssarepr.name
+ else:
+ name = 'unnamed' % id(ssarepr)
+ i = 1
+ extra = ''
+ while name+extra in self._seen_files:
+ i += 1
+ extra = '.%d' % i
+ self._seen_files.add(name+extra)
+ dir.join(name+extra).write(format_assembler(ssarepr))
From arigo at codespeak.net Mon Oct 18 15:43:33 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 15:43:33 +0200 (CEST)
Subject: [pypy-svn] r78042 - pypy/branch/rsre-jit/pypy/jit/codewriter
Message-ID: <20101018134333.0AD90282B90@codespeak.net>
Author: arigo
Date: Mon Oct 18 15:43:31 2010
New Revision: 78042
Modified:
pypy/branch/rsre-jit/pypy/jit/codewriter/support.py
Log:
Fix comment.
Modified: pypy/branch/rsre-jit/pypy/jit/codewriter/support.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/codewriter/support.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/codewriter/support.py Mon Oct 18 15:43:31 2010
@@ -60,7 +60,7 @@
return rtyper.annotator.translator.graphs[0]
def split_before_jit_merge_point(graph, portalblock, portalopindex):
- """Find the block with 'jit_merge_point' and split just before,
+ """Split the block just before the 'jit_merge_point',
making sure the input args are in the canonical order.
"""
# split the block just before the jit_merge_point()
From arigo at codespeak.net Mon Oct 18 15:49:50 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 15:49:50 +0200 (CEST)
Subject: [pypy-svn] r78043 - in pypy/branch/rsre-jit/pypy: jit/codewriter
jit/codewriter/test jit/metainterp jit/metainterp/optimizeopt
jit/metainterp/test rlib rlib/rsre rlib/rsre/test
Message-ID: <20101018134950.EA7C036E3D6@codespeak.net>
Author: arigo
Date: Mon Oct 18 15:49:48 2010
New Revision: 78043
Added:
pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_jit.py
Modified:
pypy/branch/rsre-jit/pypy/jit/codewriter/jtransform.py
pypy/branch/rsre-jit/pypy/jit/codewriter/test/test_codewriter.py
pypy/branch/rsre-jit/pypy/jit/codewriter/test/test_jtransform.py
pypy/branch/rsre-jit/pypy/jit/metainterp/blackhole.py
pypy/branch/rsre-jit/pypy/jit/metainterp/executor.py
pypy/branch/rsre-jit/pypy/jit/metainterp/jitdriver.py
pypy/branch/rsre-jit/pypy/jit/metainterp/optimizeopt/string.py
pypy/branch/rsre-jit/pypy/jit/metainterp/optimizeopt/virtualize.py
pypy/branch/rsre-jit/pypy/jit/metainterp/pyjitpl.py
pypy/branch/rsre-jit/pypy/jit/metainterp/resoperation.py
pypy/branch/rsre-jit/pypy/jit/metainterp/resume.py
pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_basic.py
pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_greenfield.py
pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_virtualref.py
pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_warmspot.py
pypy/branch/rsre-jit/pypy/jit/metainterp/warmspot.py
pypy/branch/rsre-jit/pypy/rlib/jit.py
pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_char.py
pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_core.py
pypy/branch/rsre-jit/pypy/rlib/rsre/test/test_zjit.py
Log:
Main change: allows us to call jitdriver.jit_merge_point()
without ever calling jitdriver.can_enter_jit(). In that case
the jit_merge_point() plays both roles. The difference with
putting explicitly a can_enter_jit() just before is that such
a can_enter_jit() is not seen unless we are closing a loop;
in particular, it does not work if we have no loop at all.
Tests and fixes for greenfields.
Implement rlib.jit.jit_debug(), which just causes a JIT_DEBUG
resoperation to show up in the trace of the JIT.
Implement rlib.jit.assert_green(), which fails if the JIT considers
that the argument is not a Const.
When not translated, add the 'FORCE' prefix to the name of resoperations
produced by forcing virtualizables.
Modified: pypy/branch/rsre-jit/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/codewriter/jtransform.py Mon Oct 18 15:49:48 2010
@@ -320,6 +320,8 @@
prepare = self._handle_str2unicode_call
elif oopspec_name.startswith('virtual_ref'):
prepare = self._handle_virtual_ref_call
+ elif oopspec_name.startswith('jit.'):
+ prepare = self._handle_jit_call
else:
prepare = self.prepare_builtin_call
try:
@@ -859,6 +861,15 @@
(self.graph,))
return []
+ def _handle_jit_call(self, op, oopspec_name, args):
+ if oopspec_name == 'jit.debug':
+ return SpaceOperation('jit_debug', args, None)
+ elif oopspec_name == 'jit.assert_green':
+ kind = getkind(args[0].concretetype)
+ return SpaceOperation('%s_assert_green' % kind, args, None)
+ else:
+ raise AssertionError("missing support for %r" % oopspec_name)
+
# ----------
# Lists.
Modified: pypy/branch/rsre-jit/pypy/jit/codewriter/test/test_codewriter.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/codewriter/test/test_codewriter.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/codewriter/test/test_codewriter.py Mon Oct 18 15:49:48 2010
@@ -45,6 +45,7 @@
self.portal_graph = portal_graph
self.portal_runner_ptr = "???"
self.virtualizable_info = None
+ self.greenfield_info = None
def test_loop():
Modified: pypy/branch/rsre-jit/pypy/jit/codewriter/test/test_jtransform.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/codewriter/test/test_jtransform.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/codewriter/test/test_jtransform.py Mon Oct 18 15:49:48 2010
@@ -721,6 +721,45 @@
assert list(oplist[4].args[4]) == [v3, v4]
assert oplist[5].opname == '-live-'
+def test_getfield_gc():
+ S = lltype.GcStruct('S', ('x', lltype.Char))
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
+def test_getfield_gc_pure():
+ S = lltype.GcStruct('S', ('x', lltype.Char),
+ hints={'immutable': True})
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i_pure'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
+def test_getfield_gc_greenfield():
+ class FakeCC:
+ def get_vinfo(self, v):
+ return None
+ def could_be_green_field(self, S1, name1):
+ assert S1 is S
+ assert name1 == 'x'
+ return True
+ S = lltype.GcStruct('S', ('x', lltype.Char),
+ hints={'immutable': True})
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU(), FakeCC()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i_greenfield'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
def test_int_abs():
v1 = varoftype(lltype.Signed)
v2 = varoftype(lltype.Signed)
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/blackhole.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/blackhole.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/blackhole.py Mon Oct 18 15:49:48 2010
@@ -760,6 +760,20 @@
def bhimpl_debug_fatalerror(msg):
llop.debug_fatalerror(lltype.Void, msg)
+ @arguments("r", "i", "i", "i", "i")
+ def bhimpl_jit_debug(string, arg1=0, arg2=0, arg3=0, arg4=0):
+ pass
+
+ @arguments("i")
+ def bhimpl_int_assert_green(x):
+ pass
+ @arguments("r")
+ def bhimpl_ref_assert_green(x):
+ pass
+ @arguments("f")
+ def bhimpl_float_assert_green(x):
+ pass
+
# ----------
# the main hints and recursive calls
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/executor.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/executor.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/executor.py Mon Oct 18 15:49:48 2010
@@ -304,6 +304,7 @@
rop.CALL_ASSEMBLER,
rop.COND_CALL_GC_WB,
rop.DEBUG_MERGE_POINT,
+ rop.JIT_DEBUG,
rop.SETARRAYITEM_RAW,
): # list of opcodes never executed by pyjitpl
continue
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/jitdriver.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/jitdriver.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/jitdriver.py Mon Oct 18 15:49:48 2010
@@ -16,6 +16,7 @@
# self.greenfield_info ... pypy.jit.metainterp.warmspot
# self.warmstate ... pypy.jit.metainterp.warmspot
# self.handle_jitexc_from_bh pypy.jit.metainterp.warmspot
+ # self.no_loop_header ... pypy.jit.metainterp.warmspot
# self.portal_finishtoken... pypy.jit.metainterp.pyjitpl
# self.index ... pypy.jit.codewriter.call
# self.mainjitcode ... pypy.jit.codewriter.call
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/optimizeopt/string.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/optimizeopt/string.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/optimizeopt/string.py Mon Oct 18 15:49:48 2010
@@ -12,7 +12,7 @@
from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec
from pypy.jit.codewriter import heaptracker
from pypy.rlib.unroll import unrolling_iterable
-from pypy.rlib.objectmodel import specialize
+from pypy.rlib.objectmodel import specialize, we_are_translated
class StrOrUnicode(object):
@@ -107,7 +107,10 @@
self.box = box = self.source_op.result
newoperations = self.optimizer.newoperations
lengthbox = self.getstrlen(newoperations, self.mode)
- newoperations.append(ResOperation(self.mode.NEWSTR, [lengthbox], box))
+ op = ResOperation(self.mode.NEWSTR, [lengthbox], box)
+ if not we_are_translated():
+ op.name = 'FORCE'
+ newoperations.append(op)
self.string_copy_parts(newoperations, box, CONST_0, self.mode)
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/optimizeopt/virtualize.py Mon Oct 18 15:49:48 2010
@@ -74,6 +74,8 @@
assert self.source_op is not None
# ^^^ This case should not occur any more (see test_bug_3).
#
+ if not we_are_translated():
+ self.source_op.name = 'FORCE ' + self.source_op.name
newoperations = self.optimizer.newoperations
newoperations.append(self.source_op)
self.box = box = self.source_op.result
@@ -165,6 +167,8 @@
def _really_force(self):
assert self.source_op is not None
+ if not we_are_translated():
+ self.source_op.name = 'FORCE ' + self.source_op.name
newoperations = self.optimizer.newoperations
newoperations.append(self.source_op)
self.box = box = self.source_op.result
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/pyjitpl.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/pyjitpl.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/pyjitpl.py Mon Oct 18 15:49:48 2010
@@ -1,4 +1,4 @@
-import py, os
+import py, os, sys
from pypy.rpython.lltypesystem import lltype, llmemory, rclass
from pypy.rlib.objectmodel import we_are_translated
from pypy.rlib.unroll import unrolling_iterable
@@ -816,12 +816,16 @@
@arguments("orgpc", "int", "boxes3", "boxes3")
def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes, redboxes):
+ any_operation = len(self.metainterp.history.operations) > 0
jitdriver_sd = self.metainterp.staticdata.jitdrivers_sd[jdindex]
self.verify_green_args(jitdriver_sd, greenboxes)
# xxx we may disable the following line in some context later
self.debug_merge_point(jitdriver_sd, greenboxes)
if self.metainterp.seen_loop_header_for_jdindex < 0:
- return
+ if not jitdriver_sd.no_loop_header or not any_operation:
+ return
+ # automatically add a loop_header if there is none
+ self.metainterp.seen_loop_header_for_jdindex = jdindex
#
assert self.metainterp.seen_loop_header_for_jdindex == jdindex, (
"found a loop_header for a JitDriver that does not match "
@@ -910,6 +914,40 @@
msg = box.getref(lltype.Ptr(rstr.STR))
lloperation.llop.debug_fatalerror(msg)
+ @arguments("box", "box", "box", "box", "box")
+ def opimpl_jit_debug(self, stringbox, arg1box, arg2box, arg3box, arg4box):
+ from pypy.rpython.lltypesystem import rstr
+ from pypy.rpython.annlowlevel import hlstr
+ msg = stringbox.getref(lltype.Ptr(rstr.STR))
+ debug_print('jit_debug:', hlstr(msg),
+ arg1box.getint(), arg2box.getint(),
+ arg3box.getint(), arg4box.getint())
+ args = [stringbox, arg1box, arg2box, arg3box, arg4box]
+ i = 4
+ while i > 0 and args[i].getint() == -sys.maxint-1:
+ i -= 1
+ assert i >= 0
+ op = self.metainterp.history.record(rop.JIT_DEBUG, args[:i+1], None)
+ self.metainterp.attach_debug_info(op)
+
+ @arguments("box")
+ def _opimpl_assert_green(self, box):
+ if not isinstance(box, Const):
+ msg = "assert_green failed at %s:%d" % (
+ self.jitcode.name,
+ self.pc)
+ if we_are_translated():
+ from pypy.rpython.annlowlevel import llstr
+ from pypy.rpython.lltypesystem import lloperation
+ lloperation.llop.debug_fatalerror(lltype.Void, llstr(msg))
+ else:
+ from pypy.rlib.jit import AssertGreenFailed
+ raise AssertGreenFailed(msg)
+
+ opimpl_int_assert_green = _opimpl_assert_green
+ opimpl_ref_assert_green = _opimpl_assert_green
+ opimpl_float_assert_green = _opimpl_assert_green
+
@arguments("box")
def opimpl_virtual_ref(self, box):
# Details on the content of metainterp.virtualref_boxes:
@@ -1022,8 +1060,7 @@
if resumepc >= 0:
self.pc = resumepc
resume.capture_resumedata(metainterp.framestack, virtualizable_boxes,
- metainterp.virtualref_boxes,
- resumedescr)
+ metainterp.virtualref_boxes, resumedescr)
self.pc = saved_pc
self.metainterp.staticdata.profiler.count_ops(opnum, GUARDS)
# count
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/resoperation.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/resoperation.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/resoperation.py Mon Oct 18 15:49:48 2010
@@ -459,6 +459,7 @@
#'RUNTIMENEW/1', # ootype operation
'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier)
'DEBUG_MERGE_POINT/1', # debugging only
+ 'JIT_DEBUG/*', # debugging only
'VIRTUAL_REF_FINISH/2', # removed before it's passed to the backend
'COPYSTRCONTENT/5', # src, dst, srcstart, dststart, length
'COPYUNICODECONTENT/5',
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/resume.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/resume.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/resume.py Mon Oct 18 15:49:48 2010
@@ -738,15 +738,18 @@
assert (end & 1) == 0
return [self.decode_ref(nums[i]) for i in range(end)]
- def consume_vref_and_vable_boxes(self, vinfo):
+ def consume_vref_and_vable_boxes(self, vinfo, ginfo):
nums = self.cur_numb.nums
self.cur_numb = self.cur_numb.prev
- if vinfo is None:
- virtualizable_boxes = None
- end = len(nums)
- else:
+ if vinfo is not None:
virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, nums)
end = len(nums) - len(virtualizable_boxes)
+ elif ginfo is not None:
+ virtualizable_boxes = [self.decode_ref(nums[-1])]
+ end = len(nums) - 1
+ else:
+ virtualizable_boxes = None
+ end = len(nums)
virtualref_boxes = self.consume_virtualref_boxes(nums, end)
return virtualizable_boxes, virtualref_boxes
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_basic.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_basic.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_basic.py Mon Oct 18 15:49:48 2010
@@ -2,6 +2,7 @@
import sys
from pypy.rlib.jit import JitDriver, we_are_jitted, hint, dont_look_inside
from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_SIMPLE, loop_invariant
+from pypy.rlib.jit import jit_debug, assert_green, AssertGreenFailed
from pypy.jit.metainterp.warmspot import ll_meta_interp, get_stats
from pypy.jit.backend.llgraph import runner
from pypy.jit.metainterp import pyjitpl, history
@@ -44,6 +45,7 @@
num_green_args = 0
portal_graph = graphs[0]
virtualizable_info = None
+ greenfield_info = None
result_type = result_kind
portal_runner_ptr = "???"
@@ -1644,6 +1646,33 @@
res = self.interp_operations(f, [10, 3.5])
assert res == 3.5
+ def test_jit_debug(self):
+ myjitdriver = JitDriver(greens = [], reds = ['x'])
+ class A:
+ pass
+ def f(x):
+ while x > 0:
+ myjitdriver.can_enter_jit(x=x)
+ myjitdriver.jit_merge_point(x=x)
+ jit_debug("hi there:", x)
+ jit_debug("foobar")
+ x -= 1
+ return x
+ res = self.meta_interp(f, [8])
+ assert res == 0
+ self.check_loops(jit_debug=2)
+
+ def test_assert_green(self):
+ def f(x, promote):
+ if promote:
+ x = hint(x, promote=True)
+ assert_green(x)
+ return x
+ res = self.interp_operations(f, [8, 1])
+ assert res == 8
+ py.test.raises(AssertGreenFailed, self.interp_operations, f, [8, 0])
+
+
class TestOOtype(BasicTests, OOJitMixin):
def test_oohash(self):
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_greenfield.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_greenfield.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_greenfield.py Mon Oct 18 15:49:48 2010
@@ -27,6 +27,31 @@
self.check_loop_count(2)
self.check_loops(guard_value=0)
+ def test_green_field_2(self):
+ myjitdriver = JitDriver(greens=['ctx.x'], reds=['ctx'])
+ class Ctx(object):
+ _immutable_fields_ = ['x']
+ def __init__(self, x, y):
+ self.x = x
+ self.y = y
+ def f(x, y):
+ ctx = Ctx(x, y)
+ while 1:
+ myjitdriver.can_enter_jit(ctx=ctx)
+ myjitdriver.jit_merge_point(ctx=ctx)
+ ctx.y -= 1
+ if ctx.y < 0:
+ pass # to just make two paths
+ if ctx.y < -10:
+ return ctx.y
+ def g(y):
+ return f(5, y) + f(6, y)
+ #
+ res = self.meta_interp(g, [7])
+ assert res == -22
+ self.check_loop_count(4)
+ self.check_loops(guard_value=0)
+
class TestLLtypeGreenFieldsTests(GreenFieldsTests, LLJitMixin):
pass
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_virtualref.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_virtualref.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_virtualref.py Mon Oct 18 15:49:48 2010
@@ -93,7 +93,7 @@
lst = []
vrefinfo.continue_tracing = lambda vref, virtual: \
lst.append((vref, virtual))
- resumereader.consume_vref_and_vable(vrefinfo, None)
+ resumereader.consume_vref_and_vable(vrefinfo, None, None)
del vrefinfo.continue_tracing
assert len(lst) == 1
lltype.cast_opaque_ptr(lltype.Ptr(JIT_VIRTUAL_REF),
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_warmspot.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_warmspot.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_warmspot.py Mon Oct 18 15:49:48 2010
@@ -296,6 +296,69 @@
assert res == 1
self.check_loops(int_add=1) # I get 13 without the loop_header()
+ def test_omit_can_enter_jit(self):
+ # Simple test comparing the effects of always giving a can_enter_jit(),
+ # or not giving any. Mostly equivalent, except that if given, it is
+ # ignored the first time, and so it ends up taking one extra loop to
+ # start JITting.
+ mydriver = JitDriver(greens=[], reds=['m'])
+ #
+ for i2 in range(10):
+ def f2(m):
+ while m > 0:
+ mydriver.jit_merge_point(m=m)
+ m -= 1
+ self.meta_interp(f2, [i2])
+ try:
+ self.check_tree_loop_count(1)
+ break
+ except AssertionError:
+ print "f2: no loop generated for i2==%d" % i2
+ else:
+ raise # re-raise the AssertionError: check_loop_count never 1
+ #
+ for i1 in range(10):
+ def f1(m):
+ while m > 0:
+ mydriver.can_enter_jit(m=m)
+ mydriver.jit_merge_point(m=m)
+ m -= 1
+ self.meta_interp(f1, [i1])
+ try:
+ self.check_tree_loop_count(1)
+ break
+ except AssertionError:
+ print "f1: no loop generated for i1==%d" % i1
+ else:
+ raise # re-raise the AssertionError: check_loop_count never 1
+ #
+ assert i1 - 1 == i2
+
+ def test_no_loop_at_all(self):
+ mydriver = JitDriver(greens=[], reds=['m'])
+ def f2(m):
+ mydriver.jit_merge_point(m=m)
+ return m - 1
+ def f1(m):
+ while m > 0:
+ m = f2(m)
+ self.meta_interp(f1, [8])
+ # it should generate one "loop" only, which ends in a FINISH
+ # corresponding to the return from f2.
+ self.check_tree_loop_count(1)
+ self.check_loop_count(0)
+
+ def test_simple_loop(self):
+ mydriver = JitDriver(greens=[], reds=['m'])
+ def f1(m):
+ while m > 0:
+ mydriver.jit_merge_point(m=m)
+ m = m - 1
+ self.meta_interp(f1, [8])
+ self.check_loop_count(1)
+ self.check_loops({'int_sub': 1, 'int_gt': 1, 'guard_true': 1,
+ 'jump': 1})
+
class TestLLWarmspot(WarmspotTests, LLJitMixin):
CPUClass = runner.LLtypeCPU
Modified: pypy/branch/rsre-jit/pypy/jit/metainterp/warmspot.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/metainterp/warmspot.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/metainterp/warmspot.py Mon Oct 18 15:49:48 2010
@@ -115,10 +115,10 @@
return results
def find_can_enter_jit(graphs):
- results = _find_jit_marker(graphs, 'can_enter_jit')
- if not results:
- raise Exception("no can_enter_jit found!")
- return results
+ return _find_jit_marker(graphs, 'can_enter_jit')
+
+def find_loop_headers(graphs):
+ return _find_jit_marker(graphs, 'loop_header')
def find_jit_merge_points(graphs):
results = _find_jit_marker(graphs, 'jit_merge_point')
@@ -483,26 +483,37 @@
[lltype.Signed, llmemory.GCREF], RESTYPE)
def rewrite_can_enter_jits(self):
- can_enter_jits = find_can_enter_jit(self.translator.graphs)
sublists = {}
for jd in self.jitdrivers_sd:
- sublists[jd.jitdriver] = []
+ sublists[jd.jitdriver] = jd, []
+ jd.no_loop_header = True
+ #
+ loop_headers = find_loop_headers(self.translator.graphs)
+ for graph, block, index in loop_headers:
+ op = block.operations[index]
+ jitdriver = op.args[1].value
+ assert jitdriver in sublists, \
+ "loop_header with no matching jit_merge_point"
+ jd, sublist = sublists[jitdriver]
+ jd.no_loop_header = False
+ #
+ can_enter_jits = find_can_enter_jit(self.translator.graphs)
for graph, block, index in can_enter_jits:
op = block.operations[index]
jitdriver = op.args[1].value
assert jitdriver in sublists, \
"can_enter_jit with no matching jit_merge_point"
+ jd, sublist = sublists[jitdriver]
origportalgraph = jd._jit_merge_point_pos[0]
if graph is not origportalgraph:
- sublists[jitdriver].append((graph, block, index))
+ sublist.append((graph, block, index))
+ jd.no_loop_header = False
else:
pass # a 'can_enter_jit' before the 'jit-merge_point', but
# originally in the same function: we ignore it here
# see e.g. test_jitdriver.test_simple
for jd in self.jitdrivers_sd:
- sublist = sublists[jd.jitdriver]
- assert len(sublist) > 0, \
- "found no can_enter_jit for %r" % (jd.jitdriver,)
+ _, sublist = sublists[jd.jitdriver]
self.rewrite_can_enter_jit(jd, sublist)
def rewrite_can_enter_jit(self, jd, can_enter_jits):
@@ -510,6 +521,19 @@
FUNCPTR = jd._PTR_JIT_ENTER_FUNCTYPE
jit_enter_fnptr = self.helper_func(FUNCPTR, jd._maybe_enter_jit_fn)
+ if len(can_enter_jits) == 0:
+ # see test_warmspot.test_no_loop_at_all
+ operations = jd.portal_graph.startblock.operations
+ op1 = operations[0]
+ assert (op1.opname == 'jit_marker' and
+ op1.args[0].value == 'jit_merge_point')
+ op0 = SpaceOperation(
+ 'jit_marker',
+ [Constant('can_enter_jit', lltype.Void)] + op1.args[1:],
+ None)
+ operations.insert(0, op0)
+ can_enter_jits = [(jd.portal_graph, jd.portal_graph.startblock, 0)]
+
for graph, block, index in can_enter_jits:
if graph is jd._jit_merge_point_pos[0]:
continue
Modified: pypy/branch/rsre-jit/pypy/rlib/jit.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/rlib/jit.py (original)
+++ pypy/branch/rsre-jit/pypy/rlib/jit.py Mon Oct 18 15:49:48 2010
@@ -139,6 +139,24 @@
return hop.inputconst(lltype.Signed, _we_are_jitted)
+def jit_debug(string, arg1=-sys.maxint-1, arg2=-sys.maxint-1,
+ arg3=-sys.maxint-1, arg4=-sys.maxint-1):
+ """When JITted, cause an extra operation DEBUG_MERGE_POINT to appear in
+ the graphs. Should not be left after debugging."""
+ keepalive_until_here(string) # otherwise the whole function call is removed
+jit_debug.oopspec = 'jit.debug(string, arg1, arg2, arg3, arg4)'
+
+def assert_green(value):
+ """Very strong assert: checks that 'value' is a green
+ (a JIT compile-time constant)."""
+ keepalive_until_here(value)
+assert_green._annspecialcase_ = 'specialize:argtype(0)'
+assert_green.oopspec = 'jit.assert_green(value)'
+
+class AssertGreenFailed(Exception):
+ pass
+
+
##def force_virtualizable(virtualizable):
## pass
@@ -266,7 +284,8 @@
self.virtualizables = virtualizables
for v in self.virtualizables:
assert v in self.reds
- self._alllivevars = dict.fromkeys(self.greens + self.reds)
+ self._alllivevars = dict.fromkeys(
+ [name for name in self.greens + self.reds if '.' not in name])
self._make_extregistryentries()
self.get_jitcell_at = get_jitcell_at
self.set_jitcell_at = set_jitcell_at
Modified: pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_char.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_char.py (original)
+++ pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_char.py Mon Oct 18 15:49:48 2010
@@ -4,6 +4,7 @@
import sys
from pypy.rlib.rlocale import tolower, isalnum
from pypy.rlib.unroll import unrolling_iterable
+from pypy.rlib import jit
# Note: the unicode parts of this module require you to call
# rsre_char.set_unicode_db() first, to select one of the modules
@@ -43,6 +44,7 @@
# XXX can we import those safely from sre_constants?
SRE_INFO_PREFIX = 1
SRE_INFO_LITERAL = 2
+SRE_INFO_CHARSET = 4
SRE_FLAG_LOCALE = 4 # honour system locale
SRE_FLAG_UNICODE = 32 # use unicode locale
OPCODE_INFO = 17
@@ -64,33 +66,27 @@
#### Category helpers
-ascii_char_info = [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 6, 2,
-2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 25, 25, 25, 25, 25, 25, 25, 25,
-25, 25, 0, 0, 0, 0, 0, 0, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0,
-0, 0, 16, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0, 0, 0, 0 ]
-
+is_a_word = [(chr(i).isalnum() or chr(i) == '_') for i in range(256)]
linebreak = ord("\n")
underline = ord("_")
def is_digit(code):
- return code < 128 and (ascii_char_info[code] & 1 != 0)
+ return code <= 57 and code >= 48
def is_uni_digit(code):
assert unicodedb is not None
return unicodedb.isdigit(code)
def is_space(code):
- return code < 128 and (ascii_char_info[code] & 2 != 0)
+ return code == 32 or (code <= 13 and code >= 9)
def is_uni_space(code):
assert unicodedb is not None
return unicodedb.isspace(code)
def is_word(code):
- return code < 128 and (ascii_char_info[code] & 16 != 0)
+ assert code >= 0
+ return code < 256 and is_a_word[code]
def is_uni_word(code):
assert unicodedb is not None
@@ -142,6 +138,7 @@
SET_OK = -1
SET_NOT_OK = -2
+ at jit.unroll_safe
def check_charset(pattern, ppos, char_code):
"""Checks whether a character matches set of arbitrary length.
The set starts at pattern[ppos]."""
Modified: pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_core.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_core.py (original)
+++ pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_core.py Mon Oct 18 15:49:48 2010
@@ -5,6 +5,7 @@
from pypy.tool.sourcetools import func_with_new_name
from pypy.rlib.objectmodel import we_are_translated
from pypy.rlib import jit
+from pypy.rlib.rsre.rsre_jit import install_jitdriver, install_jitdriver_spec
OPCODE_FAILURE = 0
@@ -69,7 +70,7 @@
return getattr(ctx, specname)(*args)
dispatch._annspecialcase_ = 'specialize:argtype(0)'
dispatch._specialized_methods_ = specialized_methods
- return dispatch
+ return func_with_new_name(dispatch, specname)
# ____________________________________________________________
@@ -79,7 +80,7 @@
class AbstractMatchContext(object):
"""Abstract base class"""
- _immutable_fields_ = ['pattern[*]', 'flags']
+ _immutable_fields_ = ['pattern[*]', 'flags', 'end']
match_start = 0
match_end = 0
match_marks = None
@@ -245,7 +246,7 @@
@jit.unroll_safe
def find_first_result(self, ctx):
- ppos = self.ppos
+ ppos = jit.hint(self.ppos, promote=True)
while ctx.pat(ppos):
result = sre_match(ctx, ppos + 1, self.start_ptr, self.start_marks)
ppos += ctx.pat(ppos)
@@ -256,8 +257,10 @@
find_next_result = find_first_result
class RepeatOneMatchResult(MatchResult):
- jitdriver = jit.JitDriver(greens=['nextppos', 'pattern'],
- reds=['ptr', 'self', 'ctx'])
+ install_jitdriver('RepeatOne',
+ greens=['nextppos', 'ctx.pattern'],
+ reds=['ptr', 'self', 'ctx'],
+ debugprint=(1, 0)) # indices in 'greens'
def __init__(self, nextppos, minptr, ptr, marks):
self.nextppos = nextppos
@@ -269,15 +272,8 @@
ptr = self.start_ptr
nextppos = self.nextppos
while ptr >= self.minptr:
- #
- pattern = ctx.pattern
- self.jitdriver.can_enter_jit(self=self, ptr=ptr, ctx=ctx,
- nextppos=nextppos, pattern=pattern)
- self.jitdriver.jit_merge_point(self=self, ptr=ptr, ctx=ctx,
- nextppos=nextppos, pattern=pattern)
- if jit.we_are_jitted():
- ctx.pattern = pattern
- #
+ ctx.jitdriver_RepeatOne.jit_merge_point(
+ self=self, ptr=ptr, ctx=ctx, nextppos=nextppos)
result = sre_match(ctx, nextppos, ptr, self.start_marks)
ptr -= 1
if result is not None:
@@ -288,8 +284,10 @@
class MinRepeatOneMatchResult(MatchResult):
- jitdriver = jit.JitDriver(greens=['nextppos', 'ppos3', 'pattern'],
- reds=['ptr', 'self', 'ctx'])
+ install_jitdriver('MinRepeatOne',
+ greens=['nextppos', 'ppos3', 'ctx.pattern'],
+ reds=['ptr', 'self', 'ctx'],
+ debugprint=(2, 0)) # indices in 'greens'
def __init__(self, nextppos, ppos3, maxptr, ptr, marks):
self.nextppos = nextppos
@@ -303,17 +301,8 @@
nextppos = self.nextppos
ppos3 = self.ppos3
while ptr <= self.maxptr:
- #
- pattern = ctx.pattern
- self.jitdriver.can_enter_jit(self=self, ptr=ptr, ctx=ctx,
- nextppos=nextppos, pattern=pattern,
- ppos3=ppos3)
- self.jitdriver.jit_merge_point(self=self, ptr=ptr, ctx=ctx,
- nextppos=nextppos, pattern=pattern,
- ppos3=ppos3)
- if jit.we_are_jitted():
- ctx.pattern = pattern
- #
+ ctx.jitdriver_MinRepeatOne.jit_merge_point(
+ self=self, ptr=ptr, ctx=ctx, nextppos=nextppos, ppos3=ppos3)
result = sre_match(ctx, nextppos, ptr, self.start_marks)
if result is not None:
self.subresult = result
@@ -334,7 +323,7 @@
if ptr == ctx.end:
return False
op = ctx.pat(ppos)
- for op1, (checkerfn, _) in unroll_char_checker:
+ for op1, checkerfn in unroll_char_checker:
if op1 == op:
return checkerfn(ctx, ptr, ppos)
raise Error("next_char_ok[%d]" % op)
@@ -357,41 +346,34 @@
self.next = next # chained list
class MaxUntilMatchResult(AbstractUntilMatchResult):
+ install_jitdriver('MaxUntil',
+ greens=['ppos', 'tailppos', 'match_more', 'ctx.pattern'],
+ reds=['ptr', 'marks', 'self', 'ctx'],
+ debugprint=(3, 0, 2))
def find_first_result(self, ctx):
- enum = sre_match(ctx, self.ppos + 3, self.cur_ptr, self.cur_marks)
- return self.search_next(ctx, enum, resume=False)
+ return self.search_next(ctx, match_more=True)
def find_next_result(self, ctx):
- return self.search_next(ctx, None, resume=True)
+ return self.search_next(ctx, match_more=False)
- def search_next(self, ctx, enum, resume):
+ def search_next(self, ctx, match_more):
ppos = self.ppos
- min = ctx.pat(ppos+1)
- max = ctx.pat(ppos+2)
+ tailppos = self.tailppos
ptr = self.cur_ptr
marks = self.cur_marks
while True:
- while True:
- if (enum is not None and
- (ptr != ctx.match_end or self.num_pending < min)):
- # ^^^^^^^^^^ zero-width match protection
- # matched one more 'item'. record it and continue.
- self.pending = Pending(ptr, marks, enum, self.pending)
- self.num_pending += 1
- ptr = ctx.match_end
- marks = ctx.match_marks
- break
- # 'item' no longer matches.
- if not resume and self.num_pending >= min:
- # try to match 'tail' if we have enough 'item'
- result = sre_match(ctx, self.tailppos, ptr, marks)
- if result is not None:
- self.subresult = result
- self.cur_ptr = ptr
- self.cur_marks = marks
- return self
- resume = False
+ ctx.jitdriver_MaxUntil.jit_merge_point(
+ ppos=ppos, tailppos=tailppos, match_more=match_more,
+ ptr=ptr, marks=marks, self=self, ctx=ctx)
+ if match_more:
+ max = ctx.pat(ppos+2)
+ if max == 65535 or self.num_pending < max:
+ # try to match one more 'item'
+ enum = sre_match(ctx, ppos + 3, ptr, marks)
+ else:
+ enum = None # 'max' reached, no more matches
+ else:
p = self.pending
if p is None:
return
@@ -401,11 +383,27 @@
marks = p.marks
enum = p.enum.move_to_next_result(ctx)
#
- if max == 65535 or self.num_pending < max:
- # try to match one more 'item'
- enum = sre_match(ctx, ppos + 3, ptr, marks)
+ min = ctx.pat(ppos+1)
+ if (enum is not None and
+ (ptr != ctx.match_end or self.num_pending < min)):
+ # ^^^^^^^^^^ zero-width match protection
+ # matched one more 'item'. record it and continue.
+ self.pending = Pending(ptr, marks, enum, self.pending)
+ self.num_pending += 1
+ ptr = ctx.match_end
+ marks = ctx.match_marks
+ match_more = True
else:
- enum = None # 'max' reached, no more matches
+ # 'item' no longer matches.
+ if self.num_pending >= min:
+ # try to match 'tail' if we have enough 'item'
+ result = sre_match(ctx, tailppos, ptr, marks)
+ if result is not None:
+ self.subresult = result
+ self.cur_ptr = ptr
+ self.cur_marks = marks
+ return self
+ match_more = False
class MinUntilMatchResult(AbstractUntilMatchResult):
@@ -416,6 +414,7 @@
return self.search_next(ctx, resume=True)
def search_next(self, ctx, resume):
+ # XXX missing jit support here
ppos = self.ppos
min = ctx.pat(ppos+1)
max = ctx.pat(ppos+2)
@@ -471,6 +470,12 @@
op = ctx.pat(ppos)
ppos += 1
+ #jit.jit_debug("sre_match", op, ppos, ptr)
+ #
+ # When using the JIT, calls to sre_match() must always have a constant
+ # (green) argument for 'ppos'. If not, the following assert fails.
+ jit.assert_green(op)
+
if op == OPCODE_FAILURE:
return
@@ -745,13 +750,23 @@
@specializectx
def find_repetition_end(ctx, ppos, ptr, maxcount):
end = ctx.end
- # adjust end
- if maxcount != 65535:
+ if maxcount <= 1:
+ if maxcount == 1 and ptr < end:
+ # Relatively common case: maxcount == 1. If we are not at the
+ # end of the string, it's done by a single direct check.
+ op = ctx.pat(ppos)
+ for op1, checkerfn in unroll_char_checker:
+ if op1 == op:
+ if checkerfn(ctx, ptr, ppos):
+ return ptr + 1
+ return ptr
+ elif maxcount != 65535:
+ # adjust end
end1 = ptr + maxcount
if end1 <= end:
end = end1
op = ctx.pat(ppos)
- for op1, (_, fre) in unroll_char_checker:
+ for op1, fre in unroll_fre_checker:
if op1 == op:
return fre(ctx, ptr, end, ppos)
raise Error("rsre.find_repetition_end[%d]" % op)
@@ -784,23 +799,60 @@
if checkerfn == match_ANY_ALL:
def fre(ctx, ptr, end, ppos):
return end
+ elif checkerfn == match_IN:
+ install_jitdriver_spec('MatchIn',
+ greens=['ppos', 'ctx.pattern'],
+ reds=['ptr', 'end', 'ctx'],
+ debugprint=(1, 0))
+ @specializectx
+ def fre(ctx, ptr, end, ppos):
+ while True:
+ ctx.jitdriver_MatchIn.jit_merge_point(ctx=ctx, ptr=ptr,
+ end=end, ppos=ppos)
+ if ptr < end and checkerfn(ctx, ptr, ppos):
+ ptr += 1
+ else:
+ return ptr
+ elif checkerfn == match_IN_IGNORE:
+ install_jitdriver_spec('MatchInIgnore',
+ greens=['ppos', 'ctx.pattern'],
+ reds=['ptr', 'end', 'ctx'],
+ debugprint=(1, 0))
+ @specializectx
+ def fre(ctx, ptr, end, ppos):
+ while True:
+ ctx.jitdriver_MatchInIgnore.jit_merge_point(ctx=ctx, ptr=ptr,
+ end=end, ppos=ppos)
+ if ptr < end and checkerfn(ctx, ptr, ppos):
+ ptr += 1
+ else:
+ return ptr
else:
+ # in the other cases, the fre() function is not JITted at all
+ # and is present as a residual call.
+ @specializectx
def fre(ctx, ptr, end, ppos):
while ptr < end and checkerfn(ctx, ptr, ppos):
ptr += 1
return ptr
- return checkerfn, fre
+ fre = func_with_new_name(fre, 'fre_' + checkerfn.__name__)
+ return fre
+
+unroll_char_checker = [
+ (OPCODE_ANY, match_ANY),
+ (OPCODE_ANY_ALL, match_ANY_ALL),
+ (OPCODE_IN, match_IN),
+ (OPCODE_IN_IGNORE, match_IN_IGNORE),
+ (OPCODE_LITERAL, match_LITERAL),
+ (OPCODE_LITERAL_IGNORE, match_LITERAL_IGNORE),
+ (OPCODE_NOT_LITERAL, match_NOT_LITERAL),
+ (OPCODE_NOT_LITERAL_IGNORE, match_NOT_LITERAL_IGNORE),
+ ]
+unroll_fre_checker = [(_op, _make_fre(_fn))
+ for (_op, _fn) in unroll_char_checker]
-unroll_char_checker = unrolling_iterable([
- (OPCODE_ANY, _make_fre(match_ANY)),
- (OPCODE_ANY_ALL, _make_fre(match_ANY_ALL)),
- (OPCODE_IN, _make_fre(match_IN)),
- (OPCODE_IN_IGNORE, _make_fre(match_IN_IGNORE)),
- (OPCODE_LITERAL, _make_fre(match_LITERAL)),
- (OPCODE_LITERAL_IGNORE, _make_fre(match_LITERAL_IGNORE)),
- (OPCODE_NOT_LITERAL, _make_fre(match_NOT_LITERAL)),
- (OPCODE_NOT_LITERAL_IGNORE, _make_fre(match_NOT_LITERAL_IGNORE)),
- ])
+unroll_char_checker = unrolling_iterable(unroll_char_checker)
+unroll_fre_checker = unrolling_iterable(unroll_fre_checker)
##### At dispatch
@@ -906,74 +958,139 @@
else:
return None
+install_jitdriver('Match',
+ greens=['ctx.pattern'], reds=['ctx'],
+ debugprint=(0,))
+
def match_context(ctx):
ctx.original_pos = ctx.match_start
if ctx.end < ctx.match_start:
return False
+ ctx.jitdriver_Match.jit_merge_point(ctx=ctx)
return sre_match(ctx, 0, ctx.match_start, None) is not None
def search_context(ctx):
ctx.original_pos = ctx.match_start
if ctx.end < ctx.match_start:
return False
- if ctx.pat(0) == OPCODE_INFO:
- if ctx.pat(2) & rsre_char.SRE_INFO_PREFIX and ctx.pat(5) > 1:
- return fast_search(ctx)
- return regular_search(ctx)
+ base = 0
+ charset = False
+ if ctx.pat(base) == OPCODE_INFO:
+ flags = ctx.pat(2)
+ if flags & rsre_char.SRE_INFO_PREFIX:
+ if ctx.pat(5) > 1:
+ return fast_search(ctx)
+ else:
+ charset = (flags & rsre_char.SRE_INFO_CHARSET)
+ base += 1 + ctx.pat(1)
+ if ctx.pat(base) == OPCODE_LITERAL:
+ return literal_search(ctx, base)
+ if charset:
+ return charset_search(ctx, base)
+ return regular_search(ctx, base)
+
+install_jitdriver('RegularSearch',
+ greens=['base', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(1, 0))
-def regular_search(ctx):
+def regular_search(ctx, base):
start = ctx.match_start
while start <= ctx.end:
- if sre_match(ctx, 0, start, None) is not None:
+ ctx.jitdriver_RegularSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base)
+ if sre_match(ctx, base, start, None) is not None:
ctx.match_start = start
return True
start += 1
return False
+install_jitdriver_spec("LiteralSearch",
+ greens=['base', 'character', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(2, 0, 1))
+ at specializectx
+def literal_search(ctx, base):
+ # pattern starts with a literal character. this is used
+ # for short prefixes, and if fast search is disabled
+ character = ctx.pat(base + 1)
+ base += 2
+ start = ctx.match_start
+ while start < ctx.end:
+ ctx.jitdriver_LiteralSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base, character=character)
+ if ctx.str(start) == character:
+ if sre_match(ctx, base, start + 1, None) is not None:
+ ctx.match_start = start
+ return True
+ start += 1
+ return False
+
+install_jitdriver_spec("CharsetSearch",
+ greens=['base', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(1, 0))
+ at specializectx
+def charset_search(ctx, base):
+ # pattern starts with a character from a known set
+ start = ctx.match_start
+ while start < ctx.end:
+ ctx.jitdriver_CharsetSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base)
+ if rsre_char.check_charset(ctx.pattern, 5, ctx.str(start)):
+ if sre_match(ctx, base, start, None) is not None:
+ ctx.match_start = start
+ return True
+ start += 1
+ return False
+
+install_jitdriver_spec('FastSearch',
+ greens=['i', 'prefix_len', 'ctx.pattern'],
+ reds=['string_position', 'ctx'],
+ debugprint=(2, 0))
@specializectx
def fast_search(ctx):
# skips forward in a string as fast as possible using information from
# an optimization info block
# <1=skip> <2=flags> <3=min> <4=...>
# <5=length> <6=skip> <7=prefix data>
- flags = ctx.pat(2)
+ string_position = ctx.match_start
+ if string_position >= ctx.end:
+ return False
prefix_len = ctx.pat(5)
assert prefix_len >= 0
- prefix_skip = ctx.pat(6)
- assert prefix_skip >= 0
- overlap_offset = 7 + prefix_len - 1
- assert overlap_offset >= 0
- pattern_offset = ctx.pat(1) + 1
- ppos_start = pattern_offset + 2 * prefix_skip
- assert ppos_start >= 0
i = 0
- string_position = ctx.match_start
- end = ctx.end
- while string_position < end:
- while True:
- char_ord = ctx.str(string_position)
- if char_ord != ctx.pat(7 + i):
- if i == 0:
- break
- else:
- i = ctx.pat(overlap_offset + i)
- else:
- i += 1
- if i == prefix_len:
- # found a potential match
- start = string_position + 1 - prefix_len
- assert start >= 0
- ptr = start + prefix_skip
- if flags & rsre_char.SRE_INFO_LITERAL:
- # matched all of pure literal pattern
- ctx.match_start = start
- ctx.match_end = ptr
- ctx.match_marks = None
- return True
- if sre_match(ctx, ppos_start, ptr, None) is not None:
- ctx.match_start = start
- return True
- i = ctx.pat(overlap_offset + i)
- break
+ while True:
+ ctx.jitdriver_FastSearch.jit_merge_point(ctx=ctx,
+ string_position=string_position, i=i, prefix_len=prefix_len)
+ char_ord = ctx.str(string_position)
+ if char_ord != ctx.pat(7 + i):
+ if i > 0:
+ overlap_offset = prefix_len + (7 - 1)
+ i = ctx.pat(overlap_offset + i)
+ continue
+ else:
+ i += 1
+ if i == prefix_len:
+ # found a potential match
+ start = string_position + 1 - prefix_len
+ assert start >= 0
+ prefix_skip = ctx.pat(6)
+ ptr = start + prefix_skip
+ #flags = ctx.pat(2)
+ #if flags & rsre_char.SRE_INFO_LITERAL:
+ # # matched all of pure literal pattern
+ # ctx.match_start = start
+ # ctx.match_end = ptr
+ # ctx.match_marks = None
+ # return True
+ pattern_offset = ctx.pat(1) + 1
+ ppos_start = pattern_offset + 2 * prefix_skip
+ if sre_match(ctx, ppos_start, ptr, None) is not None:
+ ctx.match_start = start
+ return True
+ overlap_offset = prefix_len + (7 - 1)
+ i = ctx.pat(overlap_offset + i)
string_position += 1
- return False
+ if string_position >= ctx.end:
+ return False
Added: pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_jit.py
==============================================================================
--- (empty file)
+++ pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_jit.py Mon Oct 18 15:49:48 2010
@@ -0,0 +1,40 @@
+from pypy.rlib.jit import JitDriver
+
+
+class RSreJitDriver(JitDriver):
+
+ def __init__(self, name, debugprint, **kwds):
+ JitDriver.__init__(self, **kwds)
+ #
+ def get_printable_location(*args):
+ # we print based on indices in 'args'. We first print
+ # 'ctx.pattern' from the arg number debugprint[0].
+ pattern = args[debugprint[0]]
+ s = str(pattern)
+ if len(s) > 120:
+ s = s[:110] + '...'
+ if len(debugprint) > 1:
+ # then we print numbers from the args number
+ # debugprint[1] and possibly debugprint[2]
+ info = ' at %d' % (args[debugprint[1]],)
+ if len(debugprint) > 2:
+ info = '%s/%d' % (info, args[debugprint[2]])
+ else:
+ info = ''
+ return '%s%s %s' % (name, info, s)
+ #
+ self.get_printable_location = get_printable_location
+
+
+def install_jitdriver(name, **kwds):
+ from pypy.rlib.rsre.rsre_core import AbstractMatchContext
+ jitdriver = RSreJitDriver(name, **kwds)
+ setattr(AbstractMatchContext, 'jitdriver_' + name, jitdriver)
+
+def install_jitdriver_spec(name, **kwds):
+ from pypy.rlib.rsre.rsre_core import StrMatchContext
+ from pypy.rlib.rsre.rsre_core import UnicodeMatchContext
+ for prefix, concreteclass in [('Str', StrMatchContext),
+ ('Uni', UnicodeMatchContext)]:
+ jitdriver = RSreJitDriver(prefix + name, **kwds)
+ setattr(concreteclass, 'jitdriver_' + name, jitdriver)
Modified: pypy/branch/rsre-jit/pypy/rlib/rsre/test/test_zjit.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/rlib/rsre/test/test_zjit.py (original)
+++ pypy/branch/rsre-jit/pypy/rlib/rsre/test/test_zjit.py Mon Oct 18 15:49:48 2010
@@ -6,16 +6,30 @@
from pypy.rpython.lltypesystem import lltype
from pypy.rpython.annlowlevel import llstr, hlstr
-def entrypoint1(r, string):
+def entrypoint1(r, string, repeat):
r = array2list(r)
string = hlstr(string)
make_sure_not_modified(r)
- match = rsre_core.match(r, string)
+ match = None
+ for i in range(repeat):
+ match = rsre_core.match(r, string)
if match is None:
return -1
else:
return match.match_end
+def entrypoint2(r, string, repeat):
+ r = array2list(r)
+ string = hlstr(string)
+ make_sure_not_modified(r)
+ match = None
+ for i in range(repeat):
+ match = rsre_core.search(r, string)
+ if match is None:
+ return -1
+ else:
+ return match.match_start
+
def list2array(lst):
a = lltype.malloc(lltype.GcArray(lltype.Signed), len(lst))
for i, x in enumerate(lst):
@@ -35,9 +49,16 @@
class TestJitRSre(test_basic.LLJitMixin):
- def meta_interp_match(self, pattern, string):
+ def meta_interp_match(self, pattern, string, repeat=1):
r = get_code(pattern)
- return self.meta_interp(entrypoint1, [list2array(r), llstr(string)],
+ return self.meta_interp(entrypoint1, [list2array(r), llstr(string),
+ repeat],
+ listcomp=True, backendopt=True)
+
+ def meta_interp_search(self, pattern, string, repeat=1):
+ r = get_code(pattern)
+ return self.meta_interp(entrypoint2, [list2array(r), llstr(string),
+ repeat],
listcomp=True, backendopt=True)
def test_simple_match_1(self):
@@ -48,6 +69,11 @@
res = self.meta_interp_match(r".*abc", "xxabcyyyyyyyyyyyyy")
assert res == 5
+ def test_simple_match_repeated(self):
+ res = self.meta_interp_match(r"abcdef", "abcdef", repeat=10)
+ assert res == 6
+ self.check_tree_loop_count(1)
+
def test_match_minrepeat_1(self):
res = self.meta_interp_match(r".*?abc", "xxxxxxxxxxxxxxabc")
assert res == 17
@@ -67,3 +93,28 @@
"xxxxxxxxxxabbbbbbbbbbc")
res = self.meta_interp_match(r".*?ab+?c", s)
assert res == len(s)
+
+
+ def test_fast_search(self):
+ res = self.meta_interp_search(r"", "eua")
+ assert res == 15
+ self.check_loops(guard_value=0)
+
+ def test_regular_search(self):
+ res = self.meta_interp_search(r"<\w+>", "eiofweoxdiwhdohua")
+ assert res == 15
+
+ def test_regular_search_upcase(self):
+ res = self.meta_interp_search(r"<\w+>", "EIOFWEOXDIWHDOHUA")
+ assert res == 15
+
+ def test_max_until_1(self):
+ res = self.meta_interp_match(r"(ab)*abababababc",
+ "ababababababababababc")
+ assert res == 21
+
+ def test_example_1(self):
+ res = self.meta_interp_search(
+ r"Active\s+20\d\d-\d\d-\d\d\s+[[]\d+[]]([^[]+)",
+ "Active"*20 + "Active 2010-04-07 [42] Foobar baz boz blah[43]")
+ assert res == 6*20
From arigo at codespeak.net Mon Oct 18 16:05:51 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 16:05:51 +0200 (CEST)
Subject: [pypy-svn] r78044 - pypy/branch/fast-forward/lib_pypy
Message-ID: <20101018140551.CB81A282B90@codespeak.net>
Author: arigo
Date: Mon Oct 18 16:05:49 2010
New Revision: 78044
Added:
pypy/branch/fast-forward/lib_pypy/future_builtins.py
Log:
Implement the module future_builtins at app-level.
Added: pypy/branch/fast-forward/lib_pypy/future_builtins.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/lib_pypy/future_builtins.py Mon Oct 18 16:05:49 2010
@@ -0,0 +1,33 @@
+"""This module provides functions that will be builtins in Python 3.0,
+but that conflict with builtins that already exist in Python 2.x.
+
+Functions:
+
+hex(arg) -- Returns the hexadecimal representation of an integer
+oct(arg) -- Returns the octal representation of an integer
+ascii(arg) -- Same as repr(arg)
+map, filter, zip -- Same as itertools.imap, ifilter, izip
+
+The typical usage of this module is to replace existing builtins in a
+module's namespace:
+
+from future_builtins import hex, oct
+"""
+
+__all__ = ['hex', 'oct', 'ascii', 'map', 'filter', 'zip']
+
+from itertools import imap as map, ifilter as filter, izip as zip
+
+ascii = repr
+_builtin_hex = hex
+_builtin_oct = oct
+
+def hex(arg):
+ return _builtin_hex(arg).rstrip('L')
+
+def oct(arg):
+ result = _builtin_oct(arg).rstrip('L')
+ if result == '0':
+ return '0o0'
+ i = result.index('0') + 1
+ return result[:i] + 'o' + result[i:]
From arigo at codespeak.net Mon Oct 18 16:20:35 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 16:20:35 +0200 (CEST)
Subject: [pypy-svn] r78045 -
pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests
Message-ID: <20101018142035.DD8B736E3D6@codespeak.net>
Author: arigo
Date: Mon Oct 18 16:20:33 2010
New Revision: 78045
Modified:
pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c
pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py
Log:
A test for a property of ctypes: default restype.
Modified: pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c
==============================================================================
--- pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c (original)
+++ pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c Mon Oct 18 16:20:33 2010
@@ -533,3 +533,7 @@
return inp;
}
+int my_unused_function(void)
+{
+ return 42;
+}
Modified: pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py
==============================================================================
--- pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py (original)
+++ pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py Mon Oct 18 16:20:33 2010
@@ -148,3 +148,7 @@
# but it segfaults for some reason.
if sys.platform == 'win32':
assert f() == 0x12345678
+
+ def test_restype(self):
+ foo = lib.my_unused_function
+ assert foo.restype is c_int # by default
From cfbolz at codespeak.net Mon Oct 18 16:37:53 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon, 18 Oct 2010 16:37:53 +0200 (CEST)
Subject: [pypy-svn] r78046 - pypy/extradoc/talk/pepm2011
Message-ID: <20101018143753.A8351282B90@codespeak.net>
Author: cfbolz
Date: Mon Oct 18 16:37:52 2010
New Revision: 78046
Modified:
pypy/extradoc/talk/pepm2011/math.lyx
pypy/extradoc/talk/pepm2011/paper.tex
Log:
comments by stephan in section 5
Modified: pypy/extradoc/talk/pepm2011/math.lyx
==============================================================================
--- pypy/extradoc/talk/pepm2011/math.lyx (original)
+++ pypy/extradoc/talk/pepm2011/math.lyx Mon Oct 18 16:37:52 2010
@@ -408,7 +408,7 @@
\begin_inset Text
\begin_layout Plain Layout
-\begin_inset Formula ${\displaystyle \frac{E(v)\notin\mathrm{dom}(S)\, u^{*}\,\mathrm{fresh}}{u=\mathtt{get}(v,F),E,S\overset{\mathrm{opt}}{\Longrightarrow}\left\langle u^{*}=\mathtt{get}(E(v),F)\right\rangle ,E\left[u\mapsto u^{*}\right],S}}$
+\begin_inset Formula ${\displaystyle \frac{E(v)\notin\mathrm{dom}(S),\,\,\, u^{*}\,\mathrm{fresh}}{u=\mathtt{get}(v,F),E,S\overset{\mathrm{opt}}{\Longrightarrow}\left\langle u^{*}=\mathtt{get}(E(v),F)\right\rangle ,E\left[u\mapsto u^{*}\right],S}}$
\end_inset
@@ -504,7 +504,7 @@
\begin_inset Text
\begin_layout Plain Layout
-\begin_inset Formula ${\displaystyle \frac{E(v)\notin\mathrm{dom}(S)\vee\mathrm{type}(S(E(v)))\neq T,\,\left(E(v),S\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops},S^{\prime}\right)}{\mathtt{guard\_class}(v,T),E,S\overset{\mathrm{opt}}{\Longrightarrow}\left\langle \mathtt{guard\_class}(E\left(v\right),T)\right\rangle ,E,S^{\prime}}}$
+\begin_inset Formula ${\displaystyle \frac{E(v)\notin\mathrm{dom}(S)\vee\mathrm{type}(S(E(v)))\neq T,\,\left(E(v),S\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops},S^{\prime}\right)}{\mathtt{guard\_class}(v,T),E,S\overset{\mathrm{opt}}{\Longrightarrow}\mathrm{ops}::\left\langle \mathtt{guard\_class}(E\left(v\right),T)\right\rangle ,E,S^{\prime}}}$
\end_inset
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Mon Oct 18 16:37:52 2010
@@ -685,6 +685,7 @@
\emph{get} & ${\displaystyle \frac{\,}{u=\mathtt{get}(v,F),E,H\overset{\mathrm{run}}{\Longrightarrow}E\left[u\mapsto H\left(E\left(v\right)\right)_{F}\right],H}}$ & ~~~ & & ${\displaystyle \frac{\mathrm{type}(H(E(v))\neq T}{\mathtt{guard\_class}(v,T),E,H\overset{\mathrm{run}}{\Longrightarrow}\bot,\bot}}$\tabularnewline[3em]
\emph{set} & ${\displaystyle \frac{\,}{\mathtt{set}\left(v,F,u\right),E,H\overset{\mathrm{run}}{\Longrightarrow}E,H\left[E\left(v\right)\mapsto\left(H\left(E\left(v\right)\right)!_{F}E(u)\right)\right]}}$ & ~~~ & & \tabularnewline[4em]
\end{tabular}
+\end{center}
\begin{minipage}[b]{7 cm}
\emph{Object Domains:}
@@ -708,25 +709,25 @@
\end{array}
$$
\end{minipage}
-\end{center}
\caption{The Operational Semantics of Simplified Traces}
\label{fig:semantics}
\end{figure*}
In this section we want to give a formal description of the semantics of the
traces and of the optimizer and liken the optimization to partial evaluation.
-We concentrate on the operations for manipulating dynamically allocated objects,
+We focus on the operations for manipulating dynamically allocated objects,
as those are the only ones that are actually optimized. Without loss of
generality we also consider only objects with two fields in this section.
-Traces are lists of operations. The operations considered here are \lstinline{new} (to make
-a new object), \lstinline{get} (to read a field out of an object), \lstinline{set} (to write a field
-into an object) and \lstinline{guard_class} (to check the type of an object). The values of all
-variables are locations (i.e.~pointers). Locations are mapped to objects, which
+Traces are lists of operations. The operations considered here are
+\lstinline{new}, \lstinline{get}, \lstinline{set} and \lstinline{guard_class}.
+The values of all
+variables are locations (\ie pointers). Locations are mapped to objects, which
are represented by triples of a type $T$, and two locations that represent the
fields of the object. When a new object is created, the fields are initialized
to null, but we require that they are initialized to a real
-location before being read, otherwise the trace is malformed.
+location before being read, otherwise the trace is malformed (this condition is
+guaranteed by how the traces are generated in PyPy).
We use some abbreviations when dealing with object triples. To read the type of
an object, $\mathrm{type}((T,l_1,l_2))=T$ is used. Reading a field $F$ from an
@@ -737,13 +738,12 @@
Figure~\ref{fig:semantics} shows the operational semantics for traces. The
interpreter formalized there executes one operation at a time. Its state is
-represented by an environment $E$ and a heap $H$, which are potentially changed by the
+represented by an environment $E$ and a heap $H$, which may be changed by the
execution of an operation. The environment is a partial function from variables
to locations and the heap is a partial function from locations to objects. Note
-that a variable can never be null in the environment, otherwise the trace would
-be malformed. The environment could not directly map variables to object,
-because several variables can contain a pointer to the \emph{same} object.
-The "indirection" is needed to express sharing.
+that a variable can never be null in the environment, otherwise the trace would have
+been malformed. The environment could not directly map variables to objects,
+because several variables can point to the \emph{same} object, because of aliasing.
We use the following notation for updating partial functions:
$E[v\mapsto l]$ denotes the environment which is just like $E$, but maps $v$ to
@@ -776,15 +776,16 @@
\begin{tabular}{lc}
\emph{new} & ${\displaystyle \frac{v^{*}\,\mathrm{fresh}}{v=\mathtt{new}(T),E,S\overset{\mathrm{opt}}{\Longrightarrow}\left\langle \,\right\rangle ,E\left[v\mapsto v^{*}\right],S\left[v^{*}\mapsto\left(T,\mathrm{null,null}\right)\right]}}$\tabularnewline[3em]
\emph{get} & ${\displaystyle \frac{E(v)\in\mathrm{dom}(S)}{u=\mathtt{get}(v,F),E,S\overset{\mathrm{opt}}{\Longrightarrow}\left\langle \,\right\rangle ,E\left[u\mapsto S(E(v))_{F}\right],S}}$\tabularnewline[3em]
- & ${\displaystyle \frac{E(v)\notin\mathrm{dom}(S)\, u^{*}\,\mathrm{fresh}}{u=\mathtt{get}(v,F),E,S\overset{\mathrm{opt}}{\Longrightarrow}\left\langle u^{*}=\mathtt{get}(E(v),F)\right\rangle ,E\left[u\mapsto u^{*}\right],S}}$\tabularnewline[3em]
+ & ${\displaystyle \frac{E(v)\notin\mathrm{dom}(S),\,\,\, u^{*}\,\mathrm{fresh}}{u=\mathtt{get}(v,F),E,S\overset{\mathrm{opt}}{\Longrightarrow}\left\langle u^{*}=\mathtt{get}(E(v),F)\right\rangle ,E\left[u\mapsto u^{*}\right],S}}$\tabularnewline[3em]
\emph{set} & ${\displaystyle \frac{E(v)\in\mathrm{dom}(S)}{\mathtt{set}\left(v,F,u\right),E,S\overset{\mathrm{opt}}{\Longrightarrow}\left\langle \,\right\rangle ,E,S\left[E\left(v\right)\mapsto\left(S(E(v))!_{F}E(u)\right)\right]}}$\tabularnewline[3em]
& ${\displaystyle \frac{E(v)\notin\mathrm{dom}\left(S\right),\,\left(E(v),S\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops},S^{\prime}\right)}{\mathtt{set}\left(v,F,u\right),E,S\overset{\mathrm{opt}}{\Longrightarrow}\mathrm{ops}::\left\langle \mathtt{set}\left(E(v),F,E(u)\right)\right\rangle ,E,S^{\prime}}}$\tabularnewline[3em]
\emph{guard} & ${\displaystyle \frac{E(v)\in\mathrm{dom}(S),\,\mathrm{type}(S(E(v)))=T}{\mathtt{guard\_class}(v,T),E,S\overset{\mathrm{opt}}{\Longrightarrow}\left\langle \,\right\rangle ,E,S}}$\tabularnewline[3em]
- & ${\displaystyle \frac{E(v)\notin\mathrm{dom}(S)\vee\mathrm{type}(S(E(v)))\neq T,\,\left(E(v),S\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops},S^{\prime}\right)}{\mathtt{guard\_class}(v,T),E,S\overset{\mathrm{opt}}{\Longrightarrow}\left\langle \mathtt{guard\_class}(E\left(v\right),T)\right\rangle ,E,S^{\prime}}}$\tabularnewline[3em]
+ & ${\displaystyle \frac{E(v)\notin\mathrm{dom}(S)\vee\mathrm{type}(S(E(v)))\neq T,\,\left(E(v),S\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops},S^{\prime}\right)}{\mathtt{guard\_class}(v,T),E,S\overset{\mathrm{opt}}{\Longrightarrow}\mathrm{ops}::\left\langle \mathtt{guard\_class}(E\left(v\right),T)\right\rangle ,E,S^{\prime}}}$\tabularnewline[3em]
\emph{lifting} & ${\displaystyle \frac{v^{*}\notin\mathrm{dom}(S)}{v^{*},S\overset{\mathrm{lift}}{\Longrightarrow}\left\langle \,\right\rangle ,S}}$\tabularnewline[3em]
& ${\displaystyle \frac{v^{*}\in\mathrm{dom}(S),\,\left(v^{*},S\right)\overset{\mathrm{liftfields}}{=\!=\!\Longrightarrow}\left(\mathrm{ops},S^{\prime}\right)}{v^{*},S\overset{\mathrm{lift}}{\Longrightarrow}\left\langle v^{*}=\mathtt{new}\left(T\right)\right\rangle ::ops,S^{\prime}}}$\tabularnewline[3em]
& ${\displaystyle \frac{\left(S\left(v^{*}\right)_{L},S\setminus\left\{ v^{*}\mapsto S\left(v^{*}\right)\right\} \right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{L},S^{\prime}\right),\,\left(S\left(v^{*}\right)_{R},S^{\prime}\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{R},S^{\prime\prime}\right)}{v^{*},S\overset{\mathrm{liftfields}}{=\!=\!\Longrightarrow}\mathrm{ops}_{L}::ops_{R}::\left\langle \mathtt{set}\left(v^{*},L,S\left(v^{*}\right)_{L}\right),\,\mathtt{set}\left(v^{*},R,S\left(v^{*}\right)_{R}\right)\right\rangle ,S^{\prime}}}$\tabularnewline[3em]
\end{tabular}
+\end{center}
\begin{minipage}[b]{7 cm}
\emph{Object Domains:}
@@ -808,17 +809,16 @@
\end{array}
$$
\end{minipage}
-\end{center}
\caption{Optimization Rules}
\label{fig:optimization}
\end{figure*}
-To optimize the simple traces from the last section, we use online partial
-evaluation. The partial evaluator optimizes one operation of the trace at a
+To optimize the simple traces of the last section, we use online partial
+evaluation. The partial evaluator optimizes one operation of a trace at a
time. Every operation in the unoptimized trace is replaced by a list of
operations in the optimized trace. This list is empty if the operation
-can be optimized away (which hopefully happens often). The optimization rules
-can be seen in Figure~\ref{fig:optimization}.
+can be optimized away. The optimization rules can be seen in
+Figure~\ref{fig:optimization}.
The state of the optimizer is stored in an environment $E$ and a \emph{static
heap} $S$. The environment is a partial function from variables in the
@@ -827,13 +827,14 @@
$\ ^*$ for clarity). The reason for introducing new variables in the optimized
trace is that several variables that appear in the unoptimized trace can turn
into the same variables in the optimized trace. The environment of the
-optimizer serves a function similar to that of the environment in the semantics: sharing.
+optimizer serves a function similar to that of the environment in the
+semantics: to express sharing.
The static heap is a partial function from $V^*$ into the
set of static objects, which are triples of a type and two elements of $V^*$.
A variable $v^*$ is in the domain of the static heap $S$ as long as the
-optimizer can fully keep track of the object. The object $S(v^*)$ is what is
-statically known about the object stored in it, \ie its type and its fields. The
+optimizer does not need to become dynamic XXX. The object $S(v^*)$ describes
+what is statically known about the object, \ie its type and its fields. The
fields of objects in the static heap are also elements of $V^*$ (or null, for
short periods of time).
@@ -841,7 +842,7 @@
assumes that the resulting object can stay static. The optimization for all
further operations is split into two cases. One case is for when the
involved variables are in the static heap, which means that the operation can be
-performed at optimization time and removed from the trace. These rules mirror
+performed at optimization time and can be removed from the trace. These rules mirror
the execution semantics closely. The other case is for when not enough is known about
the variables, and the operation has to be residualized.
@@ -850,8 +851,9 @@
operation needs to be residualized.
If the first argument $v$ to a \lstinline{set} operation is mapped to something in the
-static heap, then the \lstinline{set} can performed at optimization time and the static heap
-updated. Otherwise the \lstinline{set} operation needs to be residualized. This needs to be
+static heap, then the \lstinline{set} can be performed at optimization time
+(which updates the static heap). Otherwise the \lstinline{set} operation needs
+to be residualized. This needs to be
done carefully, because the new value for the field, from the variable $u$,
could itself be static, in which case it needs to be lifted first.
@@ -861,10 +863,10 @@
in the static heap, the \lstinline{guard_class} is residualized. This also needs to
lift the variable on which the \lstinline{guard_class} is performed.
-Lifting takes a variable that is potentially in the static heap and makes sure
-that it is turned into a dynamic variable. This means that operations are
-emitted that construct an object with the shape described in the
-static heap, and the variable is removed from the static heap.
+Lifting takes a variable and turns it into a dynamic variable. If the variable
+is already dynamic, nothing needs to be done. If it is in the static heap,
+operations are emitted that construct an object with the shape described
+there, and the variable is removed from the static heap.
Lifting a static object needs to recursively lift its fields. Some care needs to
be taken when lifting a static object, because the structures described by the
@@ -874,7 +876,7 @@
As an example for lifting, consider the static heap $$\{v^* \mapsto (T_1, w^*,
v^*), w^* \mapsto (T_2, u^*, u^*)\}$$ which contains two static objects. If $v^*$
-now needs to be lifted, the following residual operations are produced:
+needs to be lifted, the following residual operations are produced:
\begin{lstlisting}[mathescape,xleftmargin=20pt]
$v^*$ = new($T_1$)
@@ -914,20 +916,20 @@
the algorithm only takes a total time linear in the length of the trace.
The algorithm itself is not particularly complex; our focus is
rather that \emph{in the context of tracing JITs} it is possible to find a
-simple enough algorithm that still gives very good results.
+simple enough algorithm that performs well.
-Note in particular that objects in category 1 (\ie the ones that do
+Note in particular that objects in category 1 (\ie those that do
not escape) are completely removed; moreover, objects in category 2
-(\ie escaping) are still partially dealt with: if such an object
-escapes later than its creation point, all the operations in between that
-involve the object are removed.
+(\ie escaping) are still partially optimized: all the operations in between the
+creation of the object and the point where it escapes that involve the object
+are removed.
The optimization is particularly effective for chains of operations.
For example, it is typical for an interpreter to generate sequences of
writes-followed-by-reads, where one interpreted opcode writes to some
object's field and the next interpreted opcode reads it back, possibly
dispatching on the type of the object created just before. A typical example
-would be chains of arithmetic operations.
+would be a chain of arithmetic operations.
% subsection Analysis of the Algorithm (end)
@@ -944,7 +946,7 @@
the optimizer of PyPy's tracing JIT. The optimization is independent of which
interpreter a JIT is generated for. There are some practical issues beyond the
techniques described in this paper. The actual implementation needs to deal with
-more operations than described in Section~\ref{sec:formal}, for example to
+more operations than described in Section~\ref{sec:formal}, \eg to
also support static arrays in addition to static objects. The implementation of
this optimization is about 400 lines of RPython code.
@@ -956,7 +958,7 @@
special way to solve this problem. This is a common approach in VM
implementations \cite{miranda_context_1999,andreas_gal_trace-based_2009}; the
novelty of our approach is that we generalized it enough to be usable for
-different interpreter.
+different interpreters.
To evaluate our allocation removal algorithm, we look at the effectiveness when
used in the generated tracing JIT of PyPy's Python interpreter. This interpreter
@@ -1016,15 +1018,17 @@
As the first step, we counted the occurring operations in all generated traces
before and after the optimization phase for all benchmarks. The resulting
numbers can be
-seen in Figure~\ref{fig:numops}. The optimization removes as many as 90\% and as
-little as 4\% percent of allocation operations in the traces of the benchmarks.
+seen in Figure~\ref{fig:numops}. The optimization removes between 4\% and 90\%
+and of allocation operations in the traces of the benchmarks.
All benchmarks taken together, the optimization removes 70\% percent of
allocation operations. The numbers look similar for reading and writing of
attributes. There are even more \lstinline{guard} operations that are removed,
however there is an additional optimization that removes guards, so not all the
-removed guards are an effect of the optimization described here.
+removed guards are an effect of the optimization described here (for technical
+reasons, it would be very hard to separate the two effects).
\begin{figure*}
+{\small
\begin{center}
\begin{tabular}{|l||r|rr|rr|rr|rr|}
\hline
@@ -1049,6 +1053,7 @@
\hline
\end{tabular}
\end{center}
+}
\caption{Number of Operations and Percentage Removed By Optimization}
\label{fig:numops}
\end{figure*}
@@ -1072,6 +1077,7 @@
further optimizations.
\begin{figure*}
+{\small
\begin{center}
\begin{tabular}{|l||r|r||r|r||r|r||r|r|}
\hline
@@ -1094,6 +1100,7 @@
\hline
\end{tabular}
\end{center}
+}
\caption{Benchmark Times in Milliseconds, Together With Factor Over PyPy With Optimizations}
\label{fig:times}
\end{figure*}
@@ -1160,12 +1167,12 @@
% separation logic
%; John Hughes: type specialization
-\section{Conclusion}
+\section{Conclusion and Future Work}
\label{sec:conclusion}
In this paper, we used an approach based on online partial evaluation
to optimize away allocations and type guards in the traces of a
-tracing JIT. In this context a simple approach to partial evaluation
+tracing JIT. In this context a simple approach based on partial evaluation
gives good results. This is due to the fact that the tracing JIT
itself is responsible for all control issues, which are usually the
hardest part of partial evaluation: the tracing JIT selects the parts
@@ -1185,7 +1192,8 @@
\section*{Acknowledgements}
The authors would like to thank Stefan Hallerstede, David Schneider and Thomas
-Stiehl for fruitful discussions during the writing of the paper.
+Stiehl for fruitful discussions and detailed feedback during the writing of the
+paper.
\bibliographystyle{abbrv}
\bibliography{paper}
From antocuni at codespeak.net Mon Oct 18 16:45:19 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 18 Oct 2010 16:45:19 +0200 (CEST)
Subject: [pypy-svn] r78047 - in pypy/trunk/pypy: . annotation
annotation/test config doc/config jit/backend/llgraph
jit/backend/llsupport jit/backend/llsupport/test
jit/backend/test jit/backend/x86/test jit/codewriter
jit/codewriter/test jit/metainterp jit/metainterp/optimizeopt
jit/metainterp/test jit/tl module/_ffi module/_rawffi
module/_rawffi/test module/pypyjit module/pypyjit/test rlib
rlib/test rpython rpython/lltypesystem rpython/test
translator/c/test
Message-ID: <20101018144519.D2048282B90@codespeak.net>
Author: antocuni
Date: Mon Oct 18 16:45:16 2010
New Revision: 78047
Added:
pypy/trunk/pypy/doc/config/objspace.usemodules._ffi.txt
- copied unchanged from r78044, pypy/branch/jitffi/pypy/doc/config/objspace.usemodules._ffi.txt
pypy/trunk/pypy/jit/backend/llsupport/ffisupport.py
- copied unchanged from r78044, pypy/branch/jitffi/pypy/jit/backend/llsupport/ffisupport.py
pypy/trunk/pypy/jit/backend/llsupport/test/test_ffisupport.py
- copied unchanged from r78044, pypy/branch/jitffi/pypy/jit/backend/llsupport/test/test_ffisupport.py
pypy/trunk/pypy/jit/metainterp/optimizeopt/fficall.py
- copied unchanged from r78044, pypy/branch/jitffi/pypy/jit/metainterp/optimizeopt/fficall.py
pypy/trunk/pypy/jit/metainterp/test/test_fficall.py
- copied unchanged from r78044, pypy/branch/jitffi/pypy/jit/metainterp/test/test_fficall.py
pypy/trunk/pypy/jit/metainterp/test/test_optimizefficall.py
- copied unchanged from r78044, pypy/branch/jitffi/pypy/jit/metainterp/test/test_optimizefficall.py
pypy/trunk/pypy/module/_ffi/
- copied from r78044, pypy/branch/jitffi/pypy/module/_ffi/
pypy/trunk/pypy/rlib/clibffi.py
- copied unchanged from r78044, pypy/branch/jitffi/pypy/rlib/clibffi.py
pypy/trunk/pypy/rlib/libffi.py
- copied unchanged from r78044, pypy/branch/jitffi/pypy/rlib/libffi.py
pypy/trunk/pypy/rlib/test/test_clibffi.py
- copied unchanged from r78044, pypy/branch/jitffi/pypy/rlib/test/test_clibffi.py
pypy/trunk/pypy/rlib/test/test_libffi.py
- copied unchanged from r78044, pypy/branch/jitffi/pypy/rlib/test/test_libffi.py
Modified:
pypy/trunk/pypy/ (props changed)
pypy/trunk/pypy/annotation/model.py
pypy/trunk/pypy/annotation/test/test_model.py
pypy/trunk/pypy/config/pypyoption.py
pypy/trunk/pypy/jit/backend/llgraph/llimpl.py
pypy/trunk/pypy/jit/backend/llgraph/runner.py
pypy/trunk/pypy/jit/backend/llsupport/descr.py
pypy/trunk/pypy/jit/backend/llsupport/llmodel.py
pypy/trunk/pypy/jit/backend/test/runner_test.py
pypy/trunk/pypy/jit/backend/x86/test/test_zrpy_gc.py
pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py
pypy/trunk/pypy/jit/codewriter/effectinfo.py
pypy/trunk/pypy/jit/codewriter/jtransform.py
pypy/trunk/pypy/jit/codewriter/support.py
pypy/trunk/pypy/jit/codewriter/test/test_effectinfo.py
pypy/trunk/pypy/jit/metainterp/executor.py
pypy/trunk/pypy/jit/metainterp/optimize_nopspec.py (props changed)
pypy/trunk/pypy/jit/metainterp/optimizeopt/__init__.py
pypy/trunk/pypy/jit/metainterp/optimizeopt/virtualize.py
pypy/trunk/pypy/jit/metainterp/resoperation.py
pypy/trunk/pypy/jit/metainterp/test/test_loop_nopspec.py (props changed)
pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py
pypy/trunk/pypy/jit/metainterp/warmstate.py
pypy/trunk/pypy/jit/tl/jittest.py (props changed)
pypy/trunk/pypy/jit/tl/pypyjit.py
pypy/trunk/pypy/jit/tl/pypyjit_child.py
pypy/trunk/pypy/module/_rawffi/__init__.py
pypy/trunk/pypy/module/_rawffi/callback.py
pypy/trunk/pypy/module/_rawffi/interp_rawffi.py
pypy/trunk/pypy/module/_rawffi/structure.py
pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py
pypy/trunk/pypy/module/pypyjit/policy.py
pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
pypy/trunk/pypy/rlib/jit.py
pypy/trunk/pypy/rlib/test/test_jit.py
pypy/trunk/pypy/rlib/test/test_rdynload.py
pypy/trunk/pypy/rpython/lltypesystem/lltype.py
pypy/trunk/pypy/rpython/rpbc.py
pypy/trunk/pypy/rpython/test/test_rclass.py
pypy/trunk/pypy/translator/c/test/test_newgc.py
Log:
merge the jitffi branch: it provides a new jit-friendly rlib.libffi module,
and an app-level _ffi module which exposes it.
Modified: pypy/trunk/pypy/annotation/model.py
==============================================================================
--- pypy/trunk/pypy/annotation/model.py (original)
+++ pypy/trunk/pypy/annotation/model.py Mon Oct 18 16:45:16 2010
@@ -574,11 +574,11 @@
NUMBER = object()
annotation_to_ll_map = [
+ (SomeSingleFloat(), lltype.SingleFloat),
(s_None, lltype.Void), # also matches SomeImpossibleValue()
(s_Bool, lltype.Bool),
(SomeInteger(knowntype=r_ulonglong), NUMBER),
(SomeFloat(), lltype.Float),
- (SomeSingleFloat(), lltype.SingleFloat),
(SomeChar(), lltype.Char),
(SomeUnicodeCodePoint(), lltype.UniChar),
(SomeAddress(), llmemory.Address),
Modified: pypy/trunk/pypy/annotation/test/test_model.py
==============================================================================
--- pypy/trunk/pypy/annotation/test/test_model.py (original)
+++ pypy/trunk/pypy/annotation/test/test_model.py Mon Oct 18 16:45:16 2010
@@ -128,7 +128,7 @@
assert isinstance(s_p, SomeOOInstance) and s_p.ootype == C
def test_annotation_to_lltype():
- from pypy.rlib.rarithmetic import r_uint
+ from pypy.rlib.rarithmetic import r_uint, r_singlefloat
s_i = SomeInteger()
s_pos = SomeInteger(nonneg=True)
s_1 = SomeInteger(nonneg=True); s_1.const = 1
@@ -151,6 +151,9 @@
C = ootype.Instance('C', ROOT, {})
ref = SomeOOInstance(C)
assert annotation_to_lltype(ref) == C
+ s_singlefloat = SomeSingleFloat()
+ s_singlefloat.const = r_singlefloat(0.0)
+ assert annotation_to_lltype(s_singlefloat) == lltype.SingleFloat
def test_ll_union():
PS1 = lltype.Ptr(lltype.GcStruct('s'))
Modified: pypy/trunk/pypy/config/pypyoption.py
==============================================================================
--- pypy/trunk/pypy/config/pypyoption.py (original)
+++ pypy/trunk/pypy/config/pypyoption.py Mon Oct 18 16:45:16 2010
@@ -73,9 +73,9 @@
}
module_import_dependencies = {
- # no _rawffi if importing pypy.rlib.libffi raises ImportError
+ # no _rawffi if importing pypy.rlib.clibffi raises ImportError
# or CompilationError
- "_rawffi" : ["pypy.rlib.libffi"],
+ "_rawffi" : ["pypy.rlib.clibffi"],
"zlib" : ["pypy.rlib.rzlib"],
"bz2" : ["pypy.module.bz2.interp_bz2"],
Modified: pypy/trunk/pypy/jit/backend/llgraph/llimpl.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llgraph/llimpl.py (original)
+++ pypy/trunk/pypy/jit/backend/llgraph/llimpl.py Mon Oct 18 16:45:16 2010
@@ -10,7 +10,7 @@
BoxInt, BoxPtr, BoxObj, BoxFloat,
REF, INT, FLOAT)
from pypy.jit.codewriter import heaptracker
-from pypy.rpython.lltypesystem import lltype, llmemory, rclass, rstr
+from pypy.rpython.lltypesystem import lltype, llmemory, rclass, rstr, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.module.support import LLSupport, OOSupport
from pypy.rpython.llinterp import LLException
@@ -305,12 +305,12 @@
loop = _from_opaque(loop)
loop.operations.append(Operation(opnum))
-def compile_add_descr(loop, ofs, type):
+def compile_add_descr(loop, ofs, type, arg_types):
from pypy.jit.backend.llgraph.runner import Descr
loop = _from_opaque(loop)
op = loop.operations[-1]
assert isinstance(type, str) and len(type) == 1
- op.descr = Descr(ofs, type)
+ op.descr = Descr(ofs, type, arg_types=arg_types)
def compile_add_loop_token(loop, descr):
if we_are_translated():
@@ -801,7 +801,7 @@
else:
raise TypeError(x)
try:
- return _do_call_common(func, args_in_order)
+ return _do_call_common(func, args_in_order, calldescr)
except LLException, lle:
_last_exception = lle
d = {'v': None,
@@ -1018,6 +1018,9 @@
if isinstance(TYPE, lltype.Ptr):
if isinstance(x, (int, long, llmemory.AddressAsInt)):
x = llmemory.cast_int_to_adr(x)
+ if TYPE is rffi.VOIDP:
+ # assume that we want a "C-style" cast, without typechecking the value
+ return rffi.cast(TYPE, x)
return llmemory.cast_adr_to_ptr(x, TYPE)
elif TYPE == llmemory.Address:
if isinstance(x, (int, long, llmemory.AddressAsInt)):
@@ -1411,10 +1414,26 @@
def do_call_pushfloat(x):
_call_args_f.append(x)
-def _do_call_common(f, args_in_order=None):
+kind2TYPE = {
+ 'i': lltype.Signed,
+ 'f': lltype.Float,
+ 'v': lltype.Void,
+ }
+
+def _do_call_common(f, args_in_order=None, calldescr=None):
ptr = llmemory.cast_int_to_adr(f).ptr
- FUNC = lltype.typeOf(ptr).TO
- ARGS = FUNC.ARGS
+ PTR = lltype.typeOf(ptr)
+ if PTR == rffi.VOIDP:
+ # it's a pointer to a C function, so we don't have a precise
+ # signature: create one from the descr
+ ARGS = map(kind2TYPE.get, calldescr.arg_types)
+ RESULT = kind2TYPE[calldescr.typeinfo]
+ FUNC = lltype.FuncType(ARGS, RESULT)
+ func_to_call = rffi.cast(lltype.Ptr(FUNC), ptr)
+ else:
+ FUNC = PTR.TO
+ ARGS = FUNC.ARGS
+ func_to_call = ptr._obj._callable
args = cast_call_args(ARGS, _call_args_i, _call_args_r, _call_args_f,
args_in_order)
del _call_args_i[:]
@@ -1426,7 +1445,7 @@
result = llinterp.eval_graph(ptr._obj.graph, args)
# ^^^ may raise, in which case we get an LLException
else:
- result = ptr._obj._callable(*args)
+ result = func_to_call(*args)
return result
def do_call_void(f):
Modified: pypy/trunk/pypy/jit/backend/llgraph/runner.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llgraph/runner.py (original)
+++ pypy/trunk/pypy/jit/backend/llgraph/runner.py Mon Oct 18 16:45:16 2010
@@ -154,7 +154,7 @@
llimpl.compile_add(c, op.getopnum())
descr = op.getdescr()
if isinstance(descr, Descr):
- llimpl.compile_add_descr(c, descr.ofs, descr.typeinfo)
+ llimpl.compile_add_descr(c, descr.ofs, descr.typeinfo, descr.arg_types)
if isinstance(descr, history.LoopToken) and op.getopnum() != rop.JUMP:
llimpl.compile_add_loop_token(c, descr)
if self.is_oo and isinstance(descr, (OODescr, MethDescr)):
@@ -297,6 +297,18 @@
return self.getdescr(0, token[0], extrainfo=extrainfo,
arg_types=''.join(arg_types))
+ def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo=None):
+ from pypy.jit.backend.llsupport.ffisupport import get_ffi_type_kind
+ arg_types = []
+ for arg in ffi_args:
+ kind = get_ffi_type_kind(arg)
+ if kind != history.VOID:
+ arg_types.append(kind)
+ reskind = get_ffi_type_kind(ffi_result)
+ return self.getdescr(0, reskind, extrainfo=extrainfo,
+ arg_types=''.join(arg_types))
+
+
def grab_exc_value(self):
return llimpl.grab_exc_value()
Modified: pypy/trunk/pypy/jit/backend/llsupport/descr.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llsupport/descr.py (original)
+++ pypy/trunk/pypy/jit/backend/llsupport/descr.py Mon Oct 18 16:45:16 2010
@@ -307,6 +307,21 @@
_return_type = history.INT
call_stub = staticmethod(lambda func, args_i, args_r, args_f: 0)
+class DynamicIntCallDescr(BaseIntCallDescr):
+ """
+ calldescr that works for every integer type, by explicitly passing it the
+ size of the result. Used only by get_call_descr_dynamic
+ """
+ _clsname = 'DynamicIntCallDescr'
+
+ def __init__(self, arg_classes, result_size, extrainfo=None):
+ BaseIntCallDescr.__init__(self, arg_classes, extrainfo)
+ self._result_size = result_size
+
+ def get_result_size(self, translate_support_code):
+ return self._result_size
+
+
class NonGcPtrCallDescr(BaseIntCallDescr):
_clsname = 'NonGcPtrCallDescr'
def get_result_size(self, translate_support_code):
Modified: pypy/trunk/pypy/jit/backend/llsupport/llmodel.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llsupport/llmodel.py (original)
+++ pypy/trunk/pypy/jit/backend/llsupport/llmodel.py Mon Oct 18 16:45:16 2010
@@ -17,6 +17,7 @@
from pypy.jit.backend.llsupport.descr import get_call_descr
from pypy.jit.backend.llsupport.descr import BaseIntCallDescr, GcPtrCallDescr
from pypy.jit.backend.llsupport.descr import FloatCallDescr, VoidCallDescr
+from pypy.jit.backend.llsupport.ffisupport import get_call_descr_dynamic
from pypy.rpython.annlowlevel import cast_instance_to_base_ptr
@@ -231,6 +232,9 @@
def calldescrof(self, FUNC, ARGS, RESULT, extrainfo=None):
return get_call_descr(self.gc_ll_descr, ARGS, RESULT, extrainfo)
+ def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo=None):
+ return get_call_descr_dynamic(ffi_args, ffi_result, extrainfo)
+
def get_overflow_error(self):
ovf_vtable = self.cast_adr_to_int(self._ovf_error_vtable)
ovf_inst = lltype.cast_opaque_ptr(llmemory.GCREF,
Modified: pypy/trunk/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/trunk/pypy/jit/backend/test/runner_test.py Mon Oct 18 16:45:16 2010
@@ -421,6 +421,7 @@
assert x == 3.5 - 42
def test_call(self):
+ from pypy.rlib.libffi import types
def func_int(a, b):
return a + b
@@ -428,23 +429,31 @@
return chr(ord(c) + ord(c1))
functions = [
- (func_int, lltype.Signed, 655360),
- (func_int, rffi.SHORT, 1213),
- (func_char, lltype.Char, 12)
+ (func_int, lltype.Signed, types.sint, 655360),
+ (func_int, rffi.SHORT, types.sint16, 1213),
+ (func_char, lltype.Char, types.uchar, 12)
]
- for func, TP, num in functions:
+ for func, TP, ffi_type, num in functions:
cpu = self.cpu
#
FPTR = self.Ptr(self.FuncType([TP, TP], TP))
func_ptr = llhelper(FPTR, func)
FUNC = deref(FPTR)
- calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
funcbox = self.get_funcbox(cpu, func_ptr)
+ # first, try it with the "normal" calldescr
+ calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
res = self.execute_operation(rop.CALL,
[funcbox, BoxInt(num), BoxInt(num)],
'int', descr=calldescr)
assert res.value == 2 * num
+ # then, try it with the dynamic calldescr
+ dyn_calldescr = cpu.calldescrof_dynamic([ffi_type, ffi_type], ffi_type)
+ res = self.execute_operation(rop.CALL,
+ [funcbox, BoxInt(num), BoxInt(num)],
+ 'int', descr=dyn_calldescr)
+ assert res.value == 2 * num
+
if cpu.supports_floats:
def func(f0, f1, f2, f3, f4, f5, f6, i0, i1, f7, f8, f9):
@@ -507,6 +516,23 @@
'int', descr=calldescr)
assert res.value == func_ints(*args)
+ def test_call_to_c_function(self):
+ from pypy.rlib.libffi import CDLL, types, ArgChain
+ libc = CDLL('libc.so.6')
+ c_tolower = libc.getpointer('tolower', [types.uchar], types.sint)
+ argchain = ArgChain().arg(ord('A'))
+ assert c_tolower.call(argchain, rffi.INT) == ord('a')
+
+ func_adr = llmemory.cast_ptr_to_adr(c_tolower.funcsym)
+ funcbox = ConstInt(heaptracker.adr2int(func_adr))
+ calldescr = self.cpu.calldescrof_dynamic([types.uchar], types.sint)
+ res = self.execute_operation(rop.CALL,
+ [funcbox, BoxInt(ord('A'))],
+ 'int',
+ descr=calldescr)
+ assert res.value == ord('a')
+
+
def test_field_basic(self):
t_box, T_box = self.alloc_instance(self.T)
fielddescr = self.cpu.fielddescrof(self.S, 'value')
Modified: pypy/trunk/pypy/jit/backend/x86/test/test_zrpy_gc.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/test/test_zrpy_gc.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/test/test_zrpy_gc.py Mon Oct 18 16:45:16 2010
@@ -191,6 +191,33 @@
def run_orig(self, name, n, x):
self.main_allfuncs(name, n, x)
+ def define_libffi_workaround(cls):
+ # XXX: this is a workaround for a bug in database.py. It seems that
+ # the problem is triggered by optimizeopt/fficall.py, and in
+ # particular by the ``cast_base_ptr_to_instance(Func, llfunc)``: in
+ # these tests, that line is the only place where libffi.Func is
+ # referenced.
+ #
+ # The problem occurs because the gctransformer tries to annotate a
+ # low-level helper to call the __del__ of libffi.Func when it's too
+ # late.
+ #
+ # This workaround works by forcing the annotator (and all the rest of
+ # the toolchain) to see libffi.Func in a "proper" context, not just as
+ # the target of cast_base_ptr_to_instance. Note that the function
+ # below is *never* called by any actual test, it's just annotated.
+ #
+ from pypy.rlib.libffi import get_libc_name, CDLL, types, ArgChain
+ libc_name = get_libc_name()
+ def f(n, x, *args):
+ libc = CDLL(libc_name)
+ ptr = libc.getpointer('labs', [types.slong], types.slong)
+ chain = ArgChain()
+ chain.arg(n)
+ n = ptr.call(chain, lltype.Signed)
+ return (n, x) + args
+ return None, f, None
+
def define_compile_framework_1(cls):
# a moving GC. Supports malloc_varsize_nonmovable. Simple test, works
# without write_barriers and root stack enumeration.
Modified: pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py Mon Oct 18 16:45:16 2010
@@ -1,4 +1,4 @@
-import py, os
+import py, os, sys
from pypy.tool.udir import udir
from pypy.rlib.jit import JitDriver, OPTIMIZER_FULL, unroll_parameters
from pypy.rlib.jit import PARAMETERS, dont_look_inside
@@ -63,8 +63,32 @@
if k - abs(j): raise ValueError
if k - abs(-j): raise ValueError
return total * 10
+ #
+ from pypy.rpython.lltypesystem import lltype, rffi
+ from pypy.rlib.libffi import types, CDLL, ArgChain
+ from pypy.rlib.test.test_libffi import get_libm_name
+ libm_name = get_libm_name(sys.platform)
+ jitdriver2 = JitDriver(greens=[], reds = ['i', 'func', 'res', 'x'])
+ def libffi_stuff(i, j):
+ lib = CDLL(libm_name)
+ func = lib.getpointer('fabs', [types.double], types.double)
+ res = 0.0
+ x = float(j)
+ while i > 0:
+ jitdriver2.jit_merge_point(i=i, res=res, func=func, x=x)
+ jitdriver2.can_enter_jit(i=i, res=res, func=func, x=x)
+ func = hint(func, promote=True)
+ argchain = ArgChain()
+ argchain.arg(x)
+ res = func.call(argchain, rffi.DOUBLE)
+ i -= 1
+ return res
+ #
+ def main(i, j):
+ return f(i, j) + libffi_stuff(i, j)
+ expected = f(40, -49)
res = self.meta_interp(f, [40, -49])
- assert res == f(40, -49)
+ assert res == expected
def test_direct_assembler_call_translates(self):
class Thing(object):
Modified: pypy/trunk/pypy/jit/codewriter/effectinfo.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/effectinfo.py (original)
+++ pypy/trunk/pypy/jit/codewriter/effectinfo.py Mon Oct 18 16:45:16 2010
@@ -19,7 +19,7 @@
OS_NONE = 0 # normal case, no oopspec
OS_ARRAYCOPY = 1 # "list.ll_arraycopy"
OS_STR2UNICODE = 2 # "str.str2unicode"
-
+ #
OS_STR_CONCAT = 22 # "stroruni.concat"
OS_STR_SLICE = 23 # "stroruni.slice"
OS_STR_EQUAL = 24 # "stroruni.equal"
@@ -30,7 +30,7 @@
OS_STREQ_NONNULL_CHAR = 29 # s1 == char (assert s1!=NULL)
OS_STREQ_CHECKNULL_CHAR = 30 # s1!=NULL and s1==char
OS_STREQ_LENGTHOK = 31 # s1 == s2 (assert len(s1)==len(s2))
-
+ #
OS_UNI_CONCAT = 42 #
OS_UNI_SLICE = 43 #
OS_UNI_EQUAL = 44 #
@@ -42,6 +42,10 @@
OS_UNIEQ_CHECKNULL_CHAR = 50 # STR, in the same order)
OS_UNIEQ_LENGTHOK = 51 #
_OS_offset_uni = OS_UNI_CONCAT - OS_STR_CONCAT
+ #
+ OS_LIBFFI_PREPARE = 60
+ OS_LIBFFI_PUSH_ARG = 61
+ OS_LIBFFI_CALL = 62
def __new__(cls, readonly_descrs_fields,
write_descrs_fields, write_descrs_arrays,
Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/trunk/pypy/jit/codewriter/jtransform.py Mon Oct 18 16:45:16 2010
@@ -320,6 +320,8 @@
prepare = self._handle_str2unicode_call
elif oopspec_name.startswith('virtual_ref'):
prepare = self._handle_virtual_ref_call
+ elif oopspec_name.startswith('libffi_'):
+ prepare = self._handle_libffi_call
else:
prepare = self.prepare_builtin_call
try:
@@ -1030,8 +1032,10 @@
# ----------
# Strings and Unicodes.
- def _handle_oopspec_call(self, op, args, oopspecindex):
+ def _handle_oopspec_call(self, op, args, oopspecindex, extraeffect=None):
calldescr = self.callcontrol.getcalldescr(op, oopspecindex)
+ if extraeffect:
+ calldescr.get_extra_info().extraeffect = extraeffect
if isinstance(op.args[0].value, str):
pass # for tests only
else:
@@ -1132,6 +1136,23 @@
vrefinfo.JIT_VIRTUAL_REF)
return SpaceOperation(oopspec_name, list(args), op.result)
+ # -----------
+ # rlib.libffi
+
+ def _handle_libffi_call(self, op, oopspec_name, args):
+ if oopspec_name == 'libffi_prepare_call':
+ oopspecindex = EffectInfo.OS_LIBFFI_PREPARE
+ extraeffect = EffectInfo.EF_CANNOT_RAISE
+ elif oopspec_name.startswith('libffi_push_'):
+ oopspecindex = EffectInfo.OS_LIBFFI_PUSH_ARG
+ extraeffect = EffectInfo.EF_CANNOT_RAISE
+ elif oopspec_name.startswith('libffi_call_'):
+ oopspecindex = EffectInfo.OS_LIBFFI_CALL
+ extraeffect = EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE
+ else:
+ assert False, 'unsupported oopspec: %s' % oopspec_name
+ return self._handle_oopspec_call(op, args, oopspecindex, extraeffect)
+
def rewrite_op_jit_force_virtual(self, op):
return self._do_builtin_call(op)
Modified: pypy/trunk/pypy/jit/codewriter/support.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/support.py (original)
+++ pypy/trunk/pypy/jit/codewriter/support.py Mon Oct 18 16:45:16 2010
@@ -1,5 +1,5 @@
import sys
-from pypy.rpython.lltypesystem import lltype, rclass
+from pypy.rpython.lltypesystem import lltype, rclass, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython import rlist
from pypy.rpython.lltypesystem import rstr as ll_rstr, rdict as ll_rdict
@@ -8,6 +8,7 @@
from pypy.rpython.ootypesystem import rdict as oo_rdict
from pypy.rpython.llinterp import LLInterpreter
from pypy.rpython.extregistry import ExtRegistryEntry
+from pypy.rpython.annlowlevel import cast_base_ptr_to_instance
from pypy.translator.simplify import get_funcobj
from pypy.translator.unsimplify import split_block
from pypy.objspace.flow.model import Constant
@@ -217,6 +218,33 @@
else:
return x
+
+# libffi support
+# --------------
+
+def func(llfunc):
+ from pypy.rlib.libffi import Func
+ return cast_base_ptr_to_instance(Func, llfunc)
+
+def _ll_1_libffi_prepare_call(llfunc):
+ return func(llfunc)._prepare()
+
+def _ll_4_libffi_push_int(llfunc, value, ll_args, i):
+ return func(llfunc)._push_int(value, ll_args, i)
+
+def _ll_4_libffi_push_float(llfunc, value, ll_args, i):
+ return func(llfunc)._push_float(value, ll_args, i)
+
+def _ll_3_libffi_call_int(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, rffi.LONG)
+
+def _ll_3_libffi_call_float(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, rffi.DOUBLE)
+
+def _ll_3_libffi_call_void(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, lltype.Void)
+
+
# in the following calls to builtins, the JIT is allowed to look inside:
inline_calls_to = [
('int_floordiv_ovf_zer', [lltype.Signed, lltype.Signed], lltype.Signed),
Modified: pypy/trunk/pypy/jit/codewriter/test/test_effectinfo.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/test/test_effectinfo.py (original)
+++ pypy/trunk/pypy/jit/codewriter/test/test_effectinfo.py Mon Oct 18 16:45:16 2010
@@ -1,7 +1,8 @@
from pypy.rpython.lltypesystem.rclass import OBJECT
from pypy.rpython.lltypesystem import lltype
from pypy.rpython.ootypesystem import ootype
-from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze
+from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze,\
+ EffectInfo
class FakeCPU:
def fielddescrof(self, T, fieldname):
@@ -9,6 +10,14 @@
def arraydescrof(self, A):
return ('arraydescr', A)
+def test_no_oopspec_duplicate():
+ # check that all the various EffectInfo.OS_* have unique values
+ oopspecs = set()
+ for name, value in EffectInfo.__dict__.iteritems():
+ if name.startswith('OS_'):
+ assert value not in oopspecs
+ oopspecs.add(value)
+
def test_include_read_field():
S = lltype.GcStruct("S", ("a", lltype.Signed))
effects = frozenset([("readstruct", lltype.Ptr(S), "a")])
Modified: pypy/trunk/pypy/jit/metainterp/executor.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/executor.py (original)
+++ pypy/trunk/pypy/jit/metainterp/executor.py Mon Oct 18 16:45:16 2010
@@ -80,6 +80,9 @@
do_call_loopinvariant = do_call
do_call_may_force = do_call
+def do_call_c(cpu, metainterp, argboxes, descr):
+ raise NotImplementedError("Should never be called directly")
+
def do_getarrayitem_gc(cpu, _, arraybox, indexbox, arraydescr):
array = arraybox.getref_base()
index = indexbox.getint()
Modified: pypy/trunk/pypy/jit/metainterp/optimizeopt/__init__.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/optimizeopt/__init__.py (original)
+++ pypy/trunk/pypy/jit/metainterp/optimizeopt/__init__.py Mon Oct 18 16:45:16 2010
@@ -3,6 +3,7 @@
from pypy.jit.metainterp.optimizeopt.intbounds import OptIntBounds
from pypy.jit.metainterp.optimizeopt.virtualize import OptVirtualize
from pypy.jit.metainterp.optimizeopt.heap import OptHeap
+from pypy.jit.metainterp.optimizeopt.fficall import OptFfiCall
from pypy.jit.metainterp.optimizeopt.string import OptString
def optimize_loop_1(metainterp_sd, loop, virtuals=True):
@@ -16,6 +17,7 @@
OptVirtualize(),
OptString(),
OptHeap(),
+ OptFfiCall(),
]
optimizer = Optimizer(metainterp_sd, loop, optimizations, virtuals)
optimizer.propagate_all_forward()
Modified: pypy/trunk/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/trunk/pypy/jit/metainterp/optimizeopt/virtualize.py Mon Oct 18 16:45:16 2010
@@ -134,6 +134,11 @@
fielddescrs = self._get_field_descr_list()
return modifier.make_virtual(self.known_class, fielddescrs)
+ def __repr__(self):
+ cls_name = self.known_class.value.adr.ptr._obj._TYPE._name
+ field_names = [field.name for field in self._fields]
+ return "" % (cls_name, field_names)
+
class VStructValue(AbstractVirtualStructValue):
def __init__(self, optimizer, structdescr, keybox, source_op=None):
Modified: pypy/trunk/pypy/jit/metainterp/resoperation.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/resoperation.py (original)
+++ pypy/trunk/pypy/jit/metainterp/resoperation.py Mon Oct 18 16:45:16 2010
@@ -463,7 +463,7 @@
'_CANRAISE_FIRST', # ----- start of can_raise operations -----
'CALL/*d',
- 'CALL_ASSEMBLER/*d',
+ 'CALL_ASSEMBLER/*d', # call already compiled assembler
'CALL_MAY_FORCE/*d',
'CALL_LOOPINVARIANT/*d',
#'OOSEND', # ootype operation
Modified: pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py Mon Oct 18 16:45:16 2010
@@ -132,14 +132,21 @@
# ____________________________________________________________
def equaloplists(oplist1, oplist2, strict_fail_args=True, remap={}):
- print '-'*20, 'Comparing lists', '-'*20
+ # try to use the full width of the terminal to display the list
+ # unfortunately, does not work with the default capture method of py.test
+ # (which is fd), you you need to use either -s or --capture=sys, else you
+ # get the standard 80 columns width
+ totwidth = py.io.get_terminal_width()
+ width = totwidth / 2 - 1
+ print ' Comparing lists '.center(totwidth, '-')
+ print '%s| %s' % ('optimized'.center(width), 'expected'.center(width))
for op1, op2 in zip(oplist1, oplist2):
txt1 = str(op1)
txt2 = str(op2)
while txt1 or txt2:
- print '%-39s| %s' % (txt1[:39], txt2[:39])
- txt1 = txt1[39:]
- txt2 = txt2[39:]
+ print '%s| %s' % (txt1[:width].ljust(width), txt2[:width])
+ txt1 = txt1[width:]
+ txt2 = txt2[width:]
assert op1.getopnum() == op2.getopnum()
assert op1.numargs() == op2.numargs()
for i in range(op1.numargs()):
@@ -262,6 +269,10 @@
expected = self.parse(optops)
print '\n'.join([str(o) for o in loop.operations])
self.assert_equal(loop, expected)
+ return loop
+
+
+class OptimizeOptTest(BaseTestOptimizeOpt):
def test_simple(self):
ops = """
@@ -2643,7 +2654,7 @@
''', rop.GUARD_TRUE)
-class TestLLtype(BaseTestOptimizeOpt, LLtypeMixin):
+class TestLLtype(OptimizeOptTest, LLtypeMixin):
def test_residual_call_does_not_invalidate_caches(self):
ops = """
@@ -4533,7 +4544,7 @@
# can be raised by ll_str2unicode()
-##class TestOOtype(BaseTestOptimizeOpt, OOtypeMixin):
+##class TestOOtype(OptimizeOptTest, OOtypeMixin):
## def test_instanceof(self):
## ops = """
Modified: pypy/trunk/pypy/jit/metainterp/warmstate.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/warmstate.py (original)
+++ pypy/trunk/pypy/jit/metainterp/warmstate.py Mon Oct 18 16:45:16 2010
@@ -1,5 +1,5 @@
import sys
-from pypy.rpython.lltypesystem import lltype, llmemory, rstr
+from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.annlowlevel import hlstr, llstr, cast_base_ptr_to_instance
from pypy.rpython.annlowlevel import cast_object_to_ptr
@@ -24,7 +24,11 @@
"""
INPUT = lltype.typeOf(x)
if INPUT is lltype.Signed:
- return lltype.cast_primitive(TYPE, x) # XXX missing: Ptr(non-gc)
+ if isinstance(TYPE, lltype.Ptr) and TYPE.TO._gckind == 'raw':
+ # non-gc pointer
+ return rffi.cast(TYPE, x)
+ else:
+ return lltype.cast_primitive(TYPE, x)
elif INPUT is lltype.Float:
assert TYPE is lltype.Float
return x
Modified: pypy/trunk/pypy/jit/tl/pypyjit.py
==============================================================================
--- pypy/trunk/pypy/jit/tl/pypyjit.py (original)
+++ pypy/trunk/pypy/jit/tl/pypyjit.py Mon Oct 18 16:45:16 2010
@@ -40,6 +40,9 @@
config.objspace.usemodules.array = True
config.objspace.usemodules._weakref = False
config.objspace.usemodules._sre = False
+#
+config.objspace.usemodules._ffi = True
+#
set_pypy_opt_level(config, level='jit')
config.objspace.std.withinlineddict = True
Modified: pypy/trunk/pypy/jit/tl/pypyjit_child.py
==============================================================================
--- pypy/trunk/pypy/jit/tl/pypyjit_child.py (original)
+++ pypy/trunk/pypy/jit/tl/pypyjit_child.py Mon Oct 18 16:45:16 2010
@@ -16,7 +16,7 @@
interp.heap.malloc_nonmovable = returns_null # XXX
from pypy.jit.backend.llgraph.runner import LLtypeCPU
- LLtypeCPU.supports_floats = False # for now
+ #LLtypeCPU.supports_floats = False # for now
apply_jit(interp, graph, LLtypeCPU)
Modified: pypy/trunk/pypy/module/_rawffi/__init__.py
==============================================================================
--- pypy/trunk/pypy/module/_rawffi/__init__.py (original)
+++ pypy/trunk/pypy/module/_rawffi/__init__.py Mon Oct 18 16:45:16 2010
@@ -1,5 +1,5 @@
-""" Low-level interface to libffi
+""" Low-level interface to clibffi
"""
from pypy.interpreter.mixedmodule import MixedModule
@@ -39,11 +39,11 @@
if hasattr(interp_rawffi, 'check_HRESULT'):
Module.interpleveldefs['check_HRESULT'] = 'interp_rawffi.check_HRESULT'
- from pypy.rlib import libffi
+ from pypy.rlib import clibffi
for name in ['FUNCFLAG_STDCALL', 'FUNCFLAG_CDECL', 'FUNCFLAG_PYTHONAPI',
]:
- if hasattr(libffi, name):
- Module.interpleveldefs[name] = "space.wrap(%r)" % getattr(libffi, name)
+ if hasattr(clibffi, name):
+ Module.interpleveldefs[name] = "space.wrap(%r)" % getattr(clibffi, name)
super(Module, cls).buildloaders()
buildloaders = classmethod(buildloaders)
Modified: pypy/trunk/pypy/module/_rawffi/callback.py
==============================================================================
--- pypy/trunk/pypy/module/_rawffi/callback.py (original)
+++ pypy/trunk/pypy/module/_rawffi/callback.py Mon Oct 18 16:45:16 2010
@@ -8,8 +8,8 @@
from pypy.module._rawffi.array import get_elem, push_elem
from pypy.module._rawffi.interp_rawffi import W_DataInstance, letter2tp, \
wrap_value, unwrap_value, unwrap_truncate_int
-from pypy.rlib.libffi import USERDATA_P, CallbackFuncPtr, FUNCFLAG_CDECL
-from pypy.rlib.libffi import ffi_type_void
+from pypy.rlib.clibffi import USERDATA_P, CallbackFuncPtr, FUNCFLAG_CDECL
+from pypy.rlib.clibffi import ffi_type_void
from pypy.module._rawffi.tracker import tracker
from pypy.interpreter.error import OperationError
from pypy.interpreter import gateway
Modified: pypy/trunk/pypy/module/_rawffi/interp_rawffi.py
==============================================================================
--- pypy/trunk/pypy/module/_rawffi/interp_rawffi.py (original)
+++ pypy/trunk/pypy/module/_rawffi/interp_rawffi.py Mon Oct 18 16:45:16 2010
@@ -5,7 +5,7 @@
from pypy.interpreter.gateway import interp2app, NoneNotWrapped
from pypy.interpreter.typedef import TypeDef, GetSetProperty
-from pypy.rlib.libffi import *
+from pypy.rlib.clibffi import *
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.rlib.unroll import unrolling_iterable
Modified: pypy/trunk/pypy/module/_rawffi/structure.py
==============================================================================
--- pypy/trunk/pypy/module/_rawffi/structure.py (original)
+++ pypy/trunk/pypy/module/_rawffi/structure.py Mon Oct 18 16:45:16 2010
@@ -15,7 +15,7 @@
from pypy.module._rawffi.interp_rawffi import wrap_value, unwrap_value
from pypy.module._rawffi.interp_rawffi import unpack_shape_with_length
from pypy.module._rawffi.interp_rawffi import size_alignment
-from pypy.rlib import libffi
+from pypy.rlib import clibffi
from pypy.rlib.rarithmetic import intmask, r_uint
def unpack_fields(space, w_fields):
@@ -104,14 +104,14 @@
descr_fieldoffset.unwrap_spec = ['self', ObjSpace, str]
# get the corresponding ffi_type
- ffi_struct = lltype.nullptr(libffi.FFI_STRUCT_P.TO)
+ ffi_struct = lltype.nullptr(clibffi.FFI_STRUCT_P.TO)
def get_basic_ffi_type(self):
if not self.ffi_struct:
# Repeated fields are delicate. Consider for example
# struct { int a[5]; }
# or struct { struct {int x;} a[5]; }
- # Seeing no corresponding doc in libffi, let's just repeat
+ # Seeing no corresponding doc in clibffi, let's just repeat
# the field 5 times...
fieldtypes = []
for name, tp in self.fields:
@@ -122,7 +122,7 @@
while count + basic_size <= total_size:
fieldtypes.append(basic_ffi_type)
count += basic_size
- self.ffi_struct = libffi.make_struct_ffitype_e(self.size,
+ self.ffi_struct = clibffi.make_struct_ffitype_e(self.size,
self.alignment,
fieldtypes)
return self.ffi_struct.ffistruct
Modified: pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py
==============================================================================
--- pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py (original)
+++ pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py Mon Oct 18 16:45:16 2010
@@ -191,7 +191,7 @@
prepare_c_example = staticmethod(prepare_c_example)
def setup_class(cls):
- from pypy.rlib.libffi import get_libc_name
+ from pypy.rlib.clibffi import get_libc_name
space = gettestobjspace(usemodules=('_rawffi', 'struct'))
cls.space = space
cls.w_lib_name = space.wrap(cls.prepare_c_example())
Modified: pypy/trunk/pypy/module/pypyjit/policy.py
==============================================================================
--- pypy/trunk/pypy/module/pypyjit/policy.py (original)
+++ pypy/trunk/pypy/module/pypyjit/policy.py Mon Oct 18 16:45:16 2010
@@ -11,7 +11,7 @@
if '.' in modname:
modname, _ = modname.split('.', 1)
if modname in ['pypyjit', 'signal', 'micronumpy', 'math', 'exceptions',
- 'imp', 'sys', 'array']:
+ 'imp', 'sys', 'array', '_ffi']:
return True
return False
Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Mon Oct 18 16:45:16 2010
@@ -79,8 +79,11 @@
class PyPyCJITTests(object):
- def run_source(self, source, expected_max_ops, *testcases):
+ def run_source(self, source, expected_max_ops, *testcases, **kwds):
assert isinstance(expected_max_ops, int)
+ threshold = kwds.pop('threshold', 3)
+ if kwds:
+ raise TypeError, 'Unsupported keyword arguments: %s' % kwds.keys()
source = py.code.Source(source)
filepath = self.tmpdir.join('case%d.py' % self.counter)
logfilepath = filepath.new(ext='.log')
@@ -92,7 +95,7 @@
import sys
try: # make the file runnable by CPython
import pypyjit
- pypyjit.set_param(threshold=3)
+ pypyjit.set_param(threshold=%d)
except ImportError:
pass
@@ -102,7 +105,7 @@
print >> sys.stderr, 'got:', repr(result)
assert result == expected
assert type(result) is type(expected)
- """)
+ """ % threshold)
for testcase in testcases * 2:
print >> f, "check(%r, %r)" % testcase
print >> f, "print 'OK :-)'"
@@ -123,6 +126,7 @@
if self.total_ops > expected_max_ops:
assert 0, "too many operations: got %d, expected maximum %d" % (
self.total_ops, expected_max_ops)
+ return result
def parse_loops(self, opslogfile):
from pypy.jit.metainterp.test.oparser import parse
@@ -1134,6 +1138,39 @@
return sa
''', 88, ([], 1997001))
+ def test__ffi_call(self):
+ from pypy.rlib.test.test_libffi import get_libm_name
+ libm_name = get_libm_name(sys.platform)
+ out = self.run_source('''
+ def main():
+ from _ffi import CDLL, types
+ libm = CDLL('%(libm_name)s')
+ pow = libm.getfunc('pow', [types.double, types.double],
+ types.double)
+ print pow.getaddr()
+ i = 0
+ res = 0
+ while i < 2000:
+ res += pow(2, 3)
+ i += 1
+ return res
+ ''' % locals(),
+ 76, ([], 8.0*2000), threshold=1000)
+ pow_addr = int(out.splitlines()[0])
+ ops = self.get_by_bytecode('CALL_FUNCTION')
+ assert len(ops) == 2 # we get two loops, because of specialization
+ call_function = ops[0]
+ last_ops = [op.getopname() for op in call_function[-5:]]
+ assert last_ops == ['force_token',
+ 'setfield_gc',
+ 'call_may_force',
+ 'guard_not_forced',
+ 'guard_no_exception']
+ call = call_function[-3]
+ assert call.getarg(0).value == pow_addr
+ assert call.getarg(1).value == 2.0
+ assert call.getarg(2).value == 3.0
+
# test_circular
class AppTestJIT(PyPyCJITTests):
Modified: pypy/trunk/pypy/rlib/jit.py
==============================================================================
--- pypy/trunk/pypy/rlib/jit.py (original)
+++ pypy/trunk/pypy/rlib/jit.py Mon Oct 18 16:45:16 2010
@@ -77,6 +77,12 @@
return result
return decorator
+def oopspec(spec):
+ def decorator(func):
+ func.oopspec = spec
+ return func
+ return decorator
+
class Entry(ExtRegistryEntry):
_about_ = hint
Modified: pypy/trunk/pypy/rlib/test/test_jit.py
==============================================================================
--- pypy/trunk/pypy/rlib/test/test_jit.py (original)
+++ pypy/trunk/pypy/rlib/test/test_jit.py Mon Oct 18 16:45:16 2010
@@ -1,10 +1,16 @@
import py
from pypy.rlib.jit import hint, we_are_jitted, JitDriver, purefunction_promote
-from pypy.rlib.jit import JitHintError
+from pypy.rlib.jit import JitHintError, oopspec
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.rpython.lltypesystem import lltype
+def test_oopspec():
+ @oopspec('foobar')
+ def fn():
+ pass
+ assert fn.oopspec == 'foobar'
+
class BaseTestJIT(BaseRtypingTest):
def test_hint(self):
def f():
Modified: pypy/trunk/pypy/rlib/test/test_rdynload.py
==============================================================================
--- pypy/trunk/pypy/rlib/test/test_rdynload.py (original)
+++ pypy/trunk/pypy/rlib/test/test_rdynload.py Mon Oct 18 16:45:16 2010
@@ -1,5 +1,5 @@
from pypy.rlib.rdynload import *
-from pypy.rlib.libffi import get_libc_name
+from pypy.rlib.clibffi import get_libc_name
from pypy.rpython.lltypesystem import rffi, lltype
import py
Modified: pypy/trunk/pypy/rpython/lltypesystem/lltype.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/lltype.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/lltype.py Mon Oct 18 16:45:16 2010
@@ -794,6 +794,8 @@
return llmemory.cast_adr_to_ptr(value, TGT)
elif TGT == llmemory.Address and isinstance(ORIG, Ptr):
return llmemory.cast_ptr_to_adr(value)
+ elif TGT == Signed and isinstance(ORIG, Ptr) and ORIG.TO._gckind == 'raw':
+ return llmemory.cast_adr_to_int(llmemory.cast_ptr_to_adr(value), 'symbolic')
raise TypeError("don't know how to cast from %r to %r" % (ORIG, TGT))
Modified: pypy/trunk/pypy/rpython/rpbc.py
==============================================================================
--- pypy/trunk/pypy/rpython/rpbc.py (original)
+++ pypy/trunk/pypy/rpython/rpbc.py Mon Oct 18 16:45:16 2010
@@ -256,6 +256,8 @@
def convert_const(self, value):
if isinstance(value, types.MethodType) and value.im_self is None:
value = value.im_func # unbound method -> bare function
+ elif isinstance(value, staticmethod):
+ value = value.__get__(42) # hackish, get the function wrapped by staticmethod
if self.lowleveltype is Void:
return None
if value is None:
Modified: pypy/trunk/pypy/rpython/test/test_rclass.py
==============================================================================
--- pypy/trunk/pypy/rpython/test/test_rclass.py (original)
+++ pypy/trunk/pypy/rpython/test/test_rclass.py Mon Oct 18 16:45:16 2010
@@ -319,6 +319,17 @@
res = self.interpret(f, [])
assert res == 42
+ def test_staticmethod2(self):
+ class A(object):
+ f = staticmethod(lambda x, y: x*y)
+ class B(A):
+ f = staticmethod(lambda x, y: x+y)
+ def f():
+ b = B()
+ return b.f(6, 7)
+ res = self.interpret(f, [])
+ assert res == 13
+
def test_is(self):
class A: pass
class B(A): pass
Modified: pypy/trunk/pypy/translator/c/test/test_newgc.py
==============================================================================
--- pypy/trunk/pypy/translator/c/test/test_newgc.py (original)
+++ pypy/trunk/pypy/translator/c/test/test_newgc.py Mon Oct 18 16:45:16 2010
@@ -624,13 +624,13 @@
os.unlink(self.filename)
def define_callback_with_collect(cls):
- from pypy.rlib.libffi import ffi_type_pointer, cast_type_to_ffitype,\
+ from pypy.rlib.clibffi import ffi_type_pointer, cast_type_to_ffitype,\
CDLL, ffi_type_void, CallbackFuncPtr, ffi_type_sint
from pypy.rpython.lltypesystem import rffi, ll2ctypes
import gc
ffi_size_t = cast_type_to_ffitype(rffi.SIZE_T)
- from pypy.rlib.libffi import get_libc_name
+ from pypy.rlib.clibffi import get_libc_name
def callback(ll_args, ll_res, stuff):
gc.collect()
From arigo at codespeak.net Mon Oct 18 17:18:14 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 17:18:14 +0200 (CEST)
Subject: [pypy-svn] r78048 - pypy/branch/rsre-jit/pypy/jit/backend/x86
Message-ID: <20101018151814.68676282B90@codespeak.net>
Author: arigo
Date: Mon Oct 18 17:18:11 2010
New Revision: 78048
Modified:
pypy/branch/rsre-jit/pypy/jit/backend/x86/regalloc.py
Log:
"Implement" jit_debug in the x86 backend.
Modified: pypy/branch/rsre-jit/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/branch/rsre-jit/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/branch/rsre-jit/pypy/jit/backend/x86/regalloc.py Mon Oct 18 17:18:11 2010
@@ -1068,6 +1068,9 @@
def consider_debug_merge_point(self, op):
pass
+ def consider_jit_debug(self, op):
+ pass
+
def get_mark_gc_roots(self, gcrootmap):
shape = gcrootmap.get_basic_shape(IS_X86_64)
for v, val in self.fm.frame_bindings.items():
From arigo at codespeak.net Mon Oct 18 17:24:53 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 17:24:53 +0200 (CEST)
Subject: [pypy-svn] r78049 - in pypy/trunk: dotviewer pypy/jit/backend/x86
pypy/jit/codewriter pypy/jit/codewriter/test
pypy/jit/metainterp pypy/jit/metainterp/optimizeopt
pypy/jit/metainterp/test pypy/rlib pypy/rlib/rsre
pypy/rlib/rsre/test pypy/rlib/test
Message-ID: <20101018152453.897C7282B9D@codespeak.net>
Author: arigo
Date: Mon Oct 18 17:24:50 2010
New Revision: 78049
Added:
pypy/trunk/pypy/jit/metainterp/greenfield.py
- copied unchanged from r78048, pypy/branch/rsre-jit/pypy/jit/metainterp/greenfield.py
pypy/trunk/pypy/jit/metainterp/test/test_greenfield.py
- copied unchanged from r78048, pypy/branch/rsre-jit/pypy/jit/metainterp/test/test_greenfield.py
pypy/trunk/pypy/rlib/rsre/rsre_jit.py
- copied unchanged from r78048, pypy/branch/rsre-jit/pypy/rlib/rsre/rsre_jit.py
pypy/trunk/pypy/rlib/rsre/test/conftest.py
- copied unchanged from r78048, pypy/branch/rsre-jit/pypy/rlib/rsre/test/conftest.py
pypy/trunk/pypy/rlib/rsre/test/test_zjit.py
- copied unchanged from r78048, pypy/branch/rsre-jit/pypy/rlib/rsre/test/test_zjit.py
Modified:
pypy/trunk/dotviewer/drawgraph.py
pypy/trunk/pypy/jit/backend/x86/regalloc.py
pypy/trunk/pypy/jit/codewriter/call.py
pypy/trunk/pypy/jit/codewriter/codewriter.py
pypy/trunk/pypy/jit/codewriter/jtransform.py
pypy/trunk/pypy/jit/codewriter/support.py
pypy/trunk/pypy/jit/codewriter/test/test_codewriter.py
pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py
pypy/trunk/pypy/jit/metainterp/blackhole.py
pypy/trunk/pypy/jit/metainterp/compile.py
pypy/trunk/pypy/jit/metainterp/executor.py
pypy/trunk/pypy/jit/metainterp/graphpage.py
pypy/trunk/pypy/jit/metainterp/jitdriver.py
pypy/trunk/pypy/jit/metainterp/optimizeopt/string.py
pypy/trunk/pypy/jit/metainterp/optimizeopt/virtualize.py
pypy/trunk/pypy/jit/metainterp/pyjitpl.py
pypy/trunk/pypy/jit/metainterp/resoperation.py
pypy/trunk/pypy/jit/metainterp/resume.py
pypy/trunk/pypy/jit/metainterp/test/test_basic.py
pypy/trunk/pypy/jit/metainterp/test/test_virtualref.py
pypy/trunk/pypy/jit/metainterp/test/test_warmspot.py
pypy/trunk/pypy/jit/metainterp/warmspot.py
pypy/trunk/pypy/rlib/jit.py
pypy/trunk/pypy/rlib/rsre/rsre_char.py
pypy/trunk/pypy/rlib/rsre/rsre_core.py
pypy/trunk/pypy/rlib/test/test_jit.py
Log:
Merge branch/rsre-jit: put the JIT hints in the rsre module.
Requires small extensions in the JIT:
- "green fields": the ability to write 'foo.bar' in the list of greens
in the JitDriver, where 'foo' is written in the list of reds.
- the ability to only write jitdriver.jit_merge_point(), without any
jitdriver.can_enter_jit(). In that case the jit_merge_point() plays both
roles. The difference with putting explicitly a can_enter_jit() just before
is that such a can_enter_jit() is not seen unless we are closing a loop; in
particular, it does not work if we have no loop at all.
Modified: pypy/trunk/dotviewer/drawgraph.py
==============================================================================
--- pypy/trunk/dotviewer/drawgraph.py (original)
+++ pypy/trunk/dotviewer/drawgraph.py Mon Oct 18 17:24:50 2010
@@ -423,20 +423,43 @@
else:
for line in lines:
raw_line = line.replace('\\l','').replace('\r','') or ' '
- img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
- w, h = img.get_size()
- if w>wmax: wmax = w
- if raw_line.strip():
- if line.endswith('\\l'):
- def cmd(img=img, y=hmax):
- img.draw(xleft, ytop+y)
- elif line.endswith('\r'):
- def cmd(img=img, y=hmax, w=w):
- img.draw(xright-w, ytop+y)
- else:
- def cmd(img=img, y=hmax, w=w):
- img.draw(xcenter-w//2, ytop+y)
+ if '\f' in raw_line: # grayed out parts of the line
+ imgs = []
+ graytext = True
+ h = 16
+ w_total = 0
+ for linepart in raw_line.split('\f'):
+ graytext = not graytext
+ if not linepart.strip():
+ continue
+ if graytext:
+ fgcolor = (128, 160, 160)
+ else:
+ fgcolor = (0, 0, 0)
+ img = TextSnippet(self, linepart, fgcolor, bgcolor)
+ imgs.append((w_total, img))
+ w, h = img.get_size()
+ w_total += w
+ if w_total > wmax: wmax = w_total
+ def cmd(imgs=imgs, y=hmax):
+ for x, img in imgs:
+ img.draw(xleft+x, ytop+y)
commands.append(cmd)
+ else:
+ img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
+ w, h = img.get_size()
+ if w>wmax: wmax = w
+ if raw_line.strip():
+ if line.endswith('\\l'):
+ def cmd(img=img, y=hmax):
+ img.draw(xleft, ytop+y)
+ elif line.endswith('\r'):
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xright-w, ytop+y)
+ else:
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xcenter-w//2, ytop+y)
+ commands.append(cmd)
hmax += h
#hmax += 8
Modified: pypy/trunk/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/regalloc.py Mon Oct 18 17:24:50 2010
@@ -1068,6 +1068,9 @@
def consider_debug_merge_point(self, op):
pass
+ def consider_jit_debug(self, op):
+ pass
+
def get_mark_gc_roots(self, gcrootmap):
shape = gcrootmap.get_basic_shape(IS_X86_64)
for v, val in self.fm.frame_bindings.items():
Modified: pypy/trunk/pypy/jit/codewriter/call.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/call.py (original)
+++ pypy/trunk/pypy/jit/codewriter/call.py Mon Oct 18 17:24:50 2010
@@ -277,3 +277,11 @@
return seen.pop()
else:
return None
+
+ def could_be_green_field(self, GTYPE, fieldname):
+ GTYPE_fieldname = (GTYPE, fieldname)
+ for jd in self.jitdrivers_sd:
+ if jd.greenfield_info is not None:
+ if GTYPE_fieldname in jd.greenfield_info.green_fields:
+ return True
+ return False
Modified: pypy/trunk/pypy/jit/codewriter/codewriter.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/codewriter.py (original)
+++ pypy/trunk/pypy/jit/codewriter/codewriter.py Mon Oct 18 17:24:50 2010
@@ -95,18 +95,18 @@
print '%s:' % (ssarepr.name,)
print format_assembler(ssarepr)
else:
- dir = udir.ensure("jitcodes", dir=1)
- if portal_jitdriver:
- name = "%02d_portal_runner" % (portal_jitdriver.index,)
- elif ssarepr.name and ssarepr.name != '?':
- name = ssarepr.name
- else:
- name = 'unnamed' % id(ssarepr)
- i = 1
- extra = ''
- while name+extra in self._seen_files:
- i += 1
- extra = '.%d' % i
- self._seen_files.add(name+extra)
- dir.join(name+extra).write(format_assembler(ssarepr))
log.dot()
+ dir = udir.ensure("jitcodes", dir=1)
+ if portal_jitdriver:
+ name = "%02d_portal_runner" % (portal_jitdriver.index,)
+ elif ssarepr.name and ssarepr.name != '?':
+ name = ssarepr.name
+ else:
+ name = 'unnamed' % id(ssarepr)
+ i = 1
+ extra = ''
+ while name+extra in self._seen_files:
+ i += 1
+ extra = '.%d' % i
+ self._seen_files.add(name+extra)
+ dir.join(name+extra).write(format_assembler(ssarepr))
Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/trunk/pypy/jit/codewriter/jtransform.py Mon Oct 18 17:24:50 2010
@@ -320,6 +320,8 @@
prepare = self._handle_str2unicode_call
elif oopspec_name.startswith('virtual_ref'):
prepare = self._handle_virtual_ref_call
+ elif oopspec_name.startswith('jit.'):
+ prepare = self._handle_jit_call
elif oopspec_name.startswith('libffi_'):
prepare = self._handle_libffi_call
else:
@@ -523,7 +525,12 @@
# check for deepfrozen structures that force constant-folding
immut = v_inst.concretetype.TO._immutable_field(c_fieldname.value)
if immut:
- pure = '_pure'
+ if (self.callcontrol is not None and
+ self.callcontrol.could_be_green_field(v_inst.concretetype.TO,
+ c_fieldname.value)):
+ pure = '_greenfield'
+ else:
+ pure = '_pure'
if immut == "[*]":
self.immutable_arrays[op.result] = True
else:
@@ -856,6 +863,15 @@
(self.graph,))
return []
+ def _handle_jit_call(self, op, oopspec_name, args):
+ if oopspec_name == 'jit.debug':
+ return SpaceOperation('jit_debug', args, None)
+ elif oopspec_name == 'jit.assert_green':
+ kind = getkind(args[0].concretetype)
+ return SpaceOperation('%s_assert_green' % kind, args, None)
+ else:
+ raise AssertionError("missing support for %r" % oopspec_name)
+
# ----------
# Lists.
Modified: pypy/trunk/pypy/jit/codewriter/support.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/support.py (original)
+++ pypy/trunk/pypy/jit/codewriter/support.py Mon Oct 18 17:24:50 2010
@@ -61,7 +61,7 @@
return rtyper.annotator.translator.graphs[0]
def split_before_jit_merge_point(graph, portalblock, portalopindex):
- """Find the block with 'jit_merge_point' and split just before,
+ """Split the block just before the 'jit_merge_point',
making sure the input args are in the canonical order.
"""
# split the block just before the jit_merge_point()
Modified: pypy/trunk/pypy/jit/codewriter/test/test_codewriter.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/test/test_codewriter.py (original)
+++ pypy/trunk/pypy/jit/codewriter/test/test_codewriter.py Mon Oct 18 17:24:50 2010
@@ -45,6 +45,7 @@
self.portal_graph = portal_graph
self.portal_runner_ptr = "???"
self.virtualizable_info = None
+ self.greenfield_info = None
def test_loop():
Modified: pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py (original)
+++ pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py Mon Oct 18 17:24:50 2010
@@ -687,6 +687,79 @@
assert block.operations[1].result is None
assert block.exits[0].args == [v1]
+def test_jit_merge_point_1():
+ class FakeJitDriverSD:
+ index = 42
+ class jitdriver:
+ greens = ['green1', 'green2', 'voidgreen3']
+ reds = ['red1', 'red2', 'voidred3']
+ jd = FakeJitDriverSD()
+ v1 = varoftype(lltype.Signed)
+ v2 = varoftype(lltype.Signed)
+ vvoid1 = varoftype(lltype.Void)
+ v3 = varoftype(lltype.Signed)
+ v4 = varoftype(lltype.Signed)
+ vvoid2 = varoftype(lltype.Void)
+ v5 = varoftype(lltype.Void)
+ op = SpaceOperation('jit_marker',
+ [Constant('jit_merge_point', lltype.Void),
+ Constant(jd.jitdriver, lltype.Void),
+ v1, v2, vvoid1, v3, v4, vvoid2], v5)
+ tr = Transformer()
+ tr.portal_jd = jd
+ oplist = tr.rewrite_operation(op)
+ assert len(oplist) == 6
+ assert oplist[0].opname == '-live-'
+ assert oplist[1].opname == 'int_guard_value'
+ assert oplist[1].args == [v1]
+ assert oplist[2].opname == '-live-'
+ assert oplist[3].opname == 'int_guard_value'
+ assert oplist[3].args == [v2]
+ assert oplist[4].opname == 'jit_merge_point'
+ assert oplist[4].args[0].value == 42
+ assert list(oplist[4].args[1]) == [v1, v2]
+ assert list(oplist[4].args[4]) == [v3, v4]
+ assert oplist[5].opname == '-live-'
+
+def test_getfield_gc():
+ S = lltype.GcStruct('S', ('x', lltype.Char))
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
+def test_getfield_gc_pure():
+ S = lltype.GcStruct('S', ('x', lltype.Char),
+ hints={'immutable': True})
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i_pure'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
+def test_getfield_gc_greenfield():
+ class FakeCC:
+ def get_vinfo(self, v):
+ return None
+ def could_be_green_field(self, S1, name1):
+ assert S1 is S
+ assert name1 == 'x'
+ return True
+ S = lltype.GcStruct('S', ('x', lltype.Char),
+ hints={'immutable': True})
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU(), FakeCC()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i_greenfield'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
def test_int_abs():
v1 = varoftype(lltype.Signed)
v2 = varoftype(lltype.Signed)
Modified: pypy/trunk/pypy/jit/metainterp/blackhole.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/blackhole.py (original)
+++ pypy/trunk/pypy/jit/metainterp/blackhole.py Mon Oct 18 17:24:50 2010
@@ -760,6 +760,20 @@
def bhimpl_debug_fatalerror(msg):
llop.debug_fatalerror(lltype.Void, msg)
+ @arguments("r", "i", "i", "i", "i")
+ def bhimpl_jit_debug(string, arg1=0, arg2=0, arg3=0, arg4=0):
+ pass
+
+ @arguments("i")
+ def bhimpl_int_assert_green(x):
+ pass
+ @arguments("r")
+ def bhimpl_ref_assert_green(x):
+ pass
+ @arguments("f")
+ def bhimpl_float_assert_green(x):
+ pass
+
# ----------
# the main hints and recursive calls
@@ -1073,6 +1087,10 @@
bhimpl_getfield_vable_r = bhimpl_getfield_gc_r
bhimpl_getfield_vable_f = bhimpl_getfield_gc_f
+ bhimpl_getfield_gc_i_greenfield = bhimpl_getfield_gc_i
+ bhimpl_getfield_gc_r_greenfield = bhimpl_getfield_gc_r
+ bhimpl_getfield_gc_f_greenfield = bhimpl_getfield_gc_f
+
@arguments("cpu", "i", "d", returns="i")
def bhimpl_getfield_raw_i(cpu, struct, fielddescr):
return cpu.bh_getfield_raw_i(struct, fielddescr)
Modified: pypy/trunk/pypy/jit/metainterp/compile.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/compile.py (original)
+++ pypy/trunk/pypy/jit/metainterp/compile.py Mon Oct 18 17:24:50 2010
@@ -370,7 +370,8 @@
from pypy.jit.metainterp.resume import force_from_resumedata
metainterp_sd = self.metainterp_sd
vinfo = self.jitdriver_sd.virtualizable_info
- all_virtuals = force_from_resumedata(metainterp_sd, self, vinfo)
+ ginfo = self.jitdriver_sd.greenfield_info
+ all_virtuals = force_from_resumedata(metainterp_sd, self, vinfo, ginfo)
# The virtualizable data was stored on the real virtualizable above.
# Handle all_virtuals: keep them for later blackholing from the
# future failure of the GUARD_NOT_FORCED
Modified: pypy/trunk/pypy/jit/metainterp/executor.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/executor.py (original)
+++ pypy/trunk/pypy/jit/metainterp/executor.py Mon Oct 18 17:24:50 2010
@@ -307,6 +307,7 @@
rop.CALL_ASSEMBLER,
rop.COND_CALL_GC_WB,
rop.DEBUG_MERGE_POINT,
+ rop.JIT_DEBUG,
rop.SETARRAYITEM_RAW,
): # list of opcodes never executed by pyjitpl
continue
Modified: pypy/trunk/pypy/jit/metainterp/graphpage.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/graphpage.py (original)
+++ pypy/trunk/pypy/jit/metainterp/graphpage.py Mon Oct 18 17:24:50 2010
@@ -153,7 +153,7 @@
opindex = opstartindex
while True:
op = operations[opindex]
- lines.append(repr(op))
+ lines.append(op.repr(graytext=True))
if is_interesting_guard(op):
tgt = op.getdescr()._debug_suboperations[0]
tgt_g, tgt_i = self.all_operations[tgt]
Modified: pypy/trunk/pypy/jit/metainterp/jitdriver.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/jitdriver.py (original)
+++ pypy/trunk/pypy/jit/metainterp/jitdriver.py Mon Oct 18 17:24:50 2010
@@ -13,8 +13,10 @@
# self.num_red_args ... pypy.jit.metainterp.warmspot
# self.result_type ... pypy.jit.metainterp.warmspot
# self.virtualizable_info... pypy.jit.metainterp.warmspot
+ # self.greenfield_info ... pypy.jit.metainterp.warmspot
# self.warmstate ... pypy.jit.metainterp.warmspot
# self.handle_jitexc_from_bh pypy.jit.metainterp.warmspot
+ # self.no_loop_header ... pypy.jit.metainterp.warmspot
# self.portal_finishtoken... pypy.jit.metainterp.pyjitpl
# self.index ... pypy.jit.codewriter.call
# self.mainjitcode ... pypy.jit.codewriter.call
Modified: pypy/trunk/pypy/jit/metainterp/optimizeopt/string.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/optimizeopt/string.py (original)
+++ pypy/trunk/pypy/jit/metainterp/optimizeopt/string.py Mon Oct 18 17:24:50 2010
@@ -12,7 +12,7 @@
from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec
from pypy.jit.codewriter import heaptracker
from pypy.rlib.unroll import unrolling_iterable
-from pypy.rlib.objectmodel import specialize
+from pypy.rlib.objectmodel import specialize, we_are_translated
class StrOrUnicode(object):
@@ -107,7 +107,10 @@
self.box = box = self.source_op.result
newoperations = self.optimizer.newoperations
lengthbox = self.getstrlen(newoperations, self.mode)
- newoperations.append(ResOperation(self.mode.NEWSTR, [lengthbox], box))
+ op = ResOperation(self.mode.NEWSTR, [lengthbox], box)
+ if not we_are_translated():
+ op.name = 'FORCE'
+ newoperations.append(op)
self.string_copy_parts(newoperations, box, CONST_0, self.mode)
Modified: pypy/trunk/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/trunk/pypy/jit/metainterp/optimizeopt/virtualize.py Mon Oct 18 17:24:50 2010
@@ -74,6 +74,8 @@
assert self.source_op is not None
# ^^^ This case should not occur any more (see test_bug_3).
#
+ if not we_are_translated():
+ self.source_op.name = 'FORCE ' + self.source_op.name
newoperations = self.optimizer.newoperations
newoperations.append(self.source_op)
self.box = box = self.source_op.result
@@ -170,6 +172,8 @@
def _really_force(self):
assert self.source_op is not None
+ if not we_are_translated():
+ self.source_op.name = 'FORCE ' + self.source_op.name
newoperations = self.optimizer.newoperations
newoperations.append(self.source_op)
self.box = box = self.source_op.result
Modified: pypy/trunk/pypy/jit/metainterp/pyjitpl.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/pyjitpl.py (original)
+++ pypy/trunk/pypy/jit/metainterp/pyjitpl.py Mon Oct 18 17:24:50 2010
@@ -1,4 +1,4 @@
-import py, os
+import py, os, sys
from pypy.rpython.lltypesystem import lltype, llmemory, rclass
from pypy.rlib.objectmodel import we_are_translated
from pypy.rlib.unroll import unrolling_iterable
@@ -498,6 +498,22 @@
opimpl_getfield_gc_r_pure = _opimpl_getfield_gc_pure_any
opimpl_getfield_gc_f_pure = _opimpl_getfield_gc_pure_any
+ @arguments("orgpc", "box", "descr")
+ def _opimpl_getfield_gc_greenfield_any(self, pc, box, fielddescr):
+ ginfo = self.metainterp.jitdriver_sd.greenfield_info
+ if (ginfo is not None and fielddescr in ginfo.green_field_descrs
+ and not self._nonstandard_virtualizable(pc, box)):
+ # fetch the result, but consider it as a Const box and don't
+ # record any operation
+ resbox = executor.execute(self.metainterp.cpu, self.metainterp,
+ rop.GETFIELD_GC_PURE, fielddescr, box)
+ return resbox.constbox()
+ # fall-back
+ return self.execute_with_descr(rop.GETFIELD_GC_PURE, fielddescr, box)
+ opimpl_getfield_gc_i_greenfield = _opimpl_getfield_gc_greenfield_any
+ opimpl_getfield_gc_r_greenfield = _opimpl_getfield_gc_greenfield_any
+ opimpl_getfield_gc_f_greenfield = _opimpl_getfield_gc_greenfield_any
+
@arguments("box", "descr", "box")
def _opimpl_setfield_gc_any(self, box, fielddescr, valuebox):
self.execute_with_descr(rop.SETFIELD_GC, fielddescr, box, valuebox)
@@ -529,7 +545,8 @@
def _nonstandard_virtualizable(self, pc, box):
# returns True if 'box' is actually not the "standard" virtualizable
# that is stored in metainterp.virtualizable_boxes[-1]
- if self.metainterp.jitdriver_sd.virtualizable_info is None:
+ if (self.metainterp.jitdriver_sd.virtualizable_info is None and
+ self.metainterp.jitdriver_sd.greenfield_info is None):
return True # can occur in case of multiple JITs
standard_box = self.metainterp.virtualizable_boxes[-1]
if standard_box is box:
@@ -799,12 +816,16 @@
@arguments("orgpc", "int", "boxes3", "boxes3")
def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes, redboxes):
+ any_operation = len(self.metainterp.history.operations) > 0
jitdriver_sd = self.metainterp.staticdata.jitdrivers_sd[jdindex]
self.verify_green_args(jitdriver_sd, greenboxes)
# xxx we may disable the following line in some context later
self.debug_merge_point(jitdriver_sd, greenboxes)
if self.metainterp.seen_loop_header_for_jdindex < 0:
- return
+ if not jitdriver_sd.no_loop_header or not any_operation:
+ return
+ # automatically add a loop_header if there is none
+ self.metainterp.seen_loop_header_for_jdindex = jdindex
#
assert self.metainterp.seen_loop_header_for_jdindex == jdindex, (
"found a loop_header for a JitDriver that does not match "
@@ -893,6 +914,40 @@
msg = box.getref(lltype.Ptr(rstr.STR))
lloperation.llop.debug_fatalerror(msg)
+ @arguments("box", "box", "box", "box", "box")
+ def opimpl_jit_debug(self, stringbox, arg1box, arg2box, arg3box, arg4box):
+ from pypy.rpython.lltypesystem import rstr
+ from pypy.rpython.annlowlevel import hlstr
+ msg = stringbox.getref(lltype.Ptr(rstr.STR))
+ debug_print('jit_debug:', hlstr(msg),
+ arg1box.getint(), arg2box.getint(),
+ arg3box.getint(), arg4box.getint())
+ args = [stringbox, arg1box, arg2box, arg3box, arg4box]
+ i = 4
+ while i > 0 and args[i].getint() == -sys.maxint-1:
+ i -= 1
+ assert i >= 0
+ op = self.metainterp.history.record(rop.JIT_DEBUG, args[:i+1], None)
+ self.metainterp.attach_debug_info(op)
+
+ @arguments("box")
+ def _opimpl_assert_green(self, box):
+ if not isinstance(box, Const):
+ msg = "assert_green failed at %s:%d" % (
+ self.jitcode.name,
+ self.pc)
+ if we_are_translated():
+ from pypy.rpython.annlowlevel import llstr
+ from pypy.rpython.lltypesystem import lloperation
+ lloperation.llop.debug_fatalerror(lltype.Void, llstr(msg))
+ else:
+ from pypy.rlib.jit import AssertGreenFailed
+ raise AssertGreenFailed(msg)
+
+ opimpl_int_assert_green = _opimpl_assert_green
+ opimpl_ref_assert_green = _opimpl_assert_green
+ opimpl_float_assert_green = _opimpl_assert_green
+
@arguments("box")
def opimpl_virtual_ref(self, box):
# Details on the content of metainterp.virtualref_boxes:
@@ -998,7 +1053,8 @@
guard_op = metainterp.history.record(opnum, moreargs, None,
descr=resumedescr)
virtualizable_boxes = None
- if metainterp.jitdriver_sd.virtualizable_info is not None:
+ if (metainterp.jitdriver_sd.virtualizable_info is not None or
+ metainterp.jitdriver_sd.greenfield_info is not None):
virtualizable_boxes = metainterp.virtualizable_boxes
saved_pc = self.pc
if resumepc >= 0:
@@ -1646,6 +1702,7 @@
duplicates)
live_arg_boxes += self.virtualizable_boxes
live_arg_boxes.pop()
+ #
assert len(self.virtualref_boxes) == 0, "missing virtual_ref_finish()?"
# Called whenever we reach the 'loop_header' hint.
# First, attempt to make a bridge:
@@ -1832,6 +1889,7 @@
f.setup_call(original_boxes)
assert self.in_recursion == 0
self.virtualref_boxes = []
+ self.initialize_withgreenfields(original_boxes)
self.initialize_virtualizable(original_boxes)
def initialize_state_from_guard_failure(self, resumedescr):
@@ -1856,6 +1914,14 @@
self.virtualizable_boxes.append(virtualizable_box)
self.initialize_virtualizable_enter()
+ def initialize_withgreenfields(self, original_boxes):
+ ginfo = self.jitdriver_sd.greenfield_info
+ if ginfo is not None:
+ assert self.jitdriver_sd.virtualizable_info is None
+ index = (self.jitdriver_sd.num_green_args +
+ ginfo.red_index)
+ self.virtualizable_boxes = [original_boxes[index]]
+
def initialize_virtualizable_enter(self):
vinfo = self.jitdriver_sd.virtualizable_info
virtualizable_box = self.virtualizable_boxes[-1]
@@ -1949,8 +2015,10 @@
def rebuild_state_after_failure(self, resumedescr):
vinfo = self.jitdriver_sd.virtualizable_info
+ ginfo = self.jitdriver_sd.greenfield_info
self.framestack = []
- boxlists = resume.rebuild_from_resumedata(self, resumedescr, vinfo)
+ boxlists = resume.rebuild_from_resumedata(self, resumedescr, vinfo,
+ ginfo)
inputargs_and_holes, virtualizable_boxes, virtualref_boxes = boxlists
#
# virtual refs: make the vrefs point to the freshly allocated virtuals
@@ -1975,6 +2043,12 @@
assert not virtualizable.vable_token
# fill the virtualizable with the local boxes
self.synchronize_virtualizable()
+ #
+ elif self.jitdriver_sd.greenfield_info:
+ self.virtualizable_boxes = virtualizable_boxes
+ else:
+ assert not virtualizable_boxes
+ #
return inputargs_and_holes
def check_synchronized_virtualizable(self):
@@ -2048,7 +2122,8 @@
for i in range(len(boxes)):
if boxes[i] is oldbox:
boxes[i] = newbox
- if self.jitdriver_sd.virtualizable_info is not None:
+ if (self.jitdriver_sd.virtualizable_info is not None or
+ self.jitdriver_sd.greenfield_info is not None):
boxes = self.virtualizable_boxes
for i in range(len(boxes)):
if boxes[i] is oldbox:
Modified: pypy/trunk/pypy/jit/metainterp/resoperation.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/resoperation.py (original)
+++ pypy/trunk/pypy/jit/metainterp/resoperation.py Mon Oct 18 17:24:50 2010
@@ -93,7 +93,7 @@
def __repr__(self):
return self.repr()
- def repr(self):
+ def repr(self, graytext=False):
# RPython-friendly version
if self.result is not None:
sres = '%s = ' % (self.result,)
@@ -101,6 +101,8 @@
sres = ''
if self.name:
prefix = "%s:%s " % (self.name, self.pc)
+ if graytext:
+ prefix = "\f%s\f" % prefix
else:
prefix = ""
args = self.getarglist()
@@ -457,6 +459,7 @@
#'RUNTIMENEW/1', # ootype operation
'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier)
'DEBUG_MERGE_POINT/1', # debugging only
+ 'JIT_DEBUG/*', # debugging only
'VIRTUAL_REF_FINISH/2', # removed before it's passed to the backend
'COPYSTRCONTENT/5', # src, dst, srcstart, dststart, length
'COPYUNICODECONTENT/5',
Modified: pypy/trunk/pypy/jit/metainterp/resume.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/resume.py (original)
+++ pypy/trunk/pypy/jit/metainterp/resume.py Mon Oct 18 17:24:50 2010
@@ -689,9 +689,11 @@
# ---------- when resuming for pyjitpl.py, make boxes ----------
-def rebuild_from_resumedata(metainterp, storage, virtualizable_info):
+def rebuild_from_resumedata(metainterp, storage, virtualizable_info,
+ greenfield_info):
resumereader = ResumeDataBoxReader(storage, metainterp)
- boxes = resumereader.consume_vref_and_vable_boxes(virtualizable_info)
+ boxes = resumereader.consume_vref_and_vable_boxes(virtualizable_info,
+ greenfield_info)
virtualizable_boxes, virtualref_boxes = boxes
frameinfo = storage.rd_frame_info_list
while True:
@@ -736,15 +738,18 @@
assert (end & 1) == 0
return [self.decode_ref(nums[i]) for i in range(end)]
- def consume_vref_and_vable_boxes(self, vinfo):
+ def consume_vref_and_vable_boxes(self, vinfo, ginfo):
nums = self.cur_numb.nums
self.cur_numb = self.cur_numb.prev
- if vinfo is None:
- virtualizable_boxes = None
- end = len(nums)
- else:
+ if vinfo is not None:
virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, nums)
end = len(nums) - len(virtualizable_boxes)
+ elif ginfo is not None:
+ virtualizable_boxes = [self.decode_ref(nums[-1])]
+ end = len(nums) - 1
+ else:
+ virtualizable_boxes = None
+ end = len(nums)
virtualref_boxes = self.consume_virtualref_boxes(nums, end)
return virtualizable_boxes, virtualref_boxes
@@ -901,8 +906,9 @@
resumereader = ResumeDataDirectReader(blackholeinterpbuilder.cpu, storage,
all_virtuals)
vinfo = jitdriver_sd.virtualizable_info
+ ginfo = jitdriver_sd.greenfield_info
vrefinfo = blackholeinterpbuilder.metainterp_sd.virtualref_info
- resumereader.consume_vref_and_vable(vrefinfo, vinfo)
+ resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo)
#
# First get a chain of blackhole interpreters whose length is given
# by the depth of rd_frame_info_list. The first one we get must be
@@ -932,11 +938,11 @@
resumereader.done()
return firstbh
-def force_from_resumedata(metainterp_sd, storage, vinfo=None):
+def force_from_resumedata(metainterp_sd, storage, vinfo, ginfo):
resumereader = ResumeDataDirectReader(metainterp_sd.cpu, storage)
resumereader.handling_async_forcing()
vrefinfo = metainterp_sd.virtualref_info
- resumereader.consume_vref_and_vable(vrefinfo, vinfo)
+ resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo)
return resumereader.force_all_virtuals()
class ResumeDataDirectReader(AbstractResumeDataReader):
@@ -1011,11 +1017,12 @@
return specialize_value(TYPE, x)
load_value_of_type._annspecialcase_ = 'specialize:arg(1)'
- def consume_vref_and_vable(self, vrefinfo, vinfo):
+ def consume_vref_and_vable(self, vrefinfo, vinfo, ginfo):
nums = self.cur_numb.nums
self.cur_numb = self.cur_numb.prev
if self.resume_after_guard_not_forced != 2:
end_vref = self.consume_vable_info(vinfo, nums)
+ if ginfo is not None: end_vref -= 1
self.consume_virtualref_info(vrefinfo, nums, end_vref)
def allocate_with_vtable(self, known_class):
Modified: pypy/trunk/pypy/jit/metainterp/test/test_basic.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/test_basic.py (original)
+++ pypy/trunk/pypy/jit/metainterp/test/test_basic.py Mon Oct 18 17:24:50 2010
@@ -2,6 +2,7 @@
import sys
from pypy.rlib.jit import JitDriver, we_are_jitted, hint, dont_look_inside
from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_SIMPLE, loop_invariant
+from pypy.rlib.jit import jit_debug, assert_green, AssertGreenFailed
from pypy.jit.metainterp.warmspot import ll_meta_interp, get_stats
from pypy.jit.backend.llgraph import runner
from pypy.jit.metainterp import pyjitpl, history
@@ -44,6 +45,7 @@
num_green_args = 0
portal_graph = graphs[0]
virtualizable_info = None
+ greenfield_info = None
result_type = result_kind
portal_runner_ptr = "???"
@@ -1644,6 +1646,33 @@
res = self.interp_operations(f, [10, 3.5])
assert res == 3.5
+ def test_jit_debug(self):
+ myjitdriver = JitDriver(greens = [], reds = ['x'])
+ class A:
+ pass
+ def f(x):
+ while x > 0:
+ myjitdriver.can_enter_jit(x=x)
+ myjitdriver.jit_merge_point(x=x)
+ jit_debug("hi there:", x)
+ jit_debug("foobar")
+ x -= 1
+ return x
+ res = self.meta_interp(f, [8])
+ assert res == 0
+ self.check_loops(jit_debug=2)
+
+ def test_assert_green(self):
+ def f(x, promote):
+ if promote:
+ x = hint(x, promote=True)
+ assert_green(x)
+ return x
+ res = self.interp_operations(f, [8, 1])
+ assert res == 8
+ py.test.raises(AssertGreenFailed, self.interp_operations, f, [8, 0])
+
+
class TestOOtype(BasicTests, OOJitMixin):
def test_oohash(self):
Modified: pypy/trunk/pypy/jit/metainterp/test/test_virtualref.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/test_virtualref.py (original)
+++ pypy/trunk/pypy/jit/metainterp/test/test_virtualref.py Mon Oct 18 17:24:50 2010
@@ -93,7 +93,7 @@
lst = []
vrefinfo.continue_tracing = lambda vref, virtual: \
lst.append((vref, virtual))
- resumereader.consume_vref_and_vable(vrefinfo, None)
+ resumereader.consume_vref_and_vable(vrefinfo, None, None)
del vrefinfo.continue_tracing
assert len(lst) == 1
lltype.cast_opaque_ptr(lltype.Ptr(JIT_VIRTUAL_REF),
Modified: pypy/trunk/pypy/jit/metainterp/test/test_warmspot.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/test_warmspot.py (original)
+++ pypy/trunk/pypy/jit/metainterp/test/test_warmspot.py Mon Oct 18 17:24:50 2010
@@ -296,6 +296,69 @@
assert res == 1
self.check_loops(int_add=1) # I get 13 without the loop_header()
+ def test_omit_can_enter_jit(self):
+ # Simple test comparing the effects of always giving a can_enter_jit(),
+ # or not giving any. Mostly equivalent, except that if given, it is
+ # ignored the first time, and so it ends up taking one extra loop to
+ # start JITting.
+ mydriver = JitDriver(greens=[], reds=['m'])
+ #
+ for i2 in range(10):
+ def f2(m):
+ while m > 0:
+ mydriver.jit_merge_point(m=m)
+ m -= 1
+ self.meta_interp(f2, [i2])
+ try:
+ self.check_tree_loop_count(1)
+ break
+ except AssertionError:
+ print "f2: no loop generated for i2==%d" % i2
+ else:
+ raise # re-raise the AssertionError: check_loop_count never 1
+ #
+ for i1 in range(10):
+ def f1(m):
+ while m > 0:
+ mydriver.can_enter_jit(m=m)
+ mydriver.jit_merge_point(m=m)
+ m -= 1
+ self.meta_interp(f1, [i1])
+ try:
+ self.check_tree_loop_count(1)
+ break
+ except AssertionError:
+ print "f1: no loop generated for i1==%d" % i1
+ else:
+ raise # re-raise the AssertionError: check_loop_count never 1
+ #
+ assert i1 - 1 == i2
+
+ def test_no_loop_at_all(self):
+ mydriver = JitDriver(greens=[], reds=['m'])
+ def f2(m):
+ mydriver.jit_merge_point(m=m)
+ return m - 1
+ def f1(m):
+ while m > 0:
+ m = f2(m)
+ self.meta_interp(f1, [8])
+ # it should generate one "loop" only, which ends in a FINISH
+ # corresponding to the return from f2.
+ self.check_tree_loop_count(1)
+ self.check_loop_count(0)
+
+ def test_simple_loop(self):
+ mydriver = JitDriver(greens=[], reds=['m'])
+ def f1(m):
+ while m > 0:
+ mydriver.jit_merge_point(m=m)
+ m = m - 1
+ self.meta_interp(f1, [8])
+ self.check_loop_count(1)
+ self.check_loops({'int_sub': 1, 'int_gt': 1, 'guard_true': 1,
+ 'jump': 1})
+
class TestLLWarmspot(WarmspotTests, LLJitMixin):
CPUClass = runner.LLtypeCPU
Modified: pypy/trunk/pypy/jit/metainterp/warmspot.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/warmspot.py (original)
+++ pypy/trunk/pypy/jit/metainterp/warmspot.py Mon Oct 18 17:24:50 2010
@@ -115,10 +115,10 @@
return results
def find_can_enter_jit(graphs):
- results = _find_jit_marker(graphs, 'can_enter_jit')
- if not results:
- raise Exception("no can_enter_jit found!")
- return results
+ return _find_jit_marker(graphs, 'can_enter_jit')
+
+def find_loop_headers(graphs):
+ return _find_jit_marker(graphs, 'loop_header')
def find_jit_merge_points(graphs):
results = _find_jit_marker(graphs, 'jit_merge_point')
@@ -211,9 +211,9 @@
"there are multiple jit_merge_points with the same jitdriver"
def split_graph_and_record_jitdriver(self, graph, block, pos):
- jd = JitDriverStaticData()
- jd._jit_merge_point_pos = (graph, block, pos)
op = block.operations[pos]
+ jd = JitDriverStaticData()
+ jd._jit_merge_point_pos = (graph, op)
args = op.args[2:]
s_binding = self.translator.annotator.binding
jd._portal_args_s = [s_binding(v) for v in args]
@@ -286,10 +286,20 @@
def make_virtualizable_infos(self):
vinfos = {}
for jd in self.jitdrivers_sd:
+ #
+ jd.greenfield_info = None
+ for name in jd.jitdriver.greens:
+ if '.' in name:
+ from pypy.jit.metainterp.greenfield import GreenFieldInfo
+ jd.greenfield_info = GreenFieldInfo(self.cpu, jd)
+ break
+ #
if not jd.jitdriver.virtualizables:
jd.virtualizable_info = None
jd.index_of_virtualizable = -1
continue
+ else:
+ assert jd.greenfield_info is None, "XXX not supported yet"
#
jitdriver = jd.jitdriver
assert len(jitdriver.virtualizables) == 1 # for now
@@ -457,8 +467,7 @@
self.make_args_specification(jd)
def make_args_specification(self, jd):
- graph, block, index = jd._jit_merge_point_pos
- op = block.operations[index]
+ graph, op = jd._jit_merge_point_pos
greens_v, reds_v = support.decode_hp_hint_args(op)
ALLARGS = [v.concretetype for v in (greens_v + reds_v)]
jd._green_args_spec = [v.concretetype for v in greens_v]
@@ -474,26 +483,37 @@
[lltype.Signed, llmemory.GCREF], RESTYPE)
def rewrite_can_enter_jits(self):
- can_enter_jits = find_can_enter_jit(self.translator.graphs)
sublists = {}
for jd in self.jitdrivers_sd:
- sublists[jd.jitdriver] = []
+ sublists[jd.jitdriver] = jd, []
+ jd.no_loop_header = True
+ #
+ loop_headers = find_loop_headers(self.translator.graphs)
+ for graph, block, index in loop_headers:
+ op = block.operations[index]
+ jitdriver = op.args[1].value
+ assert jitdriver in sublists, \
+ "loop_header with no matching jit_merge_point"
+ jd, sublist = sublists[jitdriver]
+ jd.no_loop_header = False
+ #
+ can_enter_jits = find_can_enter_jit(self.translator.graphs)
for graph, block, index in can_enter_jits:
op = block.operations[index]
jitdriver = op.args[1].value
assert jitdriver in sublists, \
"can_enter_jit with no matching jit_merge_point"
+ jd, sublist = sublists[jitdriver]
origportalgraph = jd._jit_merge_point_pos[0]
if graph is not origportalgraph:
- sublists[jitdriver].append((graph, block, index))
+ sublist.append((graph, block, index))
+ jd.no_loop_header = False
else:
pass # a 'can_enter_jit' before the 'jit-merge_point', but
# originally in the same function: we ignore it here
# see e.g. test_jitdriver.test_simple
for jd in self.jitdrivers_sd:
- sublist = sublists[jd.jitdriver]
- assert len(sublist) > 0, \
- "found no can_enter_jit for %r" % (jd.jitdriver,)
+ _, sublist = sublists[jd.jitdriver]
self.rewrite_can_enter_jit(jd, sublist)
def rewrite_can_enter_jit(self, jd, can_enter_jits):
@@ -501,6 +521,19 @@
FUNCPTR = jd._PTR_JIT_ENTER_FUNCTYPE
jit_enter_fnptr = self.helper_func(FUNCPTR, jd._maybe_enter_jit_fn)
+ if len(can_enter_jits) == 0:
+ # see test_warmspot.test_no_loop_at_all
+ operations = jd.portal_graph.startblock.operations
+ op1 = operations[0]
+ assert (op1.opname == 'jit_marker' and
+ op1.args[0].value == 'jit_merge_point')
+ op0 = SpaceOperation(
+ 'jit_marker',
+ [Constant('can_enter_jit', lltype.Void)] + op1.args[1:],
+ None)
+ operations.insert(0, op0)
+ can_enter_jits = [(jd.portal_graph, jd.portal_graph.startblock, 0)]
+
for graph, block, index in can_enter_jits:
if graph is jd._jit_merge_point_pos[0]:
continue
@@ -709,8 +742,14 @@
# ____________________________________________________________
# Now mutate origportalgraph to end with a call to portal_runner_ptr
#
- _, origblock, origindex = jd._jit_merge_point_pos
- op = origblock.operations[origindex]
+ _, op = jd._jit_merge_point_pos
+ for origblock in origportalgraph.iterblocks():
+ if op in origblock.operations:
+ break
+ else:
+ assert False, "lost the operation %r in the graph %r" % (
+ op, origportalgraph)
+ origindex = origblock.operations.index(op)
assert op.opname == 'jit_marker'
assert op.args[0].value == 'jit_merge_point'
greens_v, reds_v = support.decode_hp_hint_args(op)
Modified: pypy/trunk/pypy/rlib/jit.py
==============================================================================
--- pypy/trunk/pypy/rlib/jit.py (original)
+++ pypy/trunk/pypy/rlib/jit.py Mon Oct 18 17:24:50 2010
@@ -145,6 +145,24 @@
return hop.inputconst(lltype.Signed, _we_are_jitted)
+def jit_debug(string, arg1=-sys.maxint-1, arg2=-sys.maxint-1,
+ arg3=-sys.maxint-1, arg4=-sys.maxint-1):
+ """When JITted, cause an extra operation DEBUG_MERGE_POINT to appear in
+ the graphs. Should not be left after debugging."""
+ keepalive_until_here(string) # otherwise the whole function call is removed
+jit_debug.oopspec = 'jit.debug(string, arg1, arg2, arg3, arg4)'
+
+def assert_green(value):
+ """Very strong assert: checks that 'value' is a green
+ (a JIT compile-time constant)."""
+ keepalive_until_here(value)
+assert_green._annspecialcase_ = 'specialize:argtype(0)'
+assert_green.oopspec = 'jit.assert_green(value)'
+
+class AssertGreenFailed(Exception):
+ pass
+
+
##def force_virtualizable(virtualizable):
## pass
@@ -272,7 +290,8 @@
self.virtualizables = virtualizables
for v in self.virtualizables:
assert v in self.reds
- self._alllivevars = dict.fromkeys(self.greens + self.reds)
+ self._alllivevars = dict.fromkeys(
+ [name for name in self.greens + self.reds if '.' not in name])
self._make_extregistryentries()
self.get_jitcell_at = get_jitcell_at
self.set_jitcell_at = set_jitcell_at
@@ -364,7 +383,8 @@
driver = self.instance.im_self
keys = kwds_s.keys()
keys.sort()
- expected = ['s_' + name for name in driver.greens + driver.reds]
+ expected = ['s_' + name for name in driver.greens + driver.reds
+ if '.' not in name]
expected.sort()
if keys != expected:
raise JitHintError("%s expects the following keyword "
@@ -409,7 +429,13 @@
uniquekey = 'jitdriver.%s' % func.func_name
args_s = args_s[:]
for name in variables:
- s_arg = kwds_s['s_' + name]
+ if '.' not in name:
+ s_arg = kwds_s['s_' + name]
+ else:
+ objname, fieldname = name.split('.')
+ s_instance = kwds_s['s_' + objname]
+ s_arg = s_instance.classdef.about_attribute(fieldname)
+ assert s_arg is not None
args_s.append(s_arg)
bk.emulate_pbc_call(uniquekey, s_func, args_s)
@@ -422,9 +448,42 @@
greens_v = []
reds_v = []
for name in driver.greens:
- i = kwds_i['i_' + name]
- r_green = hop.args_r[i]
- v_green = hop.inputarg(r_green, arg=i)
+ if '.' not in name:
+ i = kwds_i['i_' + name]
+ r_green = hop.args_r[i]
+ v_green = hop.inputarg(r_green, arg=i)
+ else:
+ if hop.rtyper.type_system.name == 'ootypesystem':
+ py.test.skip("lltype only")
+ objname, fieldname = name.split('.') # see test_green_field
+ assert objname in driver.reds
+ i = kwds_i['i_' + objname]
+ s_red = hop.args_s[i]
+ r_red = hop.args_r[i]
+ while True:
+ try:
+ mangled_name, r_field = r_red._get_field(fieldname)
+ break
+ except KeyError:
+ pass
+ assert r_red.rbase is not None, (
+ "field %r not found in %r" % (name,
+ r_red.lowleveltype.TO))
+ r_red = r_red.rbase
+ GTYPE = r_red.lowleveltype.TO
+ assert GTYPE._immutable_field(mangled_name), (
+ "field %r must be declared as immutable" % name)
+ if not hasattr(driver, 'll_greenfields'):
+ driver.ll_greenfields = {}
+ driver.ll_greenfields[name] = GTYPE, mangled_name
+ #
+ v_red = hop.inputarg(r_red, arg=i)
+ c_llname = hop.inputconst(lltype.Void, mangled_name)
+ v_green = hop.genop('getfield', [v_red, c_llname],
+ resulttype = r_field)
+ s_green = s_red.classdef.about_attribute(fieldname)
+ assert s_green is not None
+ hop.rtyper.annotator.setbinding(v_green, s_green)
greens_v.append(v_green)
for name in driver.reds:
i = kwds_i['i_' + name]
Modified: pypy/trunk/pypy/rlib/rsre/rsre_char.py
==============================================================================
--- pypy/trunk/pypy/rlib/rsre/rsre_char.py (original)
+++ pypy/trunk/pypy/rlib/rsre/rsre_char.py Mon Oct 18 17:24:50 2010
@@ -4,6 +4,7 @@
import sys
from pypy.rlib.rlocale import tolower, isalnum
from pypy.rlib.unroll import unrolling_iterable
+from pypy.rlib import jit
# Note: the unicode parts of this module require you to call
# rsre_char.set_unicode_db() first, to select one of the modules
@@ -43,6 +44,7 @@
# XXX can we import those safely from sre_constants?
SRE_INFO_PREFIX = 1
SRE_INFO_LITERAL = 2
+SRE_INFO_CHARSET = 4
SRE_FLAG_LOCALE = 4 # honour system locale
SRE_FLAG_UNICODE = 32 # use unicode locale
OPCODE_INFO = 17
@@ -64,33 +66,27 @@
#### Category helpers
-ascii_char_info = [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 6, 2,
-2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 25, 25, 25, 25, 25, 25, 25, 25,
-25, 25, 0, 0, 0, 0, 0, 0, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0,
-0, 0, 16, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0, 0, 0, 0 ]
-
+is_a_word = [(chr(i).isalnum() or chr(i) == '_') for i in range(256)]
linebreak = ord("\n")
underline = ord("_")
def is_digit(code):
- return code < 128 and (ascii_char_info[code] & 1 != 0)
+ return code <= 57 and code >= 48
def is_uni_digit(code):
assert unicodedb is not None
return unicodedb.isdigit(code)
def is_space(code):
- return code < 128 and (ascii_char_info[code] & 2 != 0)
+ return code == 32 or (code <= 13 and code >= 9)
def is_uni_space(code):
assert unicodedb is not None
return unicodedb.isspace(code)
def is_word(code):
- return code < 128 and (ascii_char_info[code] & 16 != 0)
+ assert code >= 0
+ return code < 256 and is_a_word[code]
def is_uni_word(code):
assert unicodedb is not None
@@ -142,6 +138,7 @@
SET_OK = -1
SET_NOT_OK = -2
+ at jit.unroll_safe
def check_charset(pattern, ppos, char_code):
"""Checks whether a character matches set of arbitrary length.
The set starts at pattern[ppos]."""
Modified: pypy/trunk/pypy/rlib/rsre/rsre_core.py
==============================================================================
--- pypy/trunk/pypy/rlib/rsre/rsre_core.py (original)
+++ pypy/trunk/pypy/rlib/rsre/rsre_core.py Mon Oct 18 17:24:50 2010
@@ -4,6 +4,8 @@
from pypy.rlib.rsre import rsre_char
from pypy.tool.sourcetools import func_with_new_name
from pypy.rlib.objectmodel import we_are_translated
+from pypy.rlib import jit
+from pypy.rlib.rsre.rsre_jit import install_jitdriver, install_jitdriver_spec
OPCODE_FAILURE = 0
@@ -56,16 +58,19 @@
_seen_specname[specname] = True
# Install a copy of the function under the name '_spec_funcname' in each
# concrete subclass
+ specialized_methods = []
for prefix, concreteclass in [('str', StrMatchContext),
('uni', UnicodeMatchContext)]:
newfunc = func_with_new_name(func, prefix + specname)
assert not hasattr(concreteclass, specname)
setattr(concreteclass, specname, newfunc)
+ specialized_methods.append(newfunc)
# Return a dispatcher function, specialized on the exact type of 'ctx'
def dispatch(ctx, *args):
return getattr(ctx, specname)(*args)
dispatch._annspecialcase_ = 'specialize:argtype(0)'
- return dispatch
+ dispatch._specialized_methods_ = specialized_methods
+ return func_with_new_name(dispatch, specname)
# ____________________________________________________________
@@ -75,6 +80,7 @@
class AbstractMatchContext(object):
"""Abstract base class"""
+ _immutable_fields_ = ['pattern[*]', 'flags', 'end']
match_start = 0
match_end = 0
match_marks = None
@@ -238,8 +244,9 @@
self.start_ptr = ptr
self.start_marks = marks
+ @jit.unroll_safe
def find_first_result(self, ctx):
- ppos = self.ppos
+ ppos = jit.hint(self.ppos, promote=True)
while ctx.pat(ppos):
result = sre_match(ctx, ppos + 1, self.start_ptr, self.start_marks)
ppos += ctx.pat(ppos)
@@ -250,6 +257,10 @@
find_next_result = find_first_result
class RepeatOneMatchResult(MatchResult):
+ install_jitdriver('RepeatOne',
+ greens=['nextppos', 'ctx.pattern'],
+ reds=['ptr', 'self', 'ctx'],
+ debugprint=(1, 0)) # indices in 'greens'
def __init__(self, nextppos, minptr, ptr, marks):
self.nextppos = nextppos
@@ -259,8 +270,11 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
+ nextppos = self.nextppos
while ptr >= self.minptr:
- result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
+ ctx.jitdriver_RepeatOne.jit_merge_point(
+ self=self, ptr=ptr, ctx=ctx, nextppos=nextppos)
+ result = sre_match(ctx, nextppos, ptr, self.start_marks)
ptr -= 1
if result is not None:
self.subresult = result
@@ -270,6 +284,10 @@
class MinRepeatOneMatchResult(MatchResult):
+ install_jitdriver('MinRepeatOne',
+ greens=['nextppos', 'ppos3', 'ctx.pattern'],
+ reds=['ptr', 'self', 'ctx'],
+ debugprint=(2, 0)) # indices in 'greens'
def __init__(self, nextppos, ppos3, maxptr, ptr, marks):
self.nextppos = nextppos
@@ -280,29 +298,32 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
+ nextppos = self.nextppos
+ ppos3 = self.ppos3
while ptr <= self.maxptr:
- result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
+ ctx.jitdriver_MinRepeatOne.jit_merge_point(
+ self=self, ptr=ptr, ctx=ctx, nextppos=nextppos, ppos3=ppos3)
+ result = sre_match(ctx, nextppos, ptr, self.start_marks)
if result is not None:
self.subresult = result
self.start_ptr = ptr
return self
- if not self.next_char_ok(ctx, ptr):
+ if not self.next_char_ok(ctx, ptr, ppos3):
break
ptr += 1
def find_next_result(self, ctx):
ptr = self.start_ptr
- if not self.next_char_ok(ctx, ptr):
+ if not self.next_char_ok(ctx, ptr, self.ppos3):
return
self.start_ptr = ptr + 1
return self.find_first_result(ctx)
- def next_char_ok(self, ctx, ptr):
+ def next_char_ok(self, ctx, ptr, ppos):
if ptr == ctx.end:
return False
- ppos = self.ppos3
op = ctx.pat(ppos)
- for op1, (checkerfn, _) in unroll_char_checker:
+ for op1, checkerfn in unroll_char_checker:
if op1 == op:
return checkerfn(ctx, ptr, ppos)
raise Error("next_char_ok[%d]" % op)
@@ -325,41 +346,34 @@
self.next = next # chained list
class MaxUntilMatchResult(AbstractUntilMatchResult):
+ install_jitdriver('MaxUntil',
+ greens=['ppos', 'tailppos', 'match_more', 'ctx.pattern'],
+ reds=['ptr', 'marks', 'self', 'ctx'],
+ debugprint=(3, 0, 2))
def find_first_result(self, ctx):
- enum = sre_match(ctx, self.ppos + 3, self.cur_ptr, self.cur_marks)
- return self.search_next(ctx, enum, resume=False)
+ return self.search_next(ctx, match_more=True)
def find_next_result(self, ctx):
- return self.search_next(ctx, None, resume=True)
+ return self.search_next(ctx, match_more=False)
- def search_next(self, ctx, enum, resume):
+ def search_next(self, ctx, match_more):
ppos = self.ppos
- min = ctx.pat(ppos+1)
- max = ctx.pat(ppos+2)
+ tailppos = self.tailppos
ptr = self.cur_ptr
marks = self.cur_marks
while True:
- while True:
- if (enum is not None and
- (ptr != ctx.match_end or self.num_pending < min)):
- # ^^^^^^^^^^ zero-width match protection
- # matched one more 'item'. record it and continue.
- self.pending = Pending(ptr, marks, enum, self.pending)
- self.num_pending += 1
- ptr = ctx.match_end
- marks = ctx.match_marks
- break
- # 'item' no longer matches.
- if not resume and self.num_pending >= min:
- # try to match 'tail' if we have enough 'item'
- result = sre_match(ctx, self.tailppos, ptr, marks)
- if result is not None:
- self.subresult = result
- self.cur_ptr = ptr
- self.cur_marks = marks
- return self
- resume = False
+ ctx.jitdriver_MaxUntil.jit_merge_point(
+ ppos=ppos, tailppos=tailppos, match_more=match_more,
+ ptr=ptr, marks=marks, self=self, ctx=ctx)
+ if match_more:
+ max = ctx.pat(ppos+2)
+ if max == 65535 or self.num_pending < max:
+ # try to match one more 'item'
+ enum = sre_match(ctx, ppos + 3, ptr, marks)
+ else:
+ enum = None # 'max' reached, no more matches
+ else:
p = self.pending
if p is None:
return
@@ -369,11 +383,27 @@
marks = p.marks
enum = p.enum.move_to_next_result(ctx)
#
- if max == 65535 or self.num_pending < max:
- # try to match one more 'item'
- enum = sre_match(ctx, ppos + 3, ptr, marks)
+ min = ctx.pat(ppos+1)
+ if (enum is not None and
+ (ptr != ctx.match_end or self.num_pending < min)):
+ # ^^^^^^^^^^ zero-width match protection
+ # matched one more 'item'. record it and continue.
+ self.pending = Pending(ptr, marks, enum, self.pending)
+ self.num_pending += 1
+ ptr = ctx.match_end
+ marks = ctx.match_marks
+ match_more = True
else:
- enum = None # 'max' reached, no more matches
+ # 'item' no longer matches.
+ if self.num_pending >= min:
+ # try to match 'tail' if we have enough 'item'
+ result = sre_match(ctx, tailppos, ptr, marks)
+ if result is not None:
+ self.subresult = result
+ self.cur_ptr = ptr
+ self.cur_marks = marks
+ return self
+ match_more = False
class MinUntilMatchResult(AbstractUntilMatchResult):
@@ -384,6 +414,7 @@
return self.search_next(ctx, resume=True)
def search_next(self, ctx, resume):
+ # XXX missing jit support here
ppos = self.ppos
min = ctx.pat(ppos+1)
max = ctx.pat(ppos+2)
@@ -429,6 +460,7 @@
# ____________________________________________________________
@specializectx
+ at jit.unroll_safe
def sre_match(ctx, ppos, ptr, marks):
"""Returns either None or a MatchResult object. Usually we only need
the first result, but there is the case of REPEAT...UNTIL where we
@@ -438,6 +470,12 @@
op = ctx.pat(ppos)
ppos += 1
+ #jit.jit_debug("sre_match", op, ppos, ptr)
+ #
+ # When using the JIT, calls to sre_match() must always have a constant
+ # (green) argument for 'ppos'. If not, the following assert fails.
+ jit.assert_green(op)
+
if op == OPCODE_FAILURE:
return
@@ -712,13 +750,23 @@
@specializectx
def find_repetition_end(ctx, ppos, ptr, maxcount):
end = ctx.end
- # adjust end
- if maxcount != 65535:
+ if maxcount <= 1:
+ if maxcount == 1 and ptr < end:
+ # Relatively common case: maxcount == 1. If we are not at the
+ # end of the string, it's done by a single direct check.
+ op = ctx.pat(ppos)
+ for op1, checkerfn in unroll_char_checker:
+ if op1 == op:
+ if checkerfn(ctx, ptr, ppos):
+ return ptr + 1
+ return ptr
+ elif maxcount != 65535:
+ # adjust end
end1 = ptr + maxcount
if end1 <= end:
end = end1
op = ctx.pat(ppos)
- for op1, (_, fre) in unroll_char_checker:
+ for op1, fre in unroll_fre_checker:
if op1 == op:
return fre(ctx, ptr, end, ppos)
raise Error("rsre.find_repetition_end[%d]" % op)
@@ -751,23 +799,60 @@
if checkerfn == match_ANY_ALL:
def fre(ctx, ptr, end, ppos):
return end
+ elif checkerfn == match_IN:
+ install_jitdriver_spec('MatchIn',
+ greens=['ppos', 'ctx.pattern'],
+ reds=['ptr', 'end', 'ctx'],
+ debugprint=(1, 0))
+ @specializectx
+ def fre(ctx, ptr, end, ppos):
+ while True:
+ ctx.jitdriver_MatchIn.jit_merge_point(ctx=ctx, ptr=ptr,
+ end=end, ppos=ppos)
+ if ptr < end and checkerfn(ctx, ptr, ppos):
+ ptr += 1
+ else:
+ return ptr
+ elif checkerfn == match_IN_IGNORE:
+ install_jitdriver_spec('MatchInIgnore',
+ greens=['ppos', 'ctx.pattern'],
+ reds=['ptr', 'end', 'ctx'],
+ debugprint=(1, 0))
+ @specializectx
+ def fre(ctx, ptr, end, ppos):
+ while True:
+ ctx.jitdriver_MatchInIgnore.jit_merge_point(ctx=ctx, ptr=ptr,
+ end=end, ppos=ppos)
+ if ptr < end and checkerfn(ctx, ptr, ppos):
+ ptr += 1
+ else:
+ return ptr
else:
+ # in the other cases, the fre() function is not JITted at all
+ # and is present as a residual call.
+ @specializectx
def fre(ctx, ptr, end, ppos):
while ptr < end and checkerfn(ctx, ptr, ppos):
ptr += 1
return ptr
- return checkerfn, fre
+ fre = func_with_new_name(fre, 'fre_' + checkerfn.__name__)
+ return fre
+
+unroll_char_checker = [
+ (OPCODE_ANY, match_ANY),
+ (OPCODE_ANY_ALL, match_ANY_ALL),
+ (OPCODE_IN, match_IN),
+ (OPCODE_IN_IGNORE, match_IN_IGNORE),
+ (OPCODE_LITERAL, match_LITERAL),
+ (OPCODE_LITERAL_IGNORE, match_LITERAL_IGNORE),
+ (OPCODE_NOT_LITERAL, match_NOT_LITERAL),
+ (OPCODE_NOT_LITERAL_IGNORE, match_NOT_LITERAL_IGNORE),
+ ]
+unroll_fre_checker = [(_op, _make_fre(_fn))
+ for (_op, _fn) in unroll_char_checker]
-unroll_char_checker = unrolling_iterable([
- (OPCODE_ANY, _make_fre(match_ANY)),
- (OPCODE_ANY_ALL, _make_fre(match_ANY_ALL)),
- (OPCODE_IN, _make_fre(match_IN)),
- (OPCODE_IN_IGNORE, _make_fre(match_IN_IGNORE)),
- (OPCODE_LITERAL, _make_fre(match_LITERAL)),
- (OPCODE_LITERAL_IGNORE, _make_fre(match_LITERAL_IGNORE)),
- (OPCODE_NOT_LITERAL, _make_fre(match_NOT_LITERAL)),
- (OPCODE_NOT_LITERAL_IGNORE, _make_fre(match_NOT_LITERAL_IGNORE)),
- ])
+unroll_char_checker = unrolling_iterable(unroll_char_checker)
+unroll_fre_checker = unrolling_iterable(unroll_fre_checker)
##### At dispatch
@@ -873,74 +958,139 @@
else:
return None
+install_jitdriver('Match',
+ greens=['ctx.pattern'], reds=['ctx'],
+ debugprint=(0,))
+
def match_context(ctx):
ctx.original_pos = ctx.match_start
if ctx.end < ctx.match_start:
return False
+ ctx.jitdriver_Match.jit_merge_point(ctx=ctx)
return sre_match(ctx, 0, ctx.match_start, None) is not None
def search_context(ctx):
ctx.original_pos = ctx.match_start
if ctx.end < ctx.match_start:
return False
- if ctx.pat(0) == OPCODE_INFO:
- if ctx.pat(2) & rsre_char.SRE_INFO_PREFIX and ctx.pat(5) > 1:
- return fast_search(ctx)
- return regular_search(ctx)
+ base = 0
+ charset = False
+ if ctx.pat(base) == OPCODE_INFO:
+ flags = ctx.pat(2)
+ if flags & rsre_char.SRE_INFO_PREFIX:
+ if ctx.pat(5) > 1:
+ return fast_search(ctx)
+ else:
+ charset = (flags & rsre_char.SRE_INFO_CHARSET)
+ base += 1 + ctx.pat(1)
+ if ctx.pat(base) == OPCODE_LITERAL:
+ return literal_search(ctx, base)
+ if charset:
+ return charset_search(ctx, base)
+ return regular_search(ctx, base)
+
+install_jitdriver('RegularSearch',
+ greens=['base', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(1, 0))
-def regular_search(ctx):
+def regular_search(ctx, base):
start = ctx.match_start
while start <= ctx.end:
- if sre_match(ctx, 0, start, None) is not None:
+ ctx.jitdriver_RegularSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base)
+ if sre_match(ctx, base, start, None) is not None:
ctx.match_start = start
return True
start += 1
return False
+install_jitdriver_spec("LiteralSearch",
+ greens=['base', 'character', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(2, 0, 1))
+ at specializectx
+def literal_search(ctx, base):
+ # pattern starts with a literal character. this is used
+ # for short prefixes, and if fast search is disabled
+ character = ctx.pat(base + 1)
+ base += 2
+ start = ctx.match_start
+ while start < ctx.end:
+ ctx.jitdriver_LiteralSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base, character=character)
+ if ctx.str(start) == character:
+ if sre_match(ctx, base, start + 1, None) is not None:
+ ctx.match_start = start
+ return True
+ start += 1
+ return False
+
+install_jitdriver_spec("CharsetSearch",
+ greens=['base', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(1, 0))
+ at specializectx
+def charset_search(ctx, base):
+ # pattern starts with a character from a known set
+ start = ctx.match_start
+ while start < ctx.end:
+ ctx.jitdriver_CharsetSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base)
+ if rsre_char.check_charset(ctx.pattern, 5, ctx.str(start)):
+ if sre_match(ctx, base, start, None) is not None:
+ ctx.match_start = start
+ return True
+ start += 1
+ return False
+
+install_jitdriver_spec('FastSearch',
+ greens=['i', 'prefix_len', 'ctx.pattern'],
+ reds=['string_position', 'ctx'],
+ debugprint=(2, 0))
@specializectx
def fast_search(ctx):
# skips forward in a string as fast as possible using information from
# an optimization info block
# <1=skip> <2=flags> <3=min> <4=...>
# <5=length> <6=skip> <7=prefix data>
- flags = ctx.pat(2)
+ string_position = ctx.match_start
+ if string_position >= ctx.end:
+ return False
prefix_len = ctx.pat(5)
assert prefix_len >= 0
- prefix_skip = ctx.pat(6)
- assert prefix_skip >= 0
- overlap_offset = 7 + prefix_len - 1
- assert overlap_offset >= 0
- pattern_offset = ctx.pat(1) + 1
- ppos_start = pattern_offset + 2 * prefix_skip
- assert ppos_start >= 0
i = 0
- string_position = ctx.match_start
- end = ctx.end
- while string_position < end:
- while True:
- char_ord = ctx.str(string_position)
- if char_ord != ctx.pat(7 + i):
- if i == 0:
- break
- else:
- i = ctx.pat(overlap_offset + i)
- else:
- i += 1
- if i == prefix_len:
- # found a potential match
- start = string_position + 1 - prefix_len
- assert start >= 0
- ptr = start + prefix_skip
- if flags & rsre_char.SRE_INFO_LITERAL:
- # matched all of pure literal pattern
- ctx.match_start = start
- ctx.match_end = ptr
- ctx.match_marks = None
- return True
- if sre_match(ctx, ppos_start, ptr, None) is not None:
- ctx.match_start = start
- return True
- i = ctx.pat(overlap_offset + i)
- break
+ while True:
+ ctx.jitdriver_FastSearch.jit_merge_point(ctx=ctx,
+ string_position=string_position, i=i, prefix_len=prefix_len)
+ char_ord = ctx.str(string_position)
+ if char_ord != ctx.pat(7 + i):
+ if i > 0:
+ overlap_offset = prefix_len + (7 - 1)
+ i = ctx.pat(overlap_offset + i)
+ continue
+ else:
+ i += 1
+ if i == prefix_len:
+ # found a potential match
+ start = string_position + 1 - prefix_len
+ assert start >= 0
+ prefix_skip = ctx.pat(6)
+ ptr = start + prefix_skip
+ #flags = ctx.pat(2)
+ #if flags & rsre_char.SRE_INFO_LITERAL:
+ # # matched all of pure literal pattern
+ # ctx.match_start = start
+ # ctx.match_end = ptr
+ # ctx.match_marks = None
+ # return True
+ pattern_offset = ctx.pat(1) + 1
+ ppos_start = pattern_offset + 2 * prefix_skip
+ if sre_match(ctx, ppos_start, ptr, None) is not None:
+ ctx.match_start = start
+ return True
+ overlap_offset = prefix_len + (7 - 1)
+ i = ctx.pat(overlap_offset + i)
string_position += 1
- return False
+ if string_position >= ctx.end:
+ return False
Modified: pypy/trunk/pypy/rlib/test/test_jit.py
==============================================================================
--- pypy/trunk/pypy/rlib/test/test_jit.py (original)
+++ pypy/trunk/pypy/rlib/test/test_jit.py Mon Oct 18 17:24:50 2010
@@ -1,4 +1,5 @@
import py
+from pypy.conftest import option
from pypy.rlib.jit import hint, we_are_jitted, JitDriver, purefunction_promote
from pypy.rlib.jit import JitHintError, oopspec
from pypy.translator.translator import TranslationContext, graphof
@@ -110,6 +111,26 @@
return n
py.test.raises(JitHintError, self.gengraph, fn, [int])
+ def test_green_field(self):
+ def get_printable_location(xfoo):
+ return str(ord(xfoo)) # xfoo must be annotated as a character
+ myjitdriver = JitDriver(greens=['x.foo'], reds=['n', 'x'],
+ get_printable_location=get_printable_location)
+ class A(object):
+ _immutable_fields_ = ['foo']
+ def fn(n):
+ x = A()
+ x.foo = chr(n)
+ while n > 0:
+ myjitdriver.can_enter_jit(x=x, n=n)
+ myjitdriver.jit_merge_point(x=x, n=n)
+ n -= 1
+ return n
+ t = self.gengraph(fn, [int])[0]
+ if option.view:
+ t.view()
+ # assert did not raise
+
class TestJITLLtype(BaseTestJIT, LLRtypeMixin):
pass
From arigo at codespeak.net Mon Oct 18 17:25:39 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 17:25:39 +0200 (CEST)
Subject: [pypy-svn] r78050 - pypy/branch/rsre-jit
Message-ID: <20101018152539.9E304282B9D@codespeak.net>
Author: arigo
Date: Mon Oct 18 17:25:38 2010
New Revision: 78050
Removed:
pypy/branch/rsre-jit/
Log:
Remove merged branch.
From cfbolz at codespeak.net Mon Oct 18 17:32:19 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon, 18 Oct 2010 17:32:19 +0200 (CEST)
Subject: [pypy-svn] r78051 - pypy/extradoc/talk/pepm2011
Message-ID: <20101018153219.70A44282B9D@codespeak.net>
Author: cfbolz
Date: Mon Oct 18 17:32:17 2010
New Revision: 78051
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
small things, more fundamental changes to come
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Mon Oct 18 17:32:17 2010
@@ -557,20 +557,19 @@
\section{Allocation Removal in Traces}
\label{sec:statics}
-\subsection{Static Objects}
-
The main insight to improve the code shown in the last section is that objects
-in category 1 don't survive very long -- they are used only inside the loop and
-nobody else in the program stores a reference to them. The idea for improving
+in category 1 do not survive very long -- they are used only inside the loop and
+there is no other outside reference to them. The idea for improving
the code is to analyze which objects fall in category 1 and therefore do
not have to be allocated at all.
-This is a process that is usually called \emph{escape analysis}. In this paper we will
-perform escape analysis by using partial evaluation. The partial evalution is a
+This is a process that is usually called \emph{escape analysis}
+\cite{goldberg_higher_1990}. In this paper we will
+perform escape analysis by using partial evaluation. The use of partial evaluation is a
bit peculiar in that it receives no static input arguments for the trace,
-but it is only used to optimize operations within a trace.
+but it is only used to optimize operations within the trace.
-The partial evaluation works by traversing the trace from beginning to end.
+To optimize the trace, it is traversed from beginning to end.
Whenever a \lstinline{new} operation is seen, the operation is removed and a \emph{static
object}\footnote{Here ``static'' is meant in the sense of partial
evaluation, \ie known at partial evaluation time, not in the sense of static
From cfbolz at codespeak.net Mon Oct 18 17:39:03 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon, 18 Oct 2010 17:39:03 +0200 (CEST)
Subject: [pypy-svn] r78052 - pypy/extradoc/talk/pepm2011
Message-ID: <20101018153903.EE018282B9D@codespeak.net>
Author: cfbolz
Date: Mon Oct 18 17:39:02 2010
New Revision: 78052
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
stop using the graphic for the traces
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Mon Oct 18 17:39:02 2010
@@ -664,7 +664,29 @@
and \lstinline{guard_class} operations).
\begin{figure}
-\includegraphics{figures/step1.pdf}
+\begin{lstlisting}[mathescape]
+# arguments to the trace: $p_{0}$, $p_{1}$
+guard_class($p_1$, BoxedInteger)
+$i_2$ = getfield($p_1$, intval)
+guard_class($p_0$, BoxedInteger)
+$i_3$ = getfield($p_0$, intval)
+$i_4$ = int_add($i_2$, $i_3$)
+$i_9$ = int_add($i_4$, -100)
+
+guard_class($p_0$, BoxedInteger)
+$i_{12}$ = getfield($p_0$, intval)
+$i_{14}$ = int_add($i_{12}$, -1)
+
+$i_{17}$ = int_gt($i_{14}$, 0)
+guard_true($i_{17}$)
+
+$p_{15}$ = new(BoxedInteger)
+setfield($p_{15}$, intval, $i_{14}$)
+$p_{10}$ = new(BoxedInteger)
+setfield($p_{10}$, intval, $i_9$)
+
+jump($p_{15}$, $p_{10}$)
+\end{lstlisting}
\caption{Resulting Trace After Allocation Removal}
\label{fig:step1}
\end{figure}
From arigo at codespeak.net Mon Oct 18 17:39:11 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 17:39:11 +0200 (CEST)
Subject: [pypy-svn] r78053 - pypy/trunk/pypy/jit/backend/x86/test
Message-ID: <20101018153911.2A5F8282BAD@codespeak.net>
Author: arigo
Date: Mon Oct 18 17:39:09 2010
New Revision: 78053
Modified:
pypy/trunk/pypy/jit/backend/x86/test/test_runner.py
Log:
Write a (failing) test about calling C functions that return
a shorter integer than a word.
Modified: pypy/trunk/pypy/jit/backend/x86/test/test_runner.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/test/test_runner.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/test/test_runner.py Mon Oct 18 17:39:09 2010
@@ -12,6 +12,7 @@
from pypy.jit.backend.test.runner_test import LLtypeBackendTest
from pypy.jit.metainterp.test.oparser import parse
from pypy.tool.udir import udir
+from pypy.rlib.rarithmetic import intmask
import ctypes
import sys
import os
@@ -396,6 +397,38 @@
res = self.execute_operation(rop.CALL, [funcbox, ConstFloat(1.5), ConstFloat(2.5)], 'float', descr=calldescr)
assert res.value == 4.0
+ def test_result_of_call(self):
+ # Test that calling a function that returns a CHAR, SHORT or INT,
+ # signed or unsigned, properly gets zero-extended or sign-extended.
+ from pypy.translator.tool.cbuild import ExternalCompilationInfo
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ # Tested with a function that intentionally does not cast the
+ # result to RESTYPE, but makes sure that we return the whole
+ # value in eax or rax.
+ eci = ExternalCompilationInfo(separate_module_sources=["""
+ long fn_test_result_of_call(long x)
+ {
+ return x + 1;
+ }
+ """])
+ f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed],
+ RESTYPE, compilation_info=eci, _nowrapper=True)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1))
+ assert f(value) == expected
+ #
+ FUNC = self.FuncType([lltype.Signed], RESTYPE)
+ FPTR = self.Ptr(FUNC)
+ calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+ funcbox = self.get_funcbox(self.cpu, f)
+ res = self.execute_operation(rop.CALL, [funcbox, BoxInt(value)],
+ 'int', descr=calldescr)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
class TestX86OverflowMC(TestX86):
From antocuni at codespeak.net Mon Oct 18 17:40:22 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 18 Oct 2010 17:40:22 +0200 (CEST)
Subject: [pypy-svn] r78054 - pypy/branch/jitffi
Message-ID: <20101018154022.037F1282B90@codespeak.net>
Author: antocuni
Date: Mon Oct 18 17:40:21 2010
New Revision: 78054
Removed:
pypy/branch/jitffi/
Log:
remove merged branch
From cfbolz at codespeak.net Mon Oct 18 18:03:57 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon, 18 Oct 2010 18:03:57 +0200 (CEST)
Subject: [pypy-svn] r78055 - pypy/extradoc/talk/pepm2011
Message-ID: <20101018160357.0943F282B90@codespeak.net>
Author: cfbolz
Date: Mon Oct 18 18:03:56 2010
New Revision: 78055
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
gaaaah
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Mon Oct 18 18:03:56 2010
@@ -667,23 +667,23 @@
\begin{lstlisting}[mathescape]
# arguments to the trace: $p_{0}$, $p_{1}$
guard_class($p_1$, BoxedInteger)
-$i_2$ = getfield($p_1$, intval)
+$i_2$ = get($p_1$, intval)
guard_class($p_0$, BoxedInteger)
-$i_3$ = getfield($p_0$, intval)
+$i_3$ = get($p_0$, intval)
$i_4$ = int_add($i_2$, $i_3$)
$i_9$ = int_add($i_4$, -100)
guard_class($p_0$, BoxedInteger)
-$i_{12}$ = getfield($p_0$, intval)
+$i_{12}$ = get($p_0$, intval)
$i_{14}$ = int_add($i_{12}$, -1)
$i_{17}$ = int_gt($i_{14}$, 0)
guard_true($i_{17}$)
$p_{15}$ = new(BoxedInteger)
-setfield($p_{15}$, intval, $i_{14}$)
+set($p_{15}$, intval, $i_{14}$)
$p_{10}$ = new(BoxedInteger)
-setfield($p_{10}$, intval, $i_9$)
+set($p_{10}$, intval, $i_9$)
jump($p_{15}$, $p_{10}$)
\end{lstlisting}
From arigo at codespeak.net Mon Oct 18 18:16:05 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 18:16:05 +0200 (CEST)
Subject: [pypy-svn] r78056 - pypy/trunk/pypy/jit/backend/x86/test
Message-ID: <20101018161605.30A14282B90@codespeak.net>
Author: arigo
Date: Mon Oct 18 18:16:03 2010
New Revision: 78056
Modified:
pypy/trunk/pypy/jit/backend/x86/test/test_runner.py
Log:
Revert the test. Will be moved to its own branch.
Modified: pypy/trunk/pypy/jit/backend/x86/test/test_runner.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/test/test_runner.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/test/test_runner.py Mon Oct 18 18:16:03 2010
@@ -12,7 +12,6 @@
from pypy.jit.backend.test.runner_test import LLtypeBackendTest
from pypy.jit.metainterp.test.oparser import parse
from pypy.tool.udir import udir
-from pypy.rlib.rarithmetic import intmask
import ctypes
import sys
import os
@@ -397,38 +396,6 @@
res = self.execute_operation(rop.CALL, [funcbox, ConstFloat(1.5), ConstFloat(2.5)], 'float', descr=calldescr)
assert res.value == 4.0
- def test_result_of_call(self):
- # Test that calling a function that returns a CHAR, SHORT or INT,
- # signed or unsigned, properly gets zero-extended or sign-extended.
- from pypy.translator.tool.cbuild import ExternalCompilationInfo
- for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
- rffi.SHORT, rffi.USHORT,
- rffi.INT, rffi.UINT,
- rffi.LONG, rffi.ULONG]:
- # Tested with a function that intentionally does not cast the
- # result to RESTYPE, but makes sure that we return the whole
- # value in eax or rax.
- eci = ExternalCompilationInfo(separate_module_sources=["""
- long fn_test_result_of_call(long x)
- {
- return x + 1;
- }
- """])
- f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed],
- RESTYPE, compilation_info=eci, _nowrapper=True)
- value = intmask(0xFFEEDDCCBBAA9988)
- expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1))
- assert f(value) == expected
- #
- FUNC = self.FuncType([lltype.Signed], RESTYPE)
- FPTR = self.Ptr(FUNC)
- calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
- funcbox = self.get_funcbox(self.cpu, f)
- res = self.execute_operation(rop.CALL, [funcbox, BoxInt(value)],
- 'int', descr=calldescr)
- assert res.value == expected, (
- "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
-
class TestX86OverflowMC(TestX86):
From arigo at codespeak.net Mon Oct 18 18:16:56 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 18:16:56 +0200 (CEST)
Subject: [pypy-svn] r78057 - pypy/branch/jit-sign-descr
Message-ID: <20101018161656.CCE67282B90@codespeak.net>
Author: arigo
Date: Mon Oct 18 18:16:55 2010
New Revision: 78057
Added:
pypy/branch/jit-sign-descr/
- copied from r78056, pypy/trunk/
Log:
We are missing a flag in all int-supporting descrs to know if the type
is signed or unsigned.
From cfbolz at codespeak.net Mon Oct 18 18:20:33 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon, 18 Oct 2010 18:20:33 +0200 (CEST)
Subject: [pypy-svn] r78058 - pypy/extradoc/talk/pepm2011
Message-ID: <20101018162033.4F8EC282B90@codespeak.net>
Author: cfbolz
Date: Mon Oct 18 18:20:31 2010
New Revision: 78058
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
more nonsense
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Mon Oct 18 18:20:31 2010
@@ -610,8 +610,8 @@
\begin{lstlisting}[mathescape,xleftmargin=20pt]
guard_class($p_{5}$, BoxedInteger)
-$i_{7}$ = get($p_{5}$, intval)
# inside BoxedInteger.add
+$i_{7}$ = get($p_{5}$, intval)
guard_class($p_{6}$, BoxedInteger)
# inside BoxedInteger.add__int
$i_{8}$ = get($p_{6}$, intval)
From arigo at codespeak.net Mon Oct 18 18:23:09 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 18:23:09 +0200 (CEST)
Subject: [pypy-svn] r78059 - pypy/branch/jit-sign-descr/pypy/jit/backend/test
Message-ID: <20101018162309.80EA0282B90@codespeak.net>
Author: arigo
Date: Mon Oct 18 18:23:08 2010
New Revision: 78059
Modified:
pypy/branch/jit-sign-descr/pypy/jit/backend/test/runner_test.py
Log:
Re-check-in this test, this time as a general backend test.
Modified: pypy/branch/jit-sign-descr/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/branch/jit-sign-descr/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/branch/jit-sign-descr/pypy/jit/backend/test/runner_test.py Mon Oct 18 18:23:08 2010
@@ -15,6 +15,7 @@
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.llinterp import LLException
from pypy.jit.codewriter import heaptracker
+from pypy.rlib.rarithmetic import intmask
class Runner(object):
@@ -2027,6 +2028,38 @@
assert self.cpu.get_latest_value_float(0) == 13.5
assert called
+ def test_result_of_call(self):
+ # Test that calling a function that returns a CHAR, SHORT or INT,
+ # signed or unsigned, properly gets zero-extended or sign-extended.
+ from pypy.translator.tool.cbuild import ExternalCompilationInfo
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ # Tested with a function that intentionally does not cast the
+ # result to RESTYPE, but makes sure that we return the whole
+ # value in eax or rax.
+ eci = ExternalCompilationInfo(separate_module_sources=["""
+ long fn_test_result_of_call(long x)
+ {
+ return x + 1;
+ }
+ """])
+ f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed],
+ RESTYPE, compilation_info=eci, _nowrapper=True)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1))
+ assert intmask(f(value)) == expected
+ #
+ FUNC = self.FuncType([lltype.Signed], RESTYPE)
+ FPTR = self.Ptr(FUNC)
+ calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+ funcbox = self.get_funcbox(self.cpu, f)
+ res = self.execute_operation(rop.CALL, [funcbox, BoxInt(value)],
+ 'int', descr=calldescr)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
class OOtypeBackendTest(BaseBackendTest):
From arigo at codespeak.net Mon Oct 18 18:41:34 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 18:41:34 +0200 (CEST)
Subject: [pypy-svn] r78060 - pypy/branch/jit-sign-descr/pypy/jit/backend/test
Message-ID: <20101018164134.647B6282B9D@codespeak.net>
Author: arigo
Date: Mon Oct 18 18:41:30 2010
New Revision: 78060
Modified:
pypy/branch/jit-sign-descr/pypy/jit/backend/test/runner_test.py
Log:
More tests, trying to cover all descrs, both in "directly executed by
bh_xxx" mode and in "rop.XXX" operation mode.
Modified: pypy/branch/jit-sign-descr/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/branch/jit-sign-descr/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/branch/jit-sign-descr/pypy/jit/backend/test/runner_test.py Mon Oct 18 18:41:30 2010
@@ -2028,7 +2028,165 @@
assert self.cpu.get_latest_value_float(0) == 13.5
assert called
- def test_result_of_call(self):
+ def test_short_result_of_getfield_direct(self):
+ # Test that a getfield that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ descrfld_x = cpu.fielddescrof(S, 'x')
+ s = lltype.malloc(S)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ s.x = rffi.cast(RESTYPE, value)
+ x = cpu.bh_getfield_gc_i(lltype.cast_opaque_ptr(llmemory.GCREF, s),
+ descrfld_x)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_getfield_compiled(self):
+ # Test that a getfield that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ descrfld_x = cpu.fielddescrof(S, 'x')
+ s = lltype.malloc(S)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ s.x = rffi.cast(RESTYPE, value)
+ s_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
+ res = self.execute_operation(rop.GETFIELD_GC, [BoxPtr(s_gcref)],
+ 'int', descr=descrfld_x)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_getarrayitem_direct(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = lltype.GcArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ x = cpu.bh_getarrayitem_gc_i(
+ descrarray, lltype.cast_opaque_ptr(llmemory.GCREF, a), 3)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_getarrayitem_compiled(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = lltype.GcArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, a)
+ res = self.execute_operation(rop.GETARRAYITEM_GC,
+ [BoxPtr(a_gcref), BoxInt(3)],
+ 'int', descr=descrarray)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
+ def test_short_result_of_getarrayitem_raw_direct(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = rffi.CArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5, flavor='raw')
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_rawint = heaptracker.adr2int(llmemory.cast_ptr_to_adr(a))
+ x = cpu.bh_getarrayitem_raw_i(descrarray, a_rawint, 3)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+ lltype.free(a, flavor='raw')
+
+ def test_short_result_of_getarrayitem_raw_compiled(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = rffi.CArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5, flavor='raw')
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_rawint = heaptracker.adr2int(llmemory.cast_ptr_to_adr(a))
+ res = self.execute_operation(rop.GETARRAYITEM_RAW,
+ [BoxInt(a_rawint), BoxInt(3)],
+ 'int', descr=descrarray)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+ lltype.free(a, flavor='raw')
+
+ def test_short_result_of_call_direct(self):
+ # Test that calling a function that returns a CHAR, SHORT or INT,
+ # signed or unsigned, properly gets zero-extended or sign-extended.
+ from pypy.translator.tool.cbuild import ExternalCompilationInfo
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ # Tested with a function that intentionally does not cast the
+ # result to RESTYPE, but makes sure that we return the whole
+ # value in eax or rax.
+ eci = ExternalCompilationInfo(separate_module_sources=["""
+ long fn_test_result_of_call(long x)
+ {
+ return x + 1;
+ }
+ """])
+ f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed],
+ RESTYPE, compilation_info=eci, _nowrapper=True)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1))
+ assert intmask(f(value)) == expected
+ #
+ FUNC = self.FuncType([lltype.Signed], RESTYPE)
+ FPTR = self.Ptr(FUNC)
+ calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+ x = self.cpu.bh_call_i(self.get_funcbox(self.cpu, f).value,
+ calldescr, [value], None, None)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_call_compiled(self):
# Test that calling a function that returns a CHAR, SHORT or INT,
# signed or unsigned, properly gets zero-extended or sign-extended.
from pypy.translator.tool.cbuild import ExternalCompilationInfo
From cfbolz at codespeak.net Mon Oct 18 19:07:31 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon, 18 Oct 2010 19:07:31 +0200 (CEST)
Subject: [pypy-svn] r78061 - pypy/extradoc/talk/pepm2011
Message-ID: <20101018170731.B90B1282B9D@codespeak.net>
Author: cfbolz
Date: Mon Oct 18 19:07:26 2010
New Revision: 78061
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
more small tweaks, still missing a more aggressive rewrite
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Mon Oct 18 19:07:26 2010
@@ -302,6 +302,7 @@
code.
\subsection{Running Example}
+\label{sub:example}
For the purpose of this paper, we are going to use a tiny interpreter for a dynamic language with
a very simple object
@@ -387,6 +388,11 @@
\begin{figure}
+\lstset{
+ numbers = right,
+ numberstyle = \tiny,
+ numbersep = 0pt
+}
\begin{lstlisting}[mathescape]
# arguments to the trace: $p_{0}$, $p_{1}$
# inside f: res.add(y)
@@ -558,10 +564,10 @@
\label{sec:statics}
The main insight to improve the code shown in the last section is that objects
-in category 1 do not survive very long -- they are used only inside the loop and
-there is no other outside reference to them. The idea for improving
-the code is to analyze which objects fall in category 1 and therefore do
-not have to be allocated at all.
+in category 1 do not survive very long -- they are used only inside the loop
+and there is no other outside reference to them. Therefore the optimizer
+identifies objects in category 1 and removes the allocation of these objects,
+and all operations manipulating them.
This is a process that is usually called \emph{escape analysis}
\cite{goldberg_higher_1990}. In this paper we will
@@ -587,7 +593,7 @@
as well, because the shape description stores the type and thus the outcome of
the type check the guard does is statically known.
-In the example from last section, the following operations in the upper half
+In the example from Section~\ref{sub:example}, the following operations in the upper half
of Figure~\ref{fig:unopt-trace} produce two
static objects, and can be completely removed from the optimized trace:
@@ -737,8 +743,9 @@
In this section we want to give a formal description of the semantics of the
traces and of the optimizer and liken the optimization to partial evaluation.
We focus on the operations for manipulating dynamically allocated objects,
-as those are the only ones that are actually optimized. Without loss of
-generality we also consider only objects with two fields in this section.
+as those are the only ones that are actually optimized. We also consider only
+objects with two fields in this section, generalizing to arbitrary many fields
+is straightforward.
Traces are lists of operations. The operations considered here are
\lstinline{new}, \lstinline{get}, \lstinline{set} and \lstinline{guard_class}.
@@ -853,8 +860,8 @@
The static heap is a partial function from $V^*$ into the
set of static objects, which are triples of a type and two elements of $V^*$.
-A variable $v^*$ is in the domain of the static heap $S$ as long as the
-optimizer does not need to become dynamic XXX. The object $S(v^*)$ describes
+The object referenced by a variable $v^*$ is static, if $v^*$ is in the domain
+of the static heap $S$. The object $S(v^*)$ describes
what is statically known about the object, \ie its type and its fields. The
fields of objects in the static heap are also elements of $V^*$ (or null, for
short periods of time).
From arigo at codespeak.net Mon Oct 18 19:08:42 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 19:08:42 +0200 (CEST)
Subject: [pypy-svn] r78062 -
pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/test
Message-ID: <20101018170842.2BF70282B9D@codespeak.net>
Author: arigo
Date: Mon Oct 18 19:08:41 2010
New Revision: 78062
Modified:
pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/test/test_descr.py
Log:
Write tests for the descrs. Not implemented so far.
Modified: pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/test/test_descr.py
==============================================================================
--- pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/test/test_descr.py (original)
+++ pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/test/test_descr.py Mon Oct 18 19:08:41 2010
@@ -83,6 +83,18 @@
assert descr_f.is_float_field()
+def test_get_field_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ descr_x = get_field_descr(c2, S, 'x')
+ assert descr_x.is_field_signed() == signed
+
+
def test_get_array_descr():
U = lltype.Struct('U')
T = lltype.GcStruct('T')
@@ -164,6 +176,25 @@
assert descr.get_base_size(False) == 0
assert descr.get_ofs_length(False) == -1
+
+def test_get_array_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ A = lltype.GcArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ arraydescr = get_array_descr(c2, A)
+ assert arraydescr.is_item_signed() == signed
+ #
+ RA = rffi.CArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ arraydescr = get_array_descr(c2, RA)
+ assert arraydescr.is_item_signed() == signed
+
+
def test_get_call_descr_not_translated():
c0 = GcCache(False)
descr1 = get_call_descr(c0, [lltype.Char, lltype.Signed], lltype.Char)
@@ -219,6 +250,17 @@
extrainfo = descr3.get_extra_info()
assert extrainfo is None
+def test_get_call_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ A = lltype.GcArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ descr1 = get_call_descr(c2, [], RESTYPE)
+ assert descr1.is_result_signed() == signed
+
def test_repr_of_descr():
c0 = GcCache(False)
From afa at codespeak.net Mon Oct 18 19:09:38 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 18 Oct 2010 19:09:38 +0200 (CEST)
Subject: [pypy-svn] r78063 - in pypy/branch/fast-forward/pypy/objspace/std:
. test
Message-ID: <20101018170938.A6261282B9D@codespeak.net>
Author: afa
Date: Mon Oct 18 19:09:37 2010
New Revision: 78063
Modified:
pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py
pypy/branch/fast-forward/pypy/objspace/std/typeobject.py
Log:
Revert r78038, will do it differently.
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py Mon Oct 18 19:09:37 2010
@@ -28,39 +28,6 @@
raises(TypeError, type, 'sub', (stufftype,), {})
""")
- def test_safe_subclass(self):
- space = self.space
- class W_Base(W_Object):
- pass
- def descr__new__(space, w_subtype):
- return space.allocate_instance(W_Base, w_subtype)
- W_Base.typedef = StdTypeDef("base",
- __new__ = interp2app(descr__new__))
- w_base = space.gettypeobject(W_Base.typedef)
-
- # Cannot be created with base.__new__(derived)
- class W_Derived(W_Base):
- pass
- def descr__new__(space, w_subtype):
- return space.allocate_instance(W_Derived, w_subtype)
- W_Derived.typedef = StdTypeDef("derived", W_Base.typedef,
- __new__ = interp2app(descr__new__))
- w_derived = space.gettypeobject(W_Derived.typedef)
-
- # Can be created with base.__new__(derived2)
- class W_Derived2(W_Base):
- pass
- W_Derived2.typedef = StdTypeDef("derived2", W_Base.typedef,
- )
- w_derived2 = space.gettypeobject(W_Derived2.typedef)
-
- space.appexec([w_base, w_derived, w_derived2],
- """(base, derived, derived2):
- raises(TypeError, base.__new__, derived)
- x = base.__new__(derived2)
- assert isinstance(x, derived2)
- """)
-
def test_del_warning(self):
warnings = []
def my_warn(msg, warningscls):
Modified: pypy/branch/fast-forward/pypy/objspace/std/typeobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/typeobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/typeobject.py Mon Oct 18 19:09:37 2010
@@ -326,8 +326,7 @@
raise operationerrfmt(space.w_TypeError,
"%s.__new__(%s): %s is not a subtype of %s",
w_self.name, w_subtype.name, w_subtype.name, w_self.name)
- if (w_self.instancetypedef is not w_subtype.instancetypedef and
- w_self.lookup('__new__') is not w_subtype.lookup('__new__')):
+ if w_self.instancetypedef is not w_subtype.instancetypedef:
raise operationerrfmt(space.w_TypeError,
"%s.__new__(%s) is not safe, use %s.__new__()",
w_self.name, w_subtype.name, w_subtype.name)
From antocuni at codespeak.net Mon Oct 18 19:13:57 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Mon, 18 Oct 2010 19:13:57 +0200 (CEST)
Subject: [pypy-svn] r78064 - pypy/extradoc/talk/pepm2011
Message-ID: <20101018171357.E3B42282B9D@codespeak.net>
Author: antocuni
Date: Mon Oct 18 19:13:55 2010
New Revision: 78064
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
customize line numbers in the code listing
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Mon Oct 18 19:13:55 2010
@@ -670,16 +670,16 @@
and \lstinline{guard_class} operations).
\begin{figure}
-\begin{lstlisting}[mathescape]
-# arguments to the trace: $p_{0}$, $p_{1}$
+\begin{lstlisting}[mathescape,numbers=left,escapechar=|]
+# arguments to the trace: $p_{0}$, $p_{1}$ |\setcounter{lstnumber}{2}|
guard_class($p_1$, BoxedInteger)
$i_2$ = get($p_1$, intval)
guard_class($p_0$, BoxedInteger)
$i_3$ = get($p_0$, intval)
$i_4$ = int_add($i_2$, $i_3$)
$i_9$ = int_add($i_4$, -100)
-
-guard_class($p_0$, BoxedInteger)
+|\setcounter{lstnumber}{50}|
+guard_class($p_0$, BoxedInteger)
$i_{12}$ = get($p_0$, intval)
$i_{14}$ = int_add($i_{12}$, -1)
From afa at codespeak.net Mon Oct 18 19:19:57 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 18 Oct 2010 19:19:57 +0200 (CEST)
Subject: [pypy-svn] r78065 - in
pypy/branch/fast-forward/pypy/module/exceptions: . test
Message-ID: <20101018171957.579FE282B9D@codespeak.net>
Author: afa
Date: Mon Oct 18 19:19:55 2010
New Revision: 78065
Modified:
pypy/branch/fast-forward/pypy/module/exceptions/interp_exceptions.py
pypy/branch/fast-forward/pypy/module/exceptions/test/test_exc.py
Log:
This seems enough to fix multiple inheritance of Exception classes:
you don't need a __new__ if the class has the same layout as its base.
Modified: pypy/branch/fast-forward/pypy/module/exceptions/interp_exceptions.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/exceptions/interp_exceptions.py (original)
+++ pypy/branch/fast-forward/pypy/module/exceptions/interp_exceptions.py Mon Oct 18 19:19:55 2010
@@ -254,7 +254,6 @@
base.typedef,
__doc__ = W_Exc.__doc__,
__module__ = 'exceptions',
- __new__ = _new(W_Exc, realbase),
**kwargs
)
W_Exc.typedef.applevel_subclasses_base = realbase
Modified: pypy/branch/fast-forward/pypy/module/exceptions/test/test_exc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/exceptions/test/test_exc.py (original)
+++ pypy/branch/fast-forward/pypy/module/exceptions/test/test_exc.py Mon Oct 18 19:19:55 2010
@@ -68,6 +68,7 @@
assert isinstance(Exception(), BaseException)
assert repr(Exception(3, "x")) == "Exception(3, 'x')"
assert str(IOError("foo", "bar")) == "[Errno foo] bar"
+ assert isinstance(IOError("foo", "bar"), IOError)
def test_custom_class(self):
from exceptions import Exception, BaseException, LookupError
@@ -185,7 +186,7 @@
raises(TypeError, UnicodeEncodeError, u"x", u"y", 1, 5, "bah")
def test_multiple_inheritance(self):
- from exceptions import LookupError, ValueError, Exception
+ from exceptions import LookupError, ValueError, Exception, IOError
class A(LookupError, ValueError):
pass
assert issubclass(A, A)
@@ -209,6 +210,17 @@
else:
fail("bah")
+ class C(ValueError, IOError):
+ pass
+ c = C()
+ assert isinstance(ValueError(), ValueError)
+ assert isinstance(c, C)
+ assert isinstance(c, Exception)
+ assert isinstance(c, ValueError)
+ assert isinstance(c, IOError)
+ assert isinstance(c, EnvironmentError)
+ assert not isinstance(c, KeyError)
+
def test_doc_and_module(self):
import exceptions
for name, e in exceptions.__dict__.items():
From arigo at codespeak.net Mon Oct 18 19:22:38 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 19:22:38 +0200 (CEST)
Subject: [pypy-svn] r78066 -
pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport
Message-ID: <20101018172238.AA58C282B9D@codespeak.net>
Author: arigo
Date: Mon Oct 18 19:22:37 2010
New Revision: 78066
Modified:
pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/descr.py
Log:
Implement the sign flag in the descrs.
Modified: pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/descr.py
==============================================================================
--- pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/descr.py (original)
+++ pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/descr.py Mon Oct 18 19:22:37 2010
@@ -82,6 +82,7 @@
_is_pointer_field = False # unless overridden by GcPtrFieldDescr
_is_float_field = False # unless overridden by FloatFieldDescr
+ _is_field_signed = False # unless overridden by XxxFieldDescr
def is_pointer_field(self):
return self._is_pointer_field
@@ -89,6 +90,9 @@
def is_float_field(self):
return self._is_float_field
+ def is_field_signed(self):
+ return self._is_field_signed
+
def repr_of_descr(self):
return '<%s %s %s>' % (self._clsname, self.name, self.offset)
@@ -105,7 +109,7 @@
def getFieldDescrClass(TYPE):
return getDescrClass(TYPE, BaseFieldDescr, GcPtrFieldDescr,
NonGcPtrFieldDescr, 'Field', 'get_field_size',
- '_is_float_field')
+ '_is_float_field', '_is_field_signed')
def get_field_descr(gccache, STRUCT, fieldname):
cache = gccache._cache_field
@@ -144,6 +148,7 @@
_is_array_of_pointers = False # unless overridden by GcPtrArrayDescr
_is_array_of_floats = False # unless overridden by FloatArrayDescr
+ _is_item_signed = False # unless overridden by XxxArrayDescr
def is_array_of_pointers(self):
return self._is_array_of_pointers
@@ -151,6 +156,9 @@
def is_array_of_floats(self):
return self._is_array_of_floats
+ def is_item_signed(self):
+ return self._is_item_signed
+
def repr_of_descr(self):
return '<%s>' % self._clsname
@@ -186,12 +194,12 @@
def getArrayDescrClass(ARRAY):
return getDescrClass(ARRAY.OF, BaseArrayDescr, GcPtrArrayDescr,
NonGcPtrArrayDescr, 'Array', 'get_item_size',
- '_is_array_of_floats')
+ '_is_array_of_floats', '_is_item_signed')
def getArrayNoLengthDescrClass(ARRAY):
return getDescrClass(ARRAY.OF, BaseArrayNoLengthDescr, GcPtrArrayNoLengthDescr,
NonGcPtrArrayNoLengthDescr, 'ArrayNoLength', 'get_item_size',
- '_is_array_of_floats')
+ '_is_array_of_floats', '_is_item_signed')
def get_array_descr(gccache, ARRAY):
cache = gccache._cache_array
@@ -242,6 +250,9 @@
def get_result_size(self, translate_support_code):
raise NotImplementedError
+ def is_result_signed(self):
+ return False # unless overridden
+
def create_call_stub(self, rtyper, RESULT):
def process(c):
arg = 'args_%s[%d]' % (c, seen[c])
@@ -307,6 +318,10 @@
_return_type = history.INT
call_stub = staticmethod(lambda func, args_i, args_r, args_f: 0)
+ _is_result_signed = False # can be overridden in XxxCallDescr
+ def is_result_signed(self):
+ return self._is_result_signed
+
class DynamicIntCallDescr(BaseIntCallDescr):
"""
calldescr that works for every integer type, by explicitly passing it the
@@ -314,12 +329,17 @@
"""
_clsname = 'DynamicIntCallDescr'
- def __init__(self, arg_classes, result_size, extrainfo=None):
+ def __init__(self, arg_classes, result_size, result_sign, extrainfo=None):
BaseIntCallDescr.__init__(self, arg_classes, extrainfo)
- self._result_size = result_size
+ assert isinstance(result_sign, bool)
+ self._result_size = chr(result_size)
+ self._result_sign = result_sign
def get_result_size(self, translate_support_code):
- return self._result_size
+ return ord(self._result_size)
+
+ def is_result_signed(self):
+ return self._result_sign
class NonGcPtrCallDescr(BaseIntCallDescr):
@@ -356,7 +376,8 @@
return FloatCallDescr
return getDescrClass(RESULT, BaseIntCallDescr, GcPtrCallDescr,
NonGcPtrCallDescr, 'Call', 'get_result_size',
- Ellipsis) # <= floatattrname should not be used here
+ Ellipsis, # <= floatattrname should not be used here
+ '_is_result_signed')
def get_call_descr(gccache, ARGS, RESULT, extrainfo=None):
arg_classes = []
@@ -383,7 +404,8 @@
# ____________________________________________________________
def getDescrClass(TYPE, BaseDescr, GcPtrDescr, NonGcPtrDescr,
- nameprefix, methodname, floatattrname, _cache={}):
+ nameprefix, methodname, floatattrname, signedattrname,
+ _cache={}):
if isinstance(TYPE, lltype.Ptr):
if TYPE.TO._gckind == 'gc':
return GcPtrDescr
@@ -403,6 +425,8 @@
#
if TYPE is lltype.Float:
setattr(Descr, floatattrname, True)
+ elif rffi.cast(TYPE, -1) == -1:
+ setattr(Descr, signedattrname, True)
#
_cache[nameprefix, TYPE] = Descr
return Descr
From arigo at codespeak.net Mon Oct 18 19:38:53 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 19:38:53 +0200 (CEST)
Subject: [pypy-svn] r78067 - in pypy/branch/jit-sign-descr/pypy:
jit/backend/llsupport rlib
Message-ID: <20101018173853.21DDB282B90@codespeak.net>
Author: arigo
Date: Mon Oct 18 19:38:52 2010
New Revision: 78067
Modified:
pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/ffisupport.py
pypy/branch/jit-sign-descr/pypy/rlib/libffi.py
Log:
Pass 'signed-or-not' information from libffi to ffisupport.
Modified: pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/ffisupport.py
==============================================================================
--- pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/ffisupport.py (original)
+++ pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/ffisupport.py Mon Oct 18 19:38:52 2010
@@ -14,7 +14,8 @@
arg_classes = ''.join(argkinds)
if reskind == history.INT:
size = intmask(ffi_result.c_size)
- return DynamicIntCallDescr(arg_classes, size, extrainfo)
+ signed = is_ffi_type_signed(ffi_result)
+ return DynamicIntCallDescr(arg_classes, size, signed, extrainfo)
elif reskind == history.REF:
return NonGcPtrCallDescr(arg_classes, extrainfo)
elif reskind == history.FLOAT:
@@ -26,10 +27,15 @@
def get_ffi_type_kind(ffi_type):
from pypy.rlib.libffi import types
kind = types.getkind(ffi_type)
- if kind == 'i':
+ if kind == 'i' or kind == 'u':
return history.INT
elif kind == 'f':
return history.FLOAT
elif kind == 'v':
return history.VOID
- assert False, "Unsuported kind '%s'" % kind
+ assert False, "Unsupported kind '%s'" % kind
+
+def is_ffi_type_signed(ffi_type):
+ from pypy.rlib.libffi import types
+ kind = types.getkind(ffi_type)
+ return kind != 'u'
Modified: pypy/branch/jit-sign-descr/pypy/rlib/libffi.py
==============================================================================
--- pypy/branch/jit-sign-descr/pypy/rlib/libffi.py (original)
+++ pypy/branch/jit-sign-descr/pypy/rlib/libffi.py Mon Oct 18 19:38:52 2010
@@ -35,6 +35,9 @@
@staticmethod
@jit.purefunction
def getkind(ffi_type):
+ """Returns 'v' for void, 'f' for float, 'i' for signed integer,
+ and 'u' for unsigned integer.
+ """
if ffi_type is types.void:
return 'v'
elif ffi_type is types.double:
@@ -42,31 +45,31 @@
elif ffi_type is types.pointer:
return 'i'
elif ffi_type is types.uchar:
- return 'i'
+ return 'u'
elif ffi_type is types.uint8:
- return 'i'
+ return 'u'
elif ffi_type is types.schar:
return 'i'
elif ffi_type is types.sint8:
return 'i'
elif ffi_type is types.uint16:
- return 'i'
+ return 'u'
elif ffi_type is types.ushort:
- return 'i'
+ return 'u'
elif ffi_type is types.sint16:
return 'i'
elif ffi_type is types.sshort:
return 'i'
elif ffi_type is types.uint:
- return 'i'
+ return 'u'
elif ffi_type is types.uint32:
- return 'i'
+ return 'u'
elif ffi_type is types.sint:
return 'i'
elif ffi_type is types.sint32:
return 'i'
## elif ffi_type is types.uint64:
- ## return 'i'
+ ## return 'u'
## elif ffi_type is types.sint64:
## return 'i'
raise KeyError
From arigo at codespeak.net Mon Oct 18 19:39:42 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 18 Oct 2010 19:39:42 +0200 (CEST)
Subject: [pypy-svn] r78068 -
pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/test
Message-ID: <20101018173942.7002C282B9D@codespeak.net>
Author: arigo
Date: Mon Oct 18 19:39:41 2010
New Revision: 78068
Modified:
pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/test/test_ffisupport.py
Log:
Add a missing test for the previous checkin.
Modified: pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/test/test_ffisupport.py
==============================================================================
--- pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/test/test_ffisupport.py (original)
+++ pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/test/test_ffisupport.py Mon Oct 18 19:39:41 2010
@@ -12,6 +12,12 @@
descr = get_call_descr_dynamic([], types.sint8)
assert isinstance(descr, DynamicIntCallDescr)
assert descr.get_result_size(False) == 1
+ assert descr.is_result_signed() == True
+
+ descr = get_call_descr_dynamic([], types.uint8)
+ assert isinstance(descr, DynamicIntCallDescr)
+ assert descr.get_result_size(False) == 1
+ assert descr.is_result_signed() == False
descr = get_call_descr_dynamic([], types.float)
assert descr is None # single floats are not supported so far
From cfbolz at codespeak.net Mon Oct 18 19:47:53 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon, 18 Oct 2010 19:47:53 +0200 (CEST)
Subject: [pypy-svn] r78070 - pypy/extradoc/talk/pepm2011
Message-ID: <20101018174753.34D09282BD4@codespeak.net>
Author: cfbolz
Date: Mon Oct 18 19:47:50 2010
New Revision: 78070
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
add line numbers, try to gently rewrite things a bit
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Mon Oct 18 19:47:50 2010
@@ -575,26 +575,29 @@
bit peculiar in that it receives no static input arguments for the trace,
but it is only used to optimize operations within the trace.
-To optimize the trace, it is traversed from beginning to end.
-Whenever a \lstinline{new} operation is seen, the operation is removed and a \emph{static
-object}\footnote{Here ``static'' is meant in the sense of partial
-evaluation, \ie known at partial evaluation time, not in the sense of static
-allocation or static method.} is constructed and associated with the variable
-that would have stored
-the result of \lstinline{new}. The static object describes the shape of the
-original object, \eg where the values that would be stored in the fields of the
-allocated object come from, as well as the type of the object. Whenever the
-optimizer sees a \lstinline{set} that writes into such an object, that shape
-description is updated and the operation can be removed, which means that the
-operation was done at partial evaluation time. When the optimizer encounters a
-\lstinline{get} from such an object, the result is read from the shape
-description, and the operation is also removed. Equivalently, a
-\lstinline{guard_class} on a variable that has a shape description can be removed
-as well, because the shape description stores the type and thus the outcome of
-the type check the guard does is statically known.
+To optimize the trace, it is traversed from beginning to end. Every
+optimization in the input trace is either removed, or new operations are
+produced. Whenever a \lstinline{new} operation is seen, the operation it is
+removed optimistically and a \emph{static object}\footnote{Here ``static'' is
+meant in the sense of partial evaluation, \ie known at partial evaluation time,
+not in the sense of static allocation or static method.} is constructed and
+associated with the result variable. The static object describes the shape of
+the original object, \ie where the values that would be stored in the fields of
+the allocated object come from, as well as the type of the object.
+
+When a \lstinline{set} that writes into a static object is optimized, the
+corresponding shape description is updated and the operation is removed. This
+means that the operation was done at partial evaluation time. When the
+optimizer encounters a \lstinline{get} from a static object, the result is read
+from the shape description, and the operation is also removed. Equivalently, a
+\lstinline{guard_class} on a variable that has a shape description can be
+removed as well, because the shape description stores the type and thus the
+outcome of the type check the guard does is statically known. Operations that
+have dynamic (\ie all other) objects as arguments are just left untouched by
+the optimizer.
-In the example from Section~\ref{sub:example}, the following operations in the upper half
-of Figure~\ref{fig:unopt-trace} produce two
+In the example from Section~\ref{sub:example}, the following operations
+of Figure~\ref{fig:unopt-trace} (lines 10-17) produce two
static objects, and can be completely removed from the optimized trace:
\begin{lstlisting}[mathescape,xleftmargin=20pt]
@@ -604,15 +607,13 @@
set($p_{6}$, intval, -100)
\end{lstlisting}
-
-The static object associated with $p_{5}$ would know that it is a
+The static object associated with $p_{5}$ would store the knowledge that it is a
\lstinline{BoxedInteger} whose \lstinline{intval} field contains $i_{4}$; the
-one associated with $p_{6}$ would know that it is a \lstinline{BoxedInteger}
+one associated with $p_{6}$ would store that it is a \lstinline{BoxedInteger}
whose \lstinline{intval} field contains the constant -100.
-The subsequent operations in Figure~\ref{fig:unopt-trace},
- which use $p_{5}$ and $p_{6}$, could then be
-optimized using that knowledge:
+The subsequent operations (line 19-29) in Figure~\ref{fig:unopt-trace}, which
+use $p_{5}$ and $p_{6}$, can then be optimized using that knowledge:
\begin{lstlisting}[mathescape,xleftmargin=20pt]
guard_class($p_{5}$, BoxedInteger)
@@ -635,18 +636,20 @@
The rest of the trace from Figure~\ref{fig:unopt-trace} is optimized similarly.
+
So far we have only described what happens when static objects are used in guards and in
operations that read and write fields. When the static
object is used in any other operation, it cannot remain static. For example, when
a static object is stored in a globally accessible place, the object has to
-be allocated, as it might live longer than one iteration of the loop
-and because the partial evaluator looses track of it. This means that the static
+be allocated, as it might live longer than one iteration of the loop and as
+arbitrary \lstinline{set} operations could change it due to aliasing. This
+means that the static
object needs to be turned into a dynamic one, \ie lifted. This makes it
necessary to put operations into the residual code that allocate the
static object at runtime.
This is what happens at the end of the trace in Figure~\ref{fig:unopt-trace}, when the \lstinline{jump} operation
-is hit. The arguments of the jump are at this point static objects. Before the
+is optimized. The arguments of the jump are at this point static objects. Before the
jump is emitted, they are \emph{lifted}. This means that the optimizer produces code
that allocates a new object of the right type and sets its fields to the field
values that the static object has (if the static object points to other static
@@ -670,26 +673,26 @@
and \lstinline{guard_class} operations).
\begin{figure}
-\begin{lstlisting}[mathescape,numbers=left,escapechar=|]
+\begin{lstlisting}[mathescape,numbers=right,escapechar=|,numberstyle = \tiny,numbersep=0pt, numberblanklines=false]
# arguments to the trace: $p_{0}$, $p_{1}$ |\setcounter{lstnumber}{2}|
-guard_class($p_1$, BoxedInteger)
+guard_class($p_1$, BoxedInteger) |\setcounter{lstnumber}{4}|
$i_2$ = get($p_1$, intval)
-guard_class($p_0$, BoxedInteger)
+guard_class($p_0$, BoxedInteger) |\setcounter{lstnumber}{7}|
$i_3$ = get($p_0$, intval)
-$i_4$ = int_add($i_2$, $i_3$)
-$i_9$ = int_add($i_4$, -100)
-|\setcounter{lstnumber}{50}|
-guard_class($p_0$, BoxedInteger)
-$i_{12}$ = get($p_0$, intval)
-$i_{14}$ = int_add($i_{12}$, -1)
-
-$i_{17}$ = int_gt($i_{14}$, 0)
-guard_true($i_{17}$)
+$i_4$ = int_add($i_2$, $i_3$) |\setcounter{lstnumber}{25}|
+$i_9$ = int_add($i_4$, -100) |\setcounter{lstnumber}{35}|
-$p_{15}$ = new(BoxedInteger)
-set($p_{15}$, intval, $i_{14}$)
-$p_{10}$ = new(BoxedInteger)
-set($p_{10}$, intval, $i_9$)
+guard_class($p_0$, BoxedInteger) |\setcounter{lstnumber}{38}|
+$i_{12}$ = get($p_0$, intval) |\setcounter{lstnumber}{42}|
+$i_{14}$ = int_add($i_{12}$, -1) |\setcounter{lstnumber}{50}|
+
+$i_{17}$ = int_gt($i_{14}$, 0) |\setcounter{lstnumber}{53}|
+guard_true($i_{17}$) |\setcounter{lstnumber}{42}|
+
+$p_{15}$ = new(BoxedInteger) |\setcounter{lstnumber}{45}|
+set($p_{15}$, intval, $i_{14}$) |\setcounter{lstnumber}{26}|
+$p_{10}$ = new(BoxedInteger) |\setcounter{lstnumber}{28}|
+set($p_{10}$, intval, $i_9$) |\setcounter{lstnumber}{53}|
jump($p_{15}$, $p_{10}$)
\end{lstlisting}
@@ -699,7 +702,9 @@
The final optimized trace of the example can be seen in Figure~\ref{fig:step1}.
The optimized trace contains only two allocations, instead of the original five,
-and only three \lstinline{guard_class} operations, from the original seven.
+and only three \lstinline{guard_class} operations, from the original seven. The
+line numbers are the lines where the operations occurred in the original trace
+in Figure~\ref{fig:unopt-trace}.
\section{Formal Description of the Algorithm}
\label{sec:formal}
From afa at codespeak.net Mon Oct 18 21:30:02 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 18 Oct 2010 21:30:02 +0200 (CEST)
Subject: [pypy-svn] r78071 - pypy/branch/fast-forward/pypy/interpreter/test
Message-ID: <20101018193002.991F436C374@codespeak.net>
Author: afa
Date: Mon Oct 18 21:29:58 2010
New Revision: 78071
Modified:
pypy/branch/fast-forward/pypy/interpreter/test/test_raise.py
Log:
Fix tests about string exceptions
Modified: pypy/branch/fast-forward/pypy/interpreter/test/test_raise.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/test/test_raise.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/test/test_raise.py Mon Oct 18 21:29:58 2010
@@ -4,14 +4,8 @@
def test_arg_as_string(self):
def f():
raise "test"
+ raises(TypeError, f)
- import warnings
- warnings.simplefilter('error', DeprecationWarning)
- try:
- raises(DeprecationWarning, f)
- finally:
- warnings.simplefilter('default', DeprecationWarning)
-
def test_control_flow(self):
try:
raise Exception
@@ -48,17 +42,6 @@
assert e.args[0] == 1
assert e.args[1] == 2
- def test_stringexc(self):
- a = "hello world"
- try:
- raise a
- except a, e:
- assert e == None
- try:
- raise a, "message"
- except a, e:
- assert e == "message"
-
def test_builtin_exc(self):
try:
[][0]
From afa at codespeak.net Mon Oct 18 21:35:02 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 18 Oct 2010 21:35:02 +0200 (CEST)
Subject: [pypy-svn] r78072 - pypy/branch/fast-forward/lib-python
Message-ID: <20101018193502.C3E26282B90@codespeak.net>
Author: afa
Date: Mon Oct 18 21:34:57 2010
New Revision: 78072
Modified:
pypy/branch/fast-forward/lib-python/TODO
Log:
update TODO
(and thanks to Armin for the future_builtins module!)
Modified: pypy/branch/fast-forward/lib-python/TODO
==============================================================================
--- pypy/branch/fast-forward/lib-python/TODO (original)
+++ pypy/branch/fast-forward/lib-python/TODO Mon Oct 18 21:34:57 2010
@@ -26,8 +26,6 @@
- Fix fcntl.fcntl(fd, fcntl.F_NOTIFY, fcntl.DN_MULTISHOT) on 32bit platform.
-- Missing module future_builtins
-
Medium tasks
------------
@@ -47,7 +45,9 @@
- Finish _multiprocessing
-- Update the _ssl module (entry point is now _ssl.sslwrap)
+- many features are missing from the _ssl module
+
+- Enable -3 option to run test_py3kwarn.
More difficult issues
---------------------
From dan at codespeak.net Tue Oct 19 00:26:12 2010
From: dan at codespeak.net (dan at codespeak.net)
Date: Tue, 19 Oct 2010 00:26:12 +0200 (CEST)
Subject: [pypy-svn] r78073 -
pypy/branch/micronumpy-resync/pypy/module/micronumpy
Message-ID: <20101018222612.0ED24282B90@codespeak.net>
Author: dan
Date: Tue Oct 19 00:26:11 2010
New Revision: 78073
Removed:
pypy/branch/micronumpy-resync/pypy/module/micronumpy/
Log:
Taking the easy way out and just replacing this.
From dan at codespeak.net Tue Oct 19 07:19:28 2010
From: dan at codespeak.net (dan at codespeak.net)
Date: Tue, 19 Oct 2010 07:19:28 +0200 (CEST)
Subject: [pypy-svn] r78075 - in
pypy/branch/micronumpy-resync/pypy/module/micronumpy: . test
Message-ID: <20101019051928.73DA6282B90@codespeak.net>
Author: dan
Date: Tue Oct 19 07:19:25 2010
New Revision: 78075
Modified:
pypy/branch/micronumpy-resync/pypy/module/micronumpy/array.py
pypy/branch/micronumpy-resync/pypy/module/micronumpy/test/test_numpy.py
Log:
Added support code for array broadcasting, along with a small (unfinished) test.
Modified: pypy/branch/micronumpy-resync/pypy/module/micronumpy/array.py
==============================================================================
--- pypy/branch/micronumpy-resync/pypy/module/micronumpy/array.py (original)
+++ pypy/branch/micronumpy-resync/pypy/module/micronumpy/array.py Tue Oct 19 07:19:25 2010
@@ -32,6 +32,47 @@
else:
return 0
+def broadcast_shapes(a_shape, a_strides, b_shape, b_strides):
+ a_dim = len(a_shape)
+ b_dim = len(b_shape)
+
+ smaller_dim = a_dim if a_dim < b_dim else b_dim
+
+ if a_dim > b_dim:
+ result = a_shape
+ larger_dim = a_dim
+ smaller_dim = b_dim
+ shorter_strides = b_strides
+ else:
+ result = b_shape
+ larger_dim = b_dim
+ smaller_dim = a_dim
+ shorter_strides = a_strides
+
+ i_a = a_dim - 1
+ i_b = b_dim - 1
+ for i in range(smaller_dim):
+ assert i_a >= 0
+ a = a_shape[i_a]
+
+ assert i_b >= 0
+ b = b_shape[i_b]
+
+ if a == b or a == 1 or b == 1:
+ i_a -= 1
+ i_b -= 1
+ result[len(result) - 1 - i] = a if a > b else b
+ else:
+ raise ValueError("frames are not aligned") # FIXME: applevel?
+
+ if a_dim < b_dim:
+ i_b += 1
+ a_strides = [0] * i_b + a_strides
+ else:
+ i_a += 1
+ b_strides = [0] * i_a + b_strides
+ return result, a_strides, b_strides
+
def normalize_slice_starts(slice_starts, shape):
for i in range(len(slice_starts)):
if slice_starts[i] < 0:
Modified: pypy/branch/micronumpy-resync/pypy/module/micronumpy/test/test_numpy.py
==============================================================================
--- pypy/branch/micronumpy-resync/pypy/module/micronumpy/test/test_numpy.py (original)
+++ pypy/branch/micronumpy-resync/pypy/module/micronumpy/test/test_numpy.py Tue Oct 19 07:19:25 2010
@@ -425,6 +425,46 @@
for w_xs, typecode in data:
assert typecode == infer_from_iterable(space, w_xs).typecode
+class TestArraySupport(object):
+ def test_broadcast_shapes(self, space):
+ from pypy.module.micronumpy.array import broadcast_shapes
+ from pypy.module.micronumpy.array import stride_row as stride
+
+ def test(shape_a, shape_b, expected_result, expected_strides_a=None, expected_strides_b=None):
+ strides_a = [stride(shape_a, i) for i, x in enumerate(shape_a)]
+ strides_a_save = strides_a[:]
+
+ strides_b = [stride(shape_b, i) for i, x in enumerate(shape_b)]
+ strides_b_save = strides_b[:]
+
+ result_shape, result_strides_a, result_strides_b = broadcast_shapes(shape_a, strides_a, shape_b, strides_b)
+ assert result_shape == expected_result
+
+ if expected_strides_a:
+ assert result_strides_a == expected_strides_a
+ else:
+ assert result_strides_a == strides_a_save
+
+ if expected_strides_b:
+ assert result_strides_b == expected_strides_b
+ else:
+ assert result_strides_b == strides_b_save
+
+ shape_a = [256, 256, 3]
+ shape_b = [3]
+
+ test([256, 256, 3], [3],
+ expected_result=[256, 256, 3],
+ expected_strides_b=[0, 0, 1])
+
+ test([3], [256, 256, 3],
+ expected_result=[256, 256, 3],
+ expected_strides_a=[0, 0, 1])
+
+ test([8, 1, 6, 1], [7, 1, 5],
+ expected_result=[8, 7, 6, 5],
+ expected_strides_b=[0, 5, 5, 1])
+
class TestMicroArray(object):
@py.test.mark.xfail # XXX: return types changed
def test_index2strides(self, space):
From afa at codespeak.net Tue Oct 19 07:57:09 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 19 Oct 2010 07:57:09 +0200 (CEST)
Subject: [pypy-svn] r78076 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101019055709.B78F9282B90@codespeak.net>
Author: afa
Date: Tue Oct 19 07:57:07 2010
New Revision: 78076
Added:
pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
Log:
Progress in _io module
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Tue Oct 19 07:57:07 2010
@@ -3,10 +3,10 @@
TypeDef, interp_attrproperty_w, GetSetProperty)
from pypy.interpreter.gateway import interp2app, unwrap_spec, Arguments
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
-from pypy.interpreter.error import OperationError, wrap_oserror2
+from pypy.interpreter.error import OperationError, wrap_oserror, wrap_oserror2
from pypy.rlib.rarithmetic import r_longlong
from os import O_RDONLY, O_WRONLY, O_RDWR, O_CREAT, O_TRUNC
-import os
+import os, stat, errno
O_BINARY = getattr(os, "O_BINARY", 0)
O_APPEND = getattr(os, "O_APPEND", 0)
@@ -68,7 +68,16 @@
if append:
flags |= O_APPEND
- return readable, writable, flags
+ return readable, writable, append, flags
+
+def convert_size(space, w_size):
+ if space.is_w(w_size, space.w_None):
+ return -1
+ else:
+ return space.int_w(w_size)
+
+def verify_fd(fd):
+ return
class W_FileIO(W_RawIOBase):
def __init__(self, space):
@@ -102,21 +111,44 @@
raise OperationError(space.w_ValueError, space.wrap(
"negative file descriptor"))
- self.readable, self.writable, flags = decode_mode(space, mode)
+ self.readable, self.writable, append, flags = decode_mode(space, mode)
if fd >= 0:
+ verify_fd(fd)
+ try:
+ os.fstat(fd)
+ except OSError, e:
+ if e.errno == errno.EBADF:
+ raise wrap_oserror(space, e)
+ # else: pass
self.fd = fd
+ self.closefd = bool(closefd)
else:
+ if not closefd:
+ raise OperationError(space.w_ValueError, space.wrap(
+ "Cannot use closefd=False with file name"))
+ self.closefd = True
+
from pypy.module.posix.interp_posix import (
dispatch_filename, rposix)
try:
self.fd = dispatch_filename(rposix.open)(
space, w_name, flags, 0666)
except OSError, e:
- raise wrap_oserror2(space, e, w_name)
- self.closefd = bool(closefd)
+ raise wrap_oserror2(space, e, w_name,
+ exception_name='w_OSError')
+
+ self._dircheck(space, w_name)
self.w_name = w_name
+ if append:
+ # For consistent behaviour, we explicitly seek to the end of file
+ # (otherwise, it might be done only on the first write()).
+ try:
+ os.lseek(self.fd, 0, os.SEEK_END)
+ except OSError, e:
+ raise wrap_oserror(space, e)
+
def _mode(self):
if self.readable:
if self.writable:
@@ -134,8 +166,44 @@
raise OperationError(space.w_ValueError, space.wrap(
"I/O operation on closed file"))
+ def _close(self, space):
+ if self.fd < 0:
+ return
+ fd = self.fd
+ self.fd = -1
+
+ try:
+ verify_fd(fd)
+ os.close(fd)
+ except OSError, e:
+ raise wrap_oserror(space, e,
+ exception_name='w_IOError')
+
+ @unwrap_spec('self', ObjSpace)
+ def close_w(self, space):
+ if not self.closefd:
+ self.fd = -1
+ return
+ self._close(space)
+ W_RawIOBase.close_w(self, space)
+
+ def _dircheck(self, space, w_filename):
+ # On Unix, fopen will succeed for directories.
+ # In Python, there should be no file objects referring to
+ # directories, so we need a check.
+ if self.fd < 0:
+ return
+ try:
+ st = os.fstat(self.fd)
+ except OSError:
+ return
+ if stat.S_ISDIR(st.st_mode):
+ self._close()
+ raise wrap_oserror2(space, OSError(EISDIR), w_filename,
+ exception_name='w_IOError')
+
@unwrap_spec('self', ObjSpace, r_longlong, int)
- def seek_w(self, space, pos, whence):
+ def seek_w(self, space, pos, whence=0):
self._check_closed(space)
pos = os.lseek(self.fd, pos, whence)
return space.wrap(pos)
@@ -162,22 +230,55 @@
self.seekable = 1
return space.newbool(self.seekable == 1)
+ # ______________________________________________
+
@unwrap_spec('self', ObjSpace)
def fileno_w(self, space):
self._check_closed(space)
return space.wrap(self.fd)
+ @unwrap_spec('self', ObjSpace)
+ def isatty_w(self, space):
+ self._check_closed(space)
+ try:
+ res = os.isatty(self.fd)
+ except OSError, e:
+ raise wrap_oserror(space, e)
+ return space.wrap(res)
+
+ # ______________________________________________
+
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def read_w(self, space, w_size=None):
+ self._check_closed(space)
+ # XXX self._check_readable(space)
+ size = convert_size(space, w_size)
+
+ if size < 0:
+ return self.readall_w(self, space)
+
+ try:
+ s = os.read(self.fd, size)
+ except OSError, e:
+ raise wrap_oserror(space, e,
+ exception_name='w_OSError')
+
+ return space.wrap(s)
+
W_FileIO.typedef = TypeDef(
'FileIO', W_RawIOBase.typedef,
__new__ = interp2app(W_FileIO.descr_new.im_func),
__init__ = interp2app(W_FileIO.descr_init),
seek = interp2app(W_FileIO.seek_w),
+ read = interp2app(W_FileIO.read_w),
+ close = interp2app(W_FileIO.close_w),
readable = interp2app(W_FileIO.readable_w),
writable = interp2app(W_FileIO.writable_w),
seekable = interp2app(W_FileIO.seekable_w),
fileno = interp2app(W_FileIO.fileno_w),
+ isatty = interp2app(W_FileIO.isatty_w),
name = interp_attrproperty_w('w_name', cls=W_FileIO),
mode = GetSetProperty(W_FileIO.descr_get_mode),
)
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Tue Oct 19 07:57:07 2010
@@ -3,6 +3,13 @@
TypeDef, GetSetProperty, generic_new_descr)
from pypy.interpreter.gateway import interp2app, Arguments, unwrap_spec
from pypy.interpreter.error import OperationError
+from pypy.rlib.rstring import StringBuilder
+
+def convert_size(space, w_size):
+ if space.is_w(w_size, space.w_None):
+ return -1
+ else:
+ return space.int_w(w_size)
class W_IOBase(Wrappable):
def __init__(self, space):
@@ -94,6 +101,69 @@
space.w_IOError,
space.wrap("file or stream is not seekable"))
+ # ______________________________________________________________
+
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def readline_w(self, space, w_limit=None):
+ # For backwards compatibility, a (slowish) readline().
+ limit = convert_size(space, w_limit)
+
+ old_size = -1
+
+ has_peek = space.findattr(self, space.wrap("peek"))
+
+ builder = StringBuilder()
+
+ while limit < 0 or len(buffer) < limit:
+ nreadahead = 1
+
+ if has_peek:
+ w_readahead = space.call_method(self, "peek", space.wrap(1))
+ if not space.isinstance_w(w_readahead, space.w_str):
+ raise operationerrorfmt(
+ space.w_IOError,
+ "peek() should have returned a bytes object, "
+ "not '%s'", space.type(w_readahead).getname(space, '?'))
+ length = space.int_w(space.len(w_readahead))
+ if length > 0:
+ n = 0
+ buf = space.str_w(w_readahead)
+ if limit >= 0:
+ while True:
+ if n >= length or n >= limit:
+ break
+ n += 1
+ if buf[n-1] == '\n':
+ break
+ else:
+ while True:
+ if n >= length:
+ break
+ n += 1
+ if buf[n-1] == '\n':
+ break
+ nreadahead = n
+
+ w_read = space.call_method(self, "read", space.wrap(nreadahead))
+ print "AFA", nreadahead, w_read
+ if not space.isinstance_w(w_read, space.w_str):
+ raise operationerrorfmt(
+ space.w_IOError,
+ "peek() should have returned a bytes object, "
+ "not '%s'", space.type(w_read).getname(space, '?'))
+ read = space.str_w(w_read)
+ if not read:
+ break
+
+ print "AFA APPEND", repr(read)
+ builder.append(read)
+
+ if read[-1] == '\n':
+ break
+
+ print "AFA RETURN", repr(builder.build())
+ return space.wrap(builder.build())
+
W_IOBase.typedef = TypeDef(
'_IOBase',
__new__ = generic_new_descr(W_IOBase),
@@ -109,6 +179,8 @@
seekable = interp2app(W_IOBase.seekable_w),
_checkSeekable = interp2app(W_IOBase.check_seekable_w),
closed = GetSetProperty(W_IOBase.closed_get_w),
+
+ readline = interp2app(W_IOBase.readline_w),
)
class W_RawIOBase(W_IOBase):
Added: pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py Tue Oct 19 07:57:07 2010
@@ -0,0 +1,43 @@
+from pypy.conftest import gettestobjspace
+from pypy.tool.udir import udir
+import os
+
+class AppTestFileIO:
+ def setup_class(cls):
+ cls.space = gettestobjspace(usemodules=['_io'])
+ tmpfile = udir.join('tmpfile')
+ tmpfile.write("a\nb\nc", mode='wb')
+ cls.w_tmpfile = cls.space.wrap(str(tmpfile))
+ cls.w_tmpdir = cls.space.wrap(str(udir))
+ cls.w_posix = cls.space.appexec([], """():
+ import %s as m;
+ return m""" % os.name)
+
+ def test_constructor(self):
+ import _io
+ f = _io.FileIO(self.tmpfile, 'a')
+ assert f.name.endswith('tmpfile')
+ assert f.mode == 'wb'
+ f.close()
+
+ def test_open_fd(self):
+ import _io
+ os = self.posix
+ fd = os.open(self.tmpfile, os.O_RDONLY, 0666)
+ f = _io.FileIO(fd, "rb")
+ assert f.fileno() == fd
+ f.close()
+
+ def test_open_directory(self):
+ import _io
+ raises(OSError, _io.FileIO, self.tmpdir, "rb")
+
+ def test_readline(self):
+ import _io
+ f = _io.FileIO(self.tmpfile, 'rb')
+ f.seek(0)
+ assert f.readline() == 'a\n'
+ assert f.readline() == 'b\n'
+ assert f.readline() == 'c'
+ assert f.readline() == ''
+ f.close()
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Tue Oct 19 07:57:07 2010
@@ -1,5 +1,5 @@
from pypy.conftest import gettestobjspace
-import os
+from pypy.tool.udir import udir
class AppTestIoModule:
def setup_class(cls):
@@ -40,12 +40,8 @@
class AppTestOpen:
def setup_class(cls):
- from pypy.tool.udir import udir
tmpfile = udir.join('tmpfile').ensure()
cls.w_tmpfile = cls.space.wrap(str(tmpfile))
- cls.w_posix = cls.space.appexec([], """():
- import %s as m;
- return m""" % os.name)
def test_open(self):
import io
@@ -57,13 +53,5 @@
def test_open_writable(self):
import io
f = io.open(self.tmpfile, "w+b")
- f.seek(0)
f.close()
- def test_open_fd(self):
- import io
- os = self.posix
- fd = os.open(self.tmpfile, os.O_RDONLY, 0666)
- f = io.open(fd, "rb")
- assert f.fileno() == fd
- f.close()
From cfbolz at codespeak.net Tue Oct 19 11:40:44 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Tue, 19 Oct 2010 11:40:44 +0200 (CEST)
Subject: [pypy-svn] r78078 - pypy/extradoc/talk/pepm2011
Message-ID: <20101019094044.1C591282B90@codespeak.net>
Author: cfbolz
Date: Tue Oct 19 11:40:43 2010
New Revision: 78078
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
fixes
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Tue Oct 19 11:40:43 2010
@@ -576,7 +576,7 @@
but it is only used to optimize operations within the trace.
To optimize the trace, it is traversed from beginning to end. Every
-optimization in the input trace is either removed, or new operations are
+operation in the input trace is either removed, or new operations are
produced. Whenever a \lstinline{new} operation is seen, the operation it is
removed optimistically and a \emph{static object}\footnote{Here ``static'' is
meant in the sense of partial evaluation, \ie known at partial evaluation time,
@@ -593,7 +593,7 @@
\lstinline{guard_class} on a variable that has a shape description can be
removed as well, because the shape description stores the type and thus the
outcome of the type check the guard does is statically known. Operations that
-have dynamic (\ie all other) objects as arguments are just left untouched by
+have dynamic (\ie non-static) objects as arguments are just left untouched by
the optimizer.
In the example from Section~\ref{sub:example}, the following operations
From antocuni at codespeak.net Tue Oct 19 11:45:51 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Tue, 19 Oct 2010 11:45:51 +0200 (CEST)
Subject: [pypy-svn] r78079 - pypy/trunk/pypy/module/pypyjit/test
Message-ID: <20101019094551.A49E0282B90@codespeak.net>
Author: antocuni
Date: Tue Oct 19 11:45:50 2010
New Revision: 78079
Modified:
pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
Log:
skip the test if the _ffi module is not there
Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Tue Oct 19 11:45:50 2010
@@ -119,6 +119,8 @@
result = child_stdout.read()
child_stdout.close()
assert result
+ if result.strip().startswith('SKIP:'):
+ py.test.skip(result.strip())
assert result.splitlines()[-1].strip() == 'OK :-)'
self.parse_loops(logfilepath)
self.print_loops()
@@ -1143,7 +1145,12 @@
libm_name = get_libm_name(sys.platform)
out = self.run_source('''
def main():
- from _ffi import CDLL, types
+ try:
+ from _ffi import CDLL, types
+ except ImportError:
+ sys.stdout.write('SKIP: cannot import _ffi')
+ return 0
+
libm = CDLL('%(libm_name)s')
pow = libm.getfunc('pow', [types.double, types.double],
types.double)
From antocuni at codespeak.net Tue Oct 19 13:39:50 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Tue, 19 Oct 2010 13:39:50 +0200 (CEST)
Subject: [pypy-svn] r78080 - pypy/trunk/pypy/module/pypyjit/test
Message-ID: <20101019113950.A3751282B90@codespeak.net>
Author: antocuni
Date: Tue Oct 19 13:39:49 2010
New Revision: 78080
Modified:
pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
Log:
kill this test. It's just a special case of the previous one, and moreover it
does not really test anything, as it doesn't inspect the actual operations generated for the interesting bytecodes
Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Tue Oct 19 13:39:49 2010
@@ -789,24 +789,6 @@
return sa
'''%(e1, e2), n, ([], res))
- def test_boolrewrite_ptr_single(self):
- self.run_source('''
- class tst:
- pass
- def main():
- a = tst()
- b = tst()
- c = tst()
- sa = 0
- for i in range(1000):
- if a == b: sa += 1
- else: sa += 2
- if a != b: sa += 10000
- else: sa += 20000
- if i > 750: a = b
- return sa
- ''', 215, ([], 12481752))
-
def test_array_sum(self):
for tc, maxops in zip('bhilBHILfd', (38,) * 6 + (40, 40, 41, 38)):
res = 19352859
From afa at codespeak.net Tue Oct 19 14:46:32 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 19 Oct 2010 14:46:32 +0200 (CEST)
Subject: [pypy-svn] r78081 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101019124632.9D23F282B90@codespeak.net>
Author: afa
Date: Tue Oct 19 14:46:29 2010
New Revision: 78081
Modified:
pypy/branch/fast-forward/pypy/module/_io/__init__.py
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
Log:
Implement FileIO.readall(), io.UnsupportedOperation
+ translation fixes
Modified: pypy/branch/fast-forward/pypy/module/_io/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/__init__.py Tue Oct 19 14:46:29 2010
@@ -24,6 +24,14 @@
'TextIOWrapper': 'interp_io.W_TextIOWrapper',
'open': 'interp_io.open',
- 'UnsupportedOperation': 'space.w_None',
'IncrementalNewlineDecoder': 'space.w_None',
}
+
+ def init(self, space):
+ w_UnsupportedOperation = space.call_function(
+ space.w_type,
+ space.wrap('UnsupportedOperation'),
+ space.newtuple([space.w_ValueError, space.w_IOError]),
+ space.newdict())
+ space.setattr(self, space.wrap('UnsupportedOperation'),
+ w_UnsupportedOperation)
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Tue Oct 19 14:46:29 2010
@@ -5,6 +5,7 @@
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
from pypy.interpreter.error import OperationError, wrap_oserror, wrap_oserror2
from pypy.rlib.rarithmetic import r_longlong
+from pypy.rlib.rstring import StringBuilder
from os import O_RDONLY, O_WRONLY, O_RDWR, O_CREAT, O_TRUNC
import os, stat, errno
@@ -76,6 +77,33 @@
else:
return space.int_w(w_size)
+SMALLCHUNK = 8 * 1024
+BIGCHUNK = 512 * 1024
+
+def new_buffersize(fd, currentsize):
+ try:
+ st = os.fstat(fd)
+ end = st.st_size
+ pos = os.lseek(fd, 0, 1)
+ except OSError:
+ pass
+ else:
+ # Files claiming a size smaller than SMALLCHUNK may
+ # actually be streaming pseudo-files. In this case, we
+ # apply the more aggressive algorithm below.
+ if end >= SMALLCHUNK and end >= pos:
+ # Add 1 so if the file were to grow we'd notice.
+ return currentsize + end - pos + 1
+
+ if currentsize > SMALLCHUNK:
+ # Keep doubling until we reach BIGCHUNK;
+ # then keep adding BIGCHUNK.
+ if currentsize <= BIGCHUNK:
+ return currentsize + currentsize
+ else:
+ return currentsize + BIGCHUNK
+ return currentsize + SMALLCHUNK
+
def verify_fd(fd):
return
@@ -136,7 +164,7 @@
space, w_name, flags, 0666)
except OSError, e:
raise wrap_oserror2(space, e, w_name,
- exception_name='w_OSError')
+ exception_name='w_IOError')
self._dircheck(space, w_name)
self.w_name = w_name
@@ -198,9 +226,9 @@
except OSError:
return
if stat.S_ISDIR(st.st_mode):
- self._close()
- raise wrap_oserror2(space, OSError(EISDIR), w_filename,
- exception_name='w_IOError')
+ self._close(space)
+ raise wrap_oserror2(space, OSError(errno.EISDIR, "fstat"),
+ w_filename, exception_name='w_IOError')
@unwrap_spec('self', ObjSpace, r_longlong, int)
def seek_w(self, space, pos, whence=0):
@@ -255,16 +283,43 @@
size = convert_size(space, w_size)
if size < 0:
- return self.readall_w(self, space)
+ return self.readall_w(space)
try:
s = os.read(self.fd, size)
except OSError, e:
raise wrap_oserror(space, e,
- exception_name='w_OSError')
+ exception_name='w_IOError')
return space.wrap(s)
+ @unwrap_spec('self', ObjSpace)
+ def readall_w(self, space):
+ self._check_closed(space)
+ total = 0
+
+ builder = StringBuilder()
+ while True:
+ newsize = int(new_buffersize(self.fd, total))
+
+ try:
+ chunk = os.read(self.fd, newsize - total)
+ except OSError, e:
+ if e.errno == errno.EAGAIN:
+ if total > 0:
+ # return what we've got so far
+ break
+ return space.w_None
+ raise wrap_oserror(space, e,
+ exception_name='w_IOError')
+
+ if not chunk:
+ break
+ builder.append(chunk)
+ total += len(chunk)
+ return space.wrap(builder.build())
+
+
W_FileIO.typedef = TypeDef(
'FileIO', W_RawIOBase.typedef,
__new__ = interp2app(W_FileIO.descr_new.im_func),
@@ -272,6 +327,7 @@
seek = interp2app(W_FileIO.seek_w),
read = interp2app(W_FileIO.read_w),
+ readall = interp2app(W_FileIO.readall_w),
close = interp2app(W_FileIO.close_w),
readable = interp2app(W_FileIO.readable_w),
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Tue Oct 19 14:46:29 2010
@@ -2,7 +2,7 @@
from pypy.interpreter.typedef import (
TypeDef, GetSetProperty, generic_new_descr)
from pypy.interpreter.gateway import interp2app, Arguments, unwrap_spec
-from pypy.interpreter.error import OperationError
+from pypy.interpreter.error import OperationError, operationerrfmt
from pypy.rlib.rstring import StringBuilder
def convert_size(space, w_size):
@@ -113,14 +113,15 @@
has_peek = space.findattr(self, space.wrap("peek"))
builder = StringBuilder()
+ size = 0
- while limit < 0 or len(buffer) < limit:
+ while limit < 0 or size < limit:
nreadahead = 1
if has_peek:
w_readahead = space.call_method(self, "peek", space.wrap(1))
if not space.isinstance_w(w_readahead, space.w_str):
- raise operationerrorfmt(
+ raise operationerrfmt(
space.w_IOError,
"peek() should have returned a bytes object, "
"not '%s'", space.type(w_readahead).getname(space, '?'))
@@ -145,9 +146,8 @@
nreadahead = n
w_read = space.call_method(self, "read", space.wrap(nreadahead))
- print "AFA", nreadahead, w_read
if not space.isinstance_w(w_read, space.w_str):
- raise operationerrorfmt(
+ raise operationerrfmt(
space.w_IOError,
"peek() should have returned a bytes object, "
"not '%s'", space.type(w_read).getname(space, '?'))
@@ -155,13 +155,12 @@
if not read:
break
- print "AFA APPEND", repr(read)
+ size += len(read)
builder.append(read)
if read[-1] == '\n':
break
- print "AFA RETURN", repr(builder.build())
return space.wrap(builder.build())
W_IOBase.typedef = TypeDef(
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py Tue Oct 19 14:46:29 2010
@@ -30,7 +30,7 @@
def test_open_directory(self):
import _io
- raises(OSError, _io.FileIO, self.tmpdir, "rb")
+ raises(IOError, _io.FileIO, self.tmpdir, "rb")
def test_readline(self):
import _io
@@ -41,3 +41,8 @@
assert f.readline() == 'c'
assert f.readline() == ''
f.close()
+
+ def test_readall(self):
+ import _io
+ f = _io.FileIO(self.tmpfile, 'rb')
+ assert f.readall() == "a\nb\nc"
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Tue Oct 19 14:46:29 2010
@@ -38,6 +38,10 @@
assert list(MyFile()) == ["line1", "line2"]
+ def test_exception(self):
+ import _io
+ e = _io.UnsupportedOperation("seek")
+
class AppTestOpen:
def setup_class(cls):
tmpfile = udir.join('tmpfile').ensure()
From arigo at codespeak.net Tue Oct 19 15:41:37 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 19 Oct 2010 15:41:37 +0200 (CEST)
Subject: [pypy-svn] r78082 - pypy/trunk/pypy/jit/metainterp
Message-ID: <20101019134137.C9C55282B90@codespeak.net>
Author: arigo
Date: Tue Oct 19 15:41:35 2010
New Revision: 78082
Modified:
pypy/trunk/pypy/jit/metainterp/warmstate.py
Log:
Support an infinite threshold (set to -1).
Modified: pypy/trunk/pypy/jit/metainterp/warmstate.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/warmstate.py (original)
+++ pypy/trunk/pypy/jit/metainterp/warmstate.py Tue Oct 19 15:41:35 2010
@@ -176,6 +176,9 @@
meth(default_value)
def set_param_threshold(self, threshold):
+ if threshold < 0:
+ self.increment_threshold = 0 # never reach the THRESHOLD_LIMIT
+ return
if threshold < 2:
threshold = 2
self.increment_threshold = (self.THRESHOLD_LIMIT // threshold) + 1
From arigo at codespeak.net Tue Oct 19 15:42:35 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 19 Oct 2010 15:42:35 +0200 (CEST)
Subject: [pypy-svn] r78083 - pypy/build/bot2/pypybuildbot
Message-ID: <20101019134235.735E0282B90@codespeak.net>
Author: arigo
Date: Tue Oct 19 15:42:33 2010
New Revision: 78083
Modified:
pypy/build/bot2/pypybuildbot/builds.py
Log:
Use -1 as the threshold to mean "infinite".
The issue is that some rsre code likely overflows 32 bits when
counting how many times some things loop.
Modified: pypy/build/bot2/pypybuildbot/builds.py
==============================================================================
--- pypy/build/bot2/pypybuildbot/builds.py (original)
+++ pypy/build/bot2/pypybuildbot/builds.py Tue Oct 19 15:42:33 2010
@@ -225,7 +225,7 @@
'--revision', WithProperties('%(got_revision)s'),
'--upload', #'--force-host', 'bigdog',
'--branch', WithProperties('%(branch)s'),
- '--args', ',--jit threshold=1000000000'],
+ '--args', ',--jit threshold=-1'],
workdir='./benchmarks',
haltOnFailure=True))
resfile = os.path.expanduser("~/bench_results_nojit/%(got_revision)s.json")
From cfbolz at codespeak.net Tue Oct 19 16:43:49 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Tue, 19 Oct 2010 16:43:49 +0200 (CEST)
Subject: [pypy-svn] r78085 - in pypy/trunk/pypy/rlib/rsre: . test
Message-ID: <20101019144349.7F295282B90@codespeak.net>
Author: cfbolz
Date: Tue Oct 19 16:43:48 2010
New Revision: 78085
Modified:
pypy/trunk/pypy/rlib/rsre/rsre_core.py
pypy/trunk/pypy/rlib/rsre/test/test_zjit.py
Log:
(arigo, cfbolz, antocuni) this probably breaks translation, but makes sure that
the regex JIT actually optimize more
Modified: pypy/trunk/pypy/rlib/rsre/rsre_core.py
==============================================================================
--- pypy/trunk/pypy/rlib/rsre/rsre_core.py (original)
+++ pypy/trunk/pypy/rlib/rsre/rsre_core.py Tue Oct 19 16:43:48 2010
@@ -1,5 +1,5 @@
import sys
-from pypy.rlib.debug import check_nonneg
+from pypy.rlib.debug import check_nonneg, make_sure_not_modified
from pypy.rlib.unroll import unrolling_iterable
from pypy.rlib.rsre import rsre_char
from pypy.tool.sourcetools import func_with_new_name
@@ -469,6 +469,7 @@
while True:
op = ctx.pat(ppos)
ppos += 1
+ make_sure_not_modified(ctx.pattern)
#jit.jit_debug("sre_match", op, ppos, ptr)
#
Modified: pypy/trunk/pypy/rlib/rsre/test/test_zjit.py
==============================================================================
--- pypy/trunk/pypy/rlib/rsre/test/test_zjit.py (original)
+++ pypy/trunk/pypy/rlib/rsre/test/test_zjit.py Tue Oct 19 16:43:48 2010
@@ -9,7 +9,6 @@
def entrypoint1(r, string, repeat):
r = array2list(r)
string = hlstr(string)
- make_sure_not_modified(r)
match = None
for i in range(repeat):
match = rsre_core.match(r, string)
@@ -21,7 +20,6 @@
def entrypoint2(r, string, repeat):
r = array2list(r)
string = hlstr(string)
- make_sure_not_modified(r)
match = None
for i in range(repeat):
match = rsre_core.search(r, string)
@@ -118,3 +116,8 @@
r"Active\s+20\d\d-\d\d-\d\d\s+[[]\d+[]]([^[]+)",
"Active"*20 + "Active 2010-04-07 [42] Foobar baz boz blah[43]")
assert res == 6*20
+
+ def test_aorbstar(self):
+ res = self.meta_interp_match("(a|b)*a", "a" * 100)
+ assert res == 100
+ self.check_loops(guard_value=0)
From arigo at codespeak.net Tue Oct 19 16:56:06 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 19 Oct 2010 16:56:06 +0200 (CEST)
Subject: [pypy-svn] r78086 - pypy/trunk/pypy/module/_sre
Message-ID: <20101019145606.68F4F282B90@codespeak.net>
Author: arigo
Date: Tue Oct 19 16:56:04 2010
New Revision: 78086
Modified:
pypy/trunk/pypy/module/_sre/interp_sre.py
Log:
Fix translation by making sure that the annotator knows that
"code" is never modified.
Modified: pypy/trunk/pypy/module/_sre/interp_sre.py
==============================================================================
--- pypy/trunk/pypy/module/_sre/interp_sre.py (original)
+++ pypy/trunk/pypy/module/_sre/interp_sre.py Tue Oct 19 16:56:04 2010
@@ -290,10 +290,8 @@
def SRE_Pattern__new__(space, w_subtype, w_pattern, flags, w_code,
groups=0, w_groupindex=None, w_indexgroup=None):
n = space.int_w(space.len(w_code))
- code = [0] * n
- for i in range(n):
- x = space.uint_w(space.getitem(w_code, space.wrap(i)))
- code[i] = intmask(x)
+ code = [intmask(space.uint_w(space.getitem(w_code, space.wrap(i))))
+ for i in range(n)]
#
w_srepat = space.allocate_instance(W_SRE_Pattern, w_subtype)
srepat = space.interp_w(W_SRE_Pattern, w_srepat)
From arigo at codespeak.net Tue Oct 19 17:16:49 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 19 Oct 2010 17:16:49 +0200 (CEST)
Subject: [pypy-svn] r78089 - in pypy/trunk/pypy/module/pypyjit: . test
Message-ID: <20101019151649.7FF2A282BAD@codespeak.net>
Author: arigo
Date: Tue Oct 19 17:16:48 2010
New Revision: 78089
Modified:
pypy/trunk/pypy/module/pypyjit/__init__.py
pypy/trunk/pypy/module/pypyjit/interp_jit.py
pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
Log:
Attempt to fix test_blockstack_virtualizable.
The idea was to force a residual call, which is hard to do nowadays.
So I added a debugging API to the pypyjit module doing that.
Modified: pypy/trunk/pypy/module/pypyjit/__init__.py
==============================================================================
--- pypy/trunk/pypy/module/pypyjit/__init__.py (original)
+++ pypy/trunk/pypy/module/pypyjit/__init__.py Tue Oct 19 17:16:48 2010
@@ -6,6 +6,7 @@
interpleveldefs = {
'set_param': 'interp_jit.set_param',
+ 'residual_call': 'interp_jit.residual_call',
}
def setup_after_space_initialization(self):
Modified: pypy/trunk/pypy/module/pypyjit/interp_jit.py
==============================================================================
--- pypy/trunk/pypy/module/pypyjit/interp_jit.py (original)
+++ pypy/trunk/pypy/module/pypyjit/interp_jit.py Tue Oct 19 17:16:48 2010
@@ -5,10 +5,10 @@
from pypy.tool.pairtype import extendabletype
from pypy.rlib.rarithmetic import r_uint, intmask
-from pypy.rlib.jit import JitDriver, hint, we_are_jitted
+from pypy.rlib.jit import JitDriver, hint, we_are_jitted, dont_look_inside
import pypy.interpreter.pyopcode # for side-effects
from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.gateway import ObjSpace, Arguments
+from pypy.interpreter.gateway import ObjSpace, Arguments, W_Root
from pypy.interpreter.pycode import PyCode, CO_GENERATOR
from pypy.interpreter.pyframe import PyFrame
from pypy.interpreter.pyopcode import ExitFrame
@@ -131,3 +131,10 @@
"no JIT parameter '%s'", key)
set_param.unwrap_spec = [ObjSpace, Arguments]
+
+ at dont_look_inside
+def residual_call(space, w_callable, args):
+ '''For testing. Invokes callable(...), but without letting
+ the JIT follow the call.'''
+ return space.call_args(w_callable, args)
+residual_call.unwrap_spec = [ObjSpace, W_Root, Arguments]
Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Tue Oct 19 17:16:48 2010
@@ -560,17 +560,13 @@
def test_blockstack_virtualizable(self):
self.run_source('''
- def g(k):
- s = 0
- for i in range(k, k+2):
- s += 1
- return s
+ import pypyjit
def main():
i = 0
while i < 100:
try:
- g(i)
+ pypyjit.residual_call(len, [])
except:
pass
i += 1
@@ -1185,6 +1181,17 @@
cls.pypy_c = option.pypy_c
+def test_interface_residual_call():
+ space = gettestobjspace(usemodules=['pypyjit'])
+ space.appexec([], """():
+ import pypyjit
+ def f(*args, **kwds):
+ return (args, kwds)
+ res = pypyjit.residual_call(f, 4, x=6)
+ assert res == ((4,), {'x': 6})
+ """)
+
+
def has_info(pypy_c, option):
g = os.popen('"%s" --info' % pypy_c, 'r')
lines = g.readlines()
From arigo at codespeak.net Tue Oct 19 17:30:10 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 19 Oct 2010 17:30:10 +0200 (CEST)
Subject: [pypy-svn] r78090 - pypy/trunk/lib-python
Message-ID: <20101019153010.5DB92282B90@codespeak.net>
Author: arigo
Date: Tue Oct 19 17:30:08 2010
New Revision: 78090
Modified:
pypy/trunk/lib-python/conftest.py
Log:
Revert r76359, and skip ctypes tests again.
The comment (re-appearing because of the revert) explains why.
I'm not sure I understand why it was un-skipped but nothing
at all was done to fix the massive number of failures.
Modified: pypy/trunk/lib-python/conftest.py
==============================================================================
--- pypy/trunk/lib-python/conftest.py (original)
+++ pypy/trunk/lib-python/conftest.py Tue Oct 19 17:30:08 2010
@@ -464,7 +464,12 @@
RegrTest('test_coding.py'),
RegrTest('test_complex_args.py'),
RegrTest('test_contextlib.py', usemodules="thread"),
- RegrTest('test_ctypes.py', usemodules="_rawffi"),
+ # we skip test ctypes, since we adapted it massively in order
+ # to test what we want to support. There are real failures,
+ # but it's about missing features that we don't want to support
+ # now
+ RegrTest('test_ctypes.py', usemodules="_rawffi",
+ skip="missing features that we don't want to support now"),
RegrTest('test_defaultdict.py'),
RegrTest('test_email_renamed.py'),
RegrTest('test_exception_variations.py'),
From afa at codespeak.net Tue Oct 19 17:30:25 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 19 Oct 2010 17:30:25 +0200 (CEST)
Subject: [pypy-svn] r78091 - in pypy/branch/fast-forward/pypy/objspace/std:
. test
Message-ID: <20101019153025.3B9ED282B90@codespeak.net>
Author: afa
Date: Tue Oct 19 17:30:23 2010
New Revision: 78091
Modified:
pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py
pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py
Log:
Implement bytearray.extend
Modified: pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py Tue Oct 19 17:30:23 2010
@@ -350,5 +350,17 @@
String2Bytearray(space, w_b),
String2Bytearray(space, w_c)])
+# __________________________________________________________
+# Mutability methods
+
+def list_extend__Bytearray_Bytearray(space, w_bytearray, w_other):
+ w_bytearray.data += w_other.data
+ return space.w_None
+
+def list_extend__Bytearray_ANY(space, w_bytearray, w_other):
+ w_bytearray.data += [c for c in space.str_w(w_other)]
+ return space.w_None
+
+
from pypy.objspace.std import bytearraytype
register_all(vars(), bytearraytype)
Modified: pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py Tue Oct 19 17:30:23 2010
@@ -14,6 +14,8 @@
str_ljust, str_rjust, str_center, str_zfill,
str_join, str_split, str_rsplit, str_partition, str_rpartition,
str_splitlines)
+from pypy.objspace.std.listtype import (
+ list_extend)
def _getbytevalue(space, w_value):
if space.isinstance_w(w_value, space.w_str):
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py Tue Oct 19 17:30:23 2010
@@ -131,3 +131,9 @@
check(b.partition(b'ss'), (b'mi', b'ss', b'issippi'))
check(b.rpartition(b'ss'), (b'missi', b'ss', b'ippi'))
+
+ def test_extend(self):
+ b = bytearray('abc')
+ b.extend(bytearray('def'))
+ b.extend('ghi')
+ assert b == 'abcdefghi'
From antocuni at codespeak.net Tue Oct 19 17:47:00 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Tue, 19 Oct 2010 17:47:00 +0200 (CEST)
Subject: [pypy-svn] r78092 - pypy/branch/jitypes
Message-ID: <20101019154700.DB608282B90@codespeak.net>
Author: antocuni
Date: Tue Oct 19 17:46:59 2010
New Revision: 78092
Added:
pypy/branch/jitypes/ (props changed)
- copied from r78091, pypy/trunk/
Log:
a branch where to try to make ctypes more jit friendly by using _ffi
From antocuni at codespeak.net Tue Oct 19 17:48:54 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Tue, 19 Oct 2010 17:48:54 +0200 (CEST)
Subject: [pypy-svn] r78093 - pypy/branch/jitypes/pypy/config
Message-ID: <20101019154854.20858282B90@codespeak.net>
Author: antocuni
Date: Tue Oct 19 17:48:53 2010
New Revision: 78093
Modified:
pypy/branch/jitypes/pypy/config/pypyoption.py
Log:
enable _ffi by default on this branch
Modified: pypy/branch/jitypes/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/jitypes/pypy/config/pypyoption.py (original)
+++ pypy/branch/jitypes/pypy/config/pypyoption.py Tue Oct 19 17:48:53 2010
@@ -30,7 +30,7 @@
"rctime" , "select", "zipimport", "_lsprof",
"crypt", "signal", "_rawffi", "termios", "zlib",
"struct", "md5", "sha", "bz2", "_minimal_curses", "cStringIO",
- "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array"]
+ "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array", "_ffi"]
))
working_oo_modules = default_modules.copy()
From arigo at codespeak.net Tue Oct 19 18:18:28 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 19 Oct 2010 18:18:28 +0200 (CEST)
Subject: [pypy-svn] r78094 - in pypy/trunk/pypy/module/_rawffi: . test
Message-ID: <20101019161828.A3002282B90@codespeak.net>
Author: arigo
Date: Tue Oct 19 18:18:27 2010
New Revision: 78094
Modified:
pypy/trunk/pypy/module/_rawffi/structure.py
pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py
Log:
Support unions with explicit fields, like we support structs
with explicit fields.
Modified: pypy/trunk/pypy/module/_rawffi/structure.py
==============================================================================
--- pypy/trunk/pypy/module/_rawffi/structure.py (original)
+++ pypy/trunk/pypy/module/_rawffi/structure.py Tue Oct 19 18:18:27 2010
@@ -34,7 +34,7 @@
def round_up(size, alignment):
return (size + alignment - 1) & -alignment
-def size_alignment_pos(fields):
+def size_alignment_pos(fields, is_union=False):
size = 0
alignment = 1
pos = []
@@ -42,16 +42,20 @@
# fieldtype is a W_Array
fieldsize = fieldtype.size
fieldalignment = fieldtype.alignment
- size = round_up(size, fieldalignment)
alignment = max(alignment, fieldalignment)
- pos.append(size)
- size += intmask(fieldsize)
+ if is_union:
+ pos.append(0)
+ size = max(size, fieldsize)
+ else:
+ size = round_up(size, fieldalignment)
+ pos.append(size)
+ size += intmask(fieldsize)
size = round_up(size, alignment)
return size, alignment, pos
class W_Structure(W_DataShape):
- def __init__(self, space, fields, size, alignment):
+ def __init__(self, space, fields, size, alignment, is_union=False):
name_to_index = {}
if fields is not None:
for i in range(len(fields)):
@@ -60,7 +64,7 @@
raise operationerrfmt(space.w_ValueError,
"duplicate field name %s", name)
name_to_index[name] = i
- size, alignment, pos = size_alignment_pos(fields)
+ size, alignment, pos = size_alignment_pos(fields, is_union)
else: # opaque case
fields = []
pos = []
@@ -69,6 +73,7 @@
self.alignment = alignment
self.ll_positions = pos
self.name_to_index = name_to_index
+ self.is_union = is_union
def allocate(self, space, length, autofree=False):
# length is ignored!
@@ -133,15 +138,17 @@
-def descr_new_structure(space, w_type, w_shapeinfo):
+def descr_new_structure(space, w_type, w_shapeinfo, union=0):
+ is_union = bool(union)
if space.is_true(space.isinstance(w_shapeinfo, space.w_tuple)):
w_size, w_alignment = space.fixedview(w_shapeinfo, expected_length=2)
S = W_Structure(space, None, space.int_w(w_size),
- space.int_w(w_alignment))
+ space.int_w(w_alignment), is_union)
else:
fields = unpack_fields(space, w_shapeinfo)
- S = W_Structure(space, fields, 0, 0)
+ S = W_Structure(space, fields, 0, 0, is_union)
return space.wrap(S)
+descr_new_structure.unwrap_spec = [ObjSpace, W_Root, W_Root, int]
W_Structure.typedef = TypeDef(
'Structure',
Modified: pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py
==============================================================================
--- pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py (original)
+++ pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py Tue Oct 19 18:18:27 2010
@@ -945,14 +945,15 @@
assert a[4] == 't'
def test_union(self):
- skip("segfaulting")
import _rawffi
longsize = _rawffi.sizeof('l')
- S = _rawffi.Structure((longsize, longsize))
+ S = _rawffi.Structure([('x', 'h'), ('y', 'l')], union=True)
s = S(autofree=False)
+ s.x = 12345
lib = _rawffi.CDLL(self.lib_name)
f = lib.ptr('ret_un_func', [(S, 1)], (S, 1))
ret = f(s)
+ assert ret.y == 1234500, "ret.y == %d" % (ret.y,)
s.free()
class AppTestAutoFree:
From afa at codespeak.net Tue Oct 19 18:25:38 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 19 Oct 2010 18:25:38 +0200 (CEST)
Subject: [pypy-svn] r78095 - in pypy/branch/fast-forward/pypy/objspace/std:
. test
Message-ID: <20101019162538.986B6282B90@codespeak.net>
Author: afa
Date: Tue Oct 19 18:25:37 2010
New Revision: 78095
Modified:
pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py
Log:
Fix bytearray->str conversion
Modified: pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py Tue Oct 19 18:25:37 2010
@@ -104,7 +104,7 @@
# bytearray-to-string delegation
def delegate_Bytearray2String(space, w_bytearray):
- return W_StringObject(''.join(w_bytearray.data))
+ return str__Bytearray(space, w_bytearray)
def String2Bytearray(space, w_str):
data = [c for c in space.str_w(w_str)]
@@ -188,6 +188,9 @@
return space.wrap(buf.build())
+def str__Bytearray(space, w_bytearray):
+ return W_StringObject(''.join(w_bytearray.data))
+
def _convert_idx_params(space, w_self, w_start, w_stop):
start = slicetype._Eval_SliceIndex(space, w_start)
stop = slicetype._Eval_SliceIndex(space, w_stop)
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py Tue Oct 19 18:25:37 2010
@@ -27,6 +27,11 @@
assert repr(bytearray('test')) == "bytearray(b'test')"
assert repr(bytearray("d'oh")) == r"bytearray(b'd\'oh')"
+ def test_str(self):
+ assert str(bytearray()) == ""
+ assert str(bytearray('test')) == "test"
+ assert str(bytearray("d'oh")) == "d'oh"
+
def test_getitem(self):
b = bytearray('test')
assert b[0] == ord('t')
From arigo at codespeak.net Tue Oct 19 18:41:14 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 19 Oct 2010 18:41:14 +0200 (CEST)
Subject: [pypy-svn] r78096 - pypy/trunk/pypy/module/_rawffi
Message-ID: <20101019164114.83599282BD4@codespeak.net>
Author: arigo
Date: Tue Oct 19 18:41:12 2010
New Revision: 78096
Modified:
pypy/trunk/pypy/module/_rawffi/structure.py
Log:
We don't actually need to store this as a field.
Modified: pypy/trunk/pypy/module/_rawffi/structure.py
==============================================================================
--- pypy/trunk/pypy/module/_rawffi/structure.py (original)
+++ pypy/trunk/pypy/module/_rawffi/structure.py Tue Oct 19 18:41:12 2010
@@ -73,7 +73,6 @@
self.alignment = alignment
self.ll_positions = pos
self.name_to_index = name_to_index
- self.is_union = is_union
def allocate(self, space, length, autofree=False):
# length is ignored!
From afa at codespeak.net Tue Oct 19 19:23:16 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 19 Oct 2010 19:23:16 +0200 (CEST)
Subject: [pypy-svn] r78097 - in pypy/branch/fast-forward/pypy/objspace/std:
. test
Message-ID: <20101019172316.C6795282B90@codespeak.net>
Author: afa
Date: Tue Oct 19 19:23:15 2010
New Revision: 78097
Modified:
pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py
pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py
Log:
bytearray.delslice, bytearray.append
Modified: pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py Tue Oct 19 19:23:15 2010
@@ -356,6 +356,11 @@
# __________________________________________________________
# Mutability methods
+def list_append__Bytearray_ANY(space, w_bytearray, w_item):
+ from pypy.objspace.std.bytearraytype import getbytevalue
+ w_bytearray.data.append(getbytevalue(space, w_item))
+ return space.w_None
+
def list_extend__Bytearray_Bytearray(space, w_bytearray, w_other):
w_bytearray.data += w_other.data
return space.w_None
@@ -364,6 +369,12 @@
w_bytearray.data += [c for c in space.str_w(w_other)]
return space.w_None
+def delslice__Bytearray_ANY_ANY(space, w_bytearray, w_start, w_stop):
+ length = len(w_bytearray.data)
+ start, stop = normalize_simple_slice(space, length, w_start, w_stop)
+ if start == stop:
+ return
+ del w_bytearray.data[start:stop]
from pypy.objspace.std import bytearraytype
register_all(vars(), bytearraytype)
Modified: pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py Tue Oct 19 19:23:15 2010
@@ -15,9 +15,9 @@
str_join, str_split, str_rsplit, str_partition, str_rpartition,
str_splitlines)
from pypy.objspace.std.listtype import (
- list_extend)
+ list_append, list_extend)
-def _getbytevalue(space, w_value):
+def getbytevalue(space, w_value):
if space.isinstance_w(w_value, space.w_str):
string = space.str_w(w_value)
if len(string) != 1:
@@ -72,7 +72,7 @@
if not e.match(space, space.w_StopIteration):
raise
break
- value = _getbytevalue(space, w_item)
+ value = getbytevalue(space, w_item)
data.append(value)
return new_bytearray(space, w_bytearraytype, data)
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py Tue Oct 19 19:23:15 2010
@@ -137,8 +137,21 @@
check(b.partition(b'ss'), (b'mi', b'ss', b'issippi'))
check(b.rpartition(b'ss'), (b'missi', b'ss', b'ippi'))
+ def test_append(self):
+ b = bytearray('abc')
+ b.append('d')
+ b.append(ord('e'))
+ assert b == 'abcde'
+
def test_extend(self):
b = bytearray('abc')
b.extend(bytearray('def'))
b.extend('ghi')
assert b == 'abcdefghi'
+
+ def test_delslice(self):
+ b = bytearray('abcdefghi')
+ del b[5:8]
+ assert b == 'abcdei'
+ del b[:3]
+ assert b == 'dei'
From afa at codespeak.net Tue Oct 19 19:29:13 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 19 Oct 2010 19:29:13 +0200 (CEST)
Subject: [pypy-svn] r78098 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101019172913.494E8282B90@codespeak.net>
Author: afa
Date: Tue Oct 19 19:29:11 2010
New Revision: 78098
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
Log:
_io.FileIO.write()
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Tue Oct 19 19:29:11 2010
@@ -277,6 +277,20 @@
# ______________________________________________
@unwrap_spec('self', ObjSpace, W_Root)
+ def write_w(self, space, w_data):
+ self._check_closed(space)
+ # XXX self._check_writable(space)
+ data = space.str_w(w_data)
+
+ try:
+ n = os.write(self.fd, data)
+ except OSError, e:
+ raise wrap_oserror(space, e,
+ exception_name='w_IOError')
+
+ return space.wrap(n)
+
+ @unwrap_spec('self', ObjSpace, W_Root)
def read_w(self, space, w_size=None):
self._check_closed(space)
# XXX self._check_readable(space)
@@ -326,6 +340,7 @@
__init__ = interp2app(W_FileIO.descr_init),
seek = interp2app(W_FileIO.seek_w),
+ write = interp2app(W_FileIO.write_w),
read = interp2app(W_FileIO.read_w),
readall = interp2app(W_FileIO.readall_w),
close = interp2app(W_FileIO.close_w),
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py Tue Oct 19 19:29:11 2010
@@ -46,3 +46,15 @@
import _io
f = _io.FileIO(self.tmpfile, 'rb')
assert f.readall() == "a\nb\nc"
+ f.close()
+
+ def test_write(self):
+ import _io
+ filename = self.tmpfile + '_w'
+ f = _io.FileIO(filename, 'wb')
+ f.write("test")
+ # try without flushing
+ f2 = _io.FileIO(filename, 'rb')
+ assert f2.read() == "test"
+ f.close()
+ f2.close()
From afa at codespeak.net Tue Oct 19 23:52:16 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 19 Oct 2010 23:52:16 +0200 (CEST)
Subject: [pypy-svn] r78099 - pypy/branch/fast-forward/lib-python
Message-ID: <20101019215216.8AD5D282BDD@codespeak.net>
Author: afa
Date: Tue Oct 19 23:52:12 2010
New Revision: 78099
Modified:
pypy/branch/fast-forward/lib-python/conftest.py
Log:
Skip test_multiprocessing. It leaves subprocesses around
which swamps the buildbot.
Modified: pypy/branch/fast-forward/lib-python/conftest.py
==============================================================================
--- pypy/branch/fast-forward/lib-python/conftest.py (original)
+++ pypy/branch/fast-forward/lib-python/conftest.py Tue Oct 19 23:52:12 2010
@@ -318,7 +318,7 @@
RegrTest('test_multibytecodec.py', skip="unsupported codecs"),
RegrTest('test_multibytecodec_support.py', skip="not a test"),
RegrTest('test_multifile.py'),
- RegrTest('test_multiprocessing.py'),
+ RegrTest('test_multiprocessing.py', skip='FIXME leaves subprocesses'),
RegrTest('test_mutants.py', core="possibly"),
RegrTest('test_mutex.py'),
RegrTest('test_netrc.py'),
From afa at codespeak.net Wed Oct 20 00:07:41 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 00:07:41 +0200 (CEST)
Subject: [pypy-svn] r78100 - pypy/branch/fast-forward/pypy
Message-ID: <20101019220741.8B645282BE3@codespeak.net>
Author: afa
Date: Wed Oct 20 00:07:40 2010
New Revision: 78100
Modified:
pypy/branch/fast-forward/pypy/testrunner_cfg.py
Log:
See if I can kill remaining processes this way
Modified: pypy/branch/fast-forward/pypy/testrunner_cfg.py
==============================================================================
--- pypy/branch/fast-forward/pypy/testrunner_cfg.py (original)
+++ pypy/branch/fast-forward/pypy/testrunner_cfg.py Wed Oct 20 00:07:40 2010
@@ -1,6 +1,8 @@
# nightly test configuration for the paraller runner
import os
+os.system('killall pypy-c')
+
def collect_one_testdir(testdirs, reldir, tests):
if (reldir.startswith('translator/c/') or
reldir.startswith('translator/jvm/') or
From afa at codespeak.net Wed Oct 20 00:10:19 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 00:10:19 +0200 (CEST)
Subject: [pypy-svn] r78101 - pypy/branch/fast-forward/pypy
Message-ID: <20101019221019.EF413282BE3@codespeak.net>
Author: afa
Date: Wed Oct 20 00:10:18 2010
New Revision: 78101
Modified:
pypy/branch/fast-forward/pypy/testrunner_cfg.py
Log:
It worked!
Modified: pypy/branch/fast-forward/pypy/testrunner_cfg.py
==============================================================================
--- pypy/branch/fast-forward/pypy/testrunner_cfg.py (original)
+++ pypy/branch/fast-forward/pypy/testrunner_cfg.py Wed Oct 20 00:10:18 2010
@@ -1,8 +1,6 @@
# nightly test configuration for the paraller runner
import os
-os.system('killall pypy-c')
-
def collect_one_testdir(testdirs, reldir, tests):
if (reldir.startswith('translator/c/') or
reldir.startswith('translator/jvm/') or
From afa at codespeak.net Wed Oct 20 00:40:46 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 00:40:46 +0200 (CEST)
Subject: [pypy-svn] r78102 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101019224046.C7F2C282BE3@codespeak.net>
Author: afa
Date: Wed Oct 20 00:40:45 2010
New Revision: 78102
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
Log:
Properly wrap OSError in FileIO.seek()
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Wed Oct 20 00:40:45 2010
@@ -233,7 +233,11 @@
@unwrap_spec('self', ObjSpace, r_longlong, int)
def seek_w(self, space, pos, whence=0):
self._check_closed(space)
- pos = os.lseek(self.fd, pos, whence)
+ try:
+ pos = os.lseek(self.fd, pos, whence)
+ except OSError, e:
+ raise wrap_oserror(space, e,
+ exception_name='w_IOError')
return space.wrap(pos)
@unwrap_spec('self', ObjSpace)
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py Wed Oct 20 00:40:45 2010
@@ -35,7 +35,6 @@
def test_readline(self):
import _io
f = _io.FileIO(self.tmpfile, 'rb')
- f.seek(0)
assert f.readline() == 'a\n'
assert f.readline() == 'b\n'
assert f.readline() == 'c'
@@ -58,3 +57,10 @@
assert f2.read() == "test"
f.close()
f2.close()
+
+ def test_seek(self):
+ import _io
+ f = _io.FileIO(self.tmpfile, 'rb')
+ f.seek(0)
+ self.posix.close(f.fileno())
+ raises(IOError, f.seek, 0)
From hakanardo at codespeak.net Wed Oct 20 07:01:18 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Wed, 20 Oct 2010 07:01:18 +0200 (CEST)
Subject: [pypy-svn] r78103 -
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark
Message-ID: <20101020050118.D8F9E282B90@codespeak.net>
Author: hakanardo
Date: Wed Oct 20 07:01:16 2010
New Revision: 78103
Added:
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circular.c
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.c
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/looptst.py
- copied, changed from r77967, pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/loop.py
Removed:
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/loop.py
Modified:
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile
Log:
circular buffer benchmark
Modified: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile Wed Oct 20 07:01:16 2010
@@ -5,4 +5,18 @@
loop: loop.o
gcc -o $@ $^
sum: sumtst.o sum.o
- gcc -o $@ $^
\ No newline at end of file
+ gcc -o $@ $^
+circular: circulartst.o circular.o
+ gcc -o $@ $^
+
+run: run-intimg run-sum run-circular
+
+run-%: %
+ @echo $^
+ @echo -n ' C: '
+ @/usr/bin/time -f '%e s' ./$^
+ @echo -n ' pypy: '
+ @/usr/bin/time -f '%e s' ../../../translator/goal/pypy-c $^tst.py 2>&1 | tail -1
+ @echo -n ' cpython: '
+ @/usr/bin/time -f '%e s' python $^tst.py
+ @echo
Added: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circular.c
==============================================================================
--- (empty file)
+++ pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circular.c Wed Oct 20 07:01:16 2010
@@ -0,0 +1,10 @@
+#include
+
+double buf[65536];
+
+double circular(double *buf);
+
+int main() {
+ double sa = circular(buf);
+ //printf("%f\n", sa);
+}
Added: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.c
==============================================================================
--- (empty file)
+++ pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.c Wed Oct 20 07:01:16 2010
@@ -0,0 +1,14 @@
+
+double circular(double *buf) {
+ int i;
+ double sa;
+ for (i=0; i<65536; i++) buf[i] = i;
+
+ i = 10;
+ sa = 0;
+ while(i<200000000) {
+ sa += buf[(i-2)&65535] + buf[(i-1)&65535] + buf[i&65535] + buf[(i+1)&65535] + buf[(i+2)&65535];
+ i += 1;
+ }
+ return sa;
+}
Added: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py
==============================================================================
--- (empty file)
+++ pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py Wed Oct 20 07:01:16 2010
@@ -0,0 +1,19 @@
+from array import array
+class Circular(array):
+ def __new__(cls):
+ self = array.__new__(cls, 'd', range(65536))
+ return self
+ def __getitem__(self, i):
+ assert self.__len__() == 65536
+ return array.__getitem__(self, i & 65535)
+
+def main():
+ buf = Circular()
+ i = 10
+ sa = 0
+ while i < 200000000:
+ sa += buf[i-2] + buf[i-1] + buf[i] + buf[i+1] + buf[i+2]
+ i += 1
+ return sa
+
+print main()
Copied: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/looptst.py (from r77967, pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/loop.py)
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/loop.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/looptst.py Wed Oct 20 07:01:16 2010
@@ -5,3 +5,5 @@
s+=i
i+=1
return s
+
+print h()
From hakanardo at codespeak.net Wed Oct 20 07:37:57 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Wed, 20 Oct 2010 07:37:57 +0200 (CEST)
Subject: [pypy-svn] r78104 - pypy/branch/jit-unroll-loops/pypy/jit/tl
Message-ID: <20101020053757.321D8282B90@codespeak.net>
Author: hakanardo
Date: Wed Oct 20 07:37:55 2010
New Revision: 78104
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py
Log:
Optimize setfield_raw/getfield_raw in the same way as the _gc versions
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py Wed Oct 20 07:37:55 2010
@@ -70,7 +70,7 @@
buf = Circular()
i = 10
sa = 0
- while i < 20:
+ while i < 20000:
sa += buf[i-2] + buf[i-1] + buf[i] + buf[i+1] + buf[i+2]
i += 1
return sa
From hakanardo at codespeak.net Wed Oct 20 07:40:35 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Wed, 20 Oct 2010 07:40:35 +0200 (CEST)
Subject: [pypy-svn] r78105 - in pypy/branch/jit-unroll-loops/pypy:
jit/metainterp/optimizeopt jit/metainterp/test jit/tl
module/array/benchmark
Message-ID: <20101020054035.42948282B90@codespeak.net>
Author: hakanardo
Date: Wed Oct 20 07:40:33 2010
New Revision: 78105
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py
pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py
Log:
Optimize
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py Wed Oct 20 07:40:33 2010
@@ -107,6 +107,7 @@
return
opnum = op.getopnum()
if (opnum == rop.SETFIELD_GC or
+ opnum == rop.SETFIELD_RAW or
opnum == rop.SETARRAYITEM_GC or
opnum == rop.DEBUG_MERGE_POINT):
return
@@ -236,6 +237,10 @@
# remember the result of future reads of the field
self.cache_field_value(op.getdescr(), value, fieldvalue, write=True)
+ optimize_GETFIELD_RAW = optimize_GETFIELD_GC
+ optimize_SETFIELD_RAW = optimize_SETFIELD_GC
+ # FIXME: Do we need separate caches for raw and gc?
+
def optimize_GETARRAYITEM_GC(self, op):
value = self.getvalue(op.getarg(0))
indexvalue = self.getvalue(op.getarg(1))
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Wed Oct 20 07:40:33 2010
@@ -73,7 +73,7 @@
args.extend(self.getvalue(arg).get_forced_boxes())
newop.initarglist(args + inputargs[len(args):])
- #print 'P: ', newop
+ #print 'P: ', str(newop)
current = len(self.optimizer.newoperations)
self.emit_operation(newop)
@@ -81,7 +81,7 @@
current = max(current-1, 0)
for op in self.optimizer.newoperations[current:]:
- #print 'E: ', op, self.optimizer.newoperations.index(op)
+ #print 'E: ', str(op)
if op.is_guard():
descr = op.getdescr()
assert isinstance(descr, ResumeGuardDescr)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Wed Oct 20 07:40:33 2010
@@ -3571,6 +3571,25 @@
"""
self.optimize_loop(ops, 'Not, Not', expected)
+ def test_setgetfield_raw(self):
+ ops = """
+ [p4, p7, i30]
+ p16 = getfield_gc(p4, descr=valuedescr)
+ guard_value(p16, ConstPtr(myptr), descr=) []
+ i1 = getfield_raw(p7, descr=nextdescr)
+ i2 = int_add(i1, i30)
+ setfield_raw(p7, 7, descr=nextdescr)
+ setfield_raw(p7, i2, descr=nextdescr)
+ jump(p4, p7, i30)
+ """
+ expected = """
+ [p4, p7, i30, i2]
+ i33 = int_add(i2, i30)
+ setfield_raw(p7, i33, descr=nextdescr)
+ jump(p4, p7, i30, i33)
+ """
+ self.optimize_loop(ops, 'Not, Not, Not', expected)
+
def test_addsub_ovf(self):
ops = """
[i0]
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py Wed Oct 20 07:40:33 2010
@@ -34,6 +34,6 @@
option.view = True
warmspot.jittify_and_run(interp, graph, [], policy=policy,
listops=True, CPUClass=CPUClass,
- backendopt=True, inline=True,
+ backendopt=True, inline=False,
optimizer=OPTIMIZER_NO_PERFECTSPEC)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py Wed Oct 20 07:40:33 2010
@@ -57,24 +57,28 @@
## #img=array('h',(1,2,3,4))
## print f(3)
+
from array import array
class Circular(array):
def __new__(cls):
self = array.__new__(cls, 'i', range(16))
return self
def __getitem__(self, i):
- assert self.__len__() == 16
+ #assert self.__len__() == 16
return array.__getitem__(self, i & 15)
def main():
buf = Circular()
i = 10
sa = 0
- while i < 20000:
- sa += buf[i-2] + buf[i-1] + buf[i] + buf[i+1] + buf[i+2]
+ while i < 20:
+ #sa += buf[i-2] + buf[i-1] + buf[i] + buf[i+1] + buf[i+2]
+ sa += buf[i]
i += 1
return sa
+ import pypyjit
+ pypyjit.set_param(threshold=3, inlining=True)
print main()
except Exception, e:
Modified: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py Wed Oct 20 07:40:33 2010
@@ -11,7 +11,7 @@
buf = Circular()
i = 10
sa = 0
- while i < 200000000:
+ while i < 20000000:
sa += buf[i-2] + buf[i-1] + buf[i] + buf[i+1] + buf[i+2]
i += 1
return sa
From afa at codespeak.net Wed Oct 20 10:15:37 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 10:15:37 +0200 (CEST)
Subject: [pypy-svn] r78106 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101020081537.4902C282B90@codespeak.net>
Author: afa
Date: Wed Oct 20 10:15:35 2010
New Revision: 78106
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
Log:
FileIO.tell()
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Wed Oct 20 10:15:35 2010
@@ -241,6 +241,16 @@
return space.wrap(pos)
@unwrap_spec('self', ObjSpace)
+ def tell_w(self, space):
+ self._check_closed(space)
+ try:
+ pos = os.lseek(self.fd, 0, 1)
+ except OSError, e:
+ raise wrap_oserror(space, e,
+ exception_name='w_IOError')
+ return space.wrap(pos)
+
+ @unwrap_spec('self', ObjSpace)
def readable_w(self, space):
self._check_closed(space)
return space.wrap(self.readable)
@@ -344,6 +354,7 @@
__init__ = interp2app(W_FileIO.descr_init),
seek = interp2app(W_FileIO.seek_w),
+ tell = interp2app(W_FileIO.tell_w),
write = interp2app(W_FileIO.write_w),
read = interp2app(W_FileIO.read_w),
readall = interp2app(W_FileIO.readall_w),
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py Wed Oct 20 10:15:35 2010
@@ -64,3 +64,10 @@
f.seek(0)
self.posix.close(f.fileno())
raises(IOError, f.seek, 0)
+
+ def test_tell(self):
+ import _io
+ f = _io.FileIO(self.tmpfile, 'rb')
+ f.seek(3)
+ assert f.tell() == 3
+ f.close()
From antocuni at codespeak.net Wed Oct 20 10:22:53 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Wed, 20 Oct 2010 10:22:53 +0200 (CEST)
Subject: [pypy-svn] r78107 - pypy/trunk/pypy/rlib
Message-ID: <20101020082253.BA141282B90@codespeak.net>
Author: antocuni
Date: Wed Oct 20 10:22:52 2010
New Revision: 78107
Modified:
pypy/trunk/pypy/rlib/libffi.py
Log:
- reformat the if-elif chain to be more readable
- add the cases for slong and ulong: this fixes test_fficall on 64 bit, because without it thr 64 bit integers were never handled
Modified: pypy/trunk/pypy/rlib/libffi.py
==============================================================================
--- pypy/trunk/pypy/rlib/libffi.py (original)
+++ pypy/trunk/pypy/rlib/libffi.py Wed Oct 20 10:22:52 2010
@@ -35,43 +35,29 @@
@staticmethod
@jit.purefunction
def getkind(ffi_type):
- if ffi_type is types.void:
- return 'v'
- elif ffi_type is types.double:
- return 'f'
- elif ffi_type is types.pointer:
- return 'i'
- elif ffi_type is types.uchar:
- return 'i'
- elif ffi_type is types.uint8:
- return 'i'
- elif ffi_type is types.schar:
- return 'i'
- elif ffi_type is types.sint8:
- return 'i'
- elif ffi_type is types.uint16:
- return 'i'
- elif ffi_type is types.ushort:
- return 'i'
- elif ffi_type is types.sint16:
- return 'i'
- elif ffi_type is types.sshort:
- return 'i'
- elif ffi_type is types.uint:
- return 'i'
- elif ffi_type is types.uint32:
- return 'i'
- elif ffi_type is types.sint:
- return 'i'
- elif ffi_type is types.sint32:
- return 'i'
- ## elif ffi_type is types.uint64:
- ## return 'i'
- ## elif ffi_type is types.sint64:
- ## return 'i'
+ if ffi_type is types.void: return 'v'
+ elif ffi_type is types.double: return 'f'
+ elif ffi_type is types.pointer: return 'i'
+ #
+ elif ffi_type is types.schar: return 'i'
+ elif ffi_type is types.uchar: return 'i'
+ elif ffi_type is types.sshort: return 'i'
+ elif ffi_type is types.ushort: return 'i'
+ elif ffi_type is types.sint: return 'i'
+ elif ffi_type is types.uint: return 'i'
+ elif ffi_type is types.slong: return 'i'
+ elif ffi_type is types.ulong: return 'i'
+ #
+ elif ffi_type is types.sint8: return 'i'
+ elif ffi_type is types.uint8: return 'i'
+ elif ffi_type is types.sint16: return 'i'
+ elif ffi_type is types.uint16: return 'i'
+ elif ffi_type is types.sint32: return 'i'
+ elif ffi_type is types.uint32: return 'i'
+ ## elif ffi_type is types.sint64: return 'i'
+ ## elif ffi_type is types.uint64: return 'i'
raise KeyError
-
types._import()
@specialize.arg(0)
From antocuni at codespeak.net Wed Oct 20 10:44:42 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Wed, 20 Oct 2010 10:44:42 +0200 (CEST)
Subject: [pypy-svn] r78108 - pypy/branch/debug-test_import
Message-ID: <20101020084442.1D246282B90@codespeak.net>
Author: antocuni
Date: Wed Oct 20 10:44:40 2010
New Revision: 78108
Added:
pypy/branch/debug-test_import/ (props changed)
- copied from r78107, pypy/trunk/
Log:
(hopefully) very short lived branch to debug the systematic failure of
module/imp/test/test_import on 64 bit. Unfortunately, I cannot reproduce the
bug outside buildbot, so I'm going to hack the pylib to dump some useful info
in the log
From antocuni at codespeak.net Wed Oct 20 10:51:56 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Wed, 20 Oct 2010 10:51:56 +0200 (CEST)
Subject: [pypy-svn] r78109 - in pypy/branch/debug-test_import: py/_process
pypy
Message-ID: <20101020085156.16326282B90@codespeak.net>
Author: antocuni
Date: Wed Oct 20 10:51:55 2010
New Revision: 78109
Modified:
pypy/branch/debug-test_import/py/_process/cmdexec.py
pypy/branch/debug-test_import/pypy/testrunner_cfg.py
Log:
run only the relevant tests in this branch, and force cmdexec to print some useful information about the exception
Modified: pypy/branch/debug-test_import/py/_process/cmdexec.py
==============================================================================
--- pypy/branch/debug-test_import/py/_process/cmdexec.py (original)
+++ pypy/branch/debug-test_import/py/_process/cmdexec.py Wed Oct 20 10:51:55 2010
@@ -25,8 +25,15 @@
default_encoding = sys.getdefaultencoding() # jython may not have it
except AttributeError:
default_encoding = sys.stdout.encoding or 'UTF-8'
- out = unicode(out, process.stdout.encoding or default_encoding)
- err = unicode(err, process.stderr.encoding or default_encoding)
+ try:
+ out = unicode(out, process.stdout.encoding or default_encoding)
+ err = unicode(err, process.stderr.encoding or default_encoding)
+ except UnicodeDecodeError:
+ print 'encoding:', process.stdout.encoding or default_encoding
+ print 'stdout:', repr(stdout)
+ print 'stderr:', repr(stderr)
+ raise
+
status = process.poll()
if status:
raise ExecutionFailed(status, status, cmd, out, err)
Modified: pypy/branch/debug-test_import/pypy/testrunner_cfg.py
==============================================================================
--- pypy/branch/debug-test_import/pypy/testrunner_cfg.py (original)
+++ pypy/branch/debug-test_import/pypy/testrunner_cfg.py Wed Oct 20 10:51:55 2010
@@ -15,5 +15,8 @@
_cherrypick = os.getenv('PYPYCHERRYPICK', '')
+
+_cherrypick = 'module/imp'
+
if _cherrypick:
cherrypick = _cherrypick.split(':')
From antocuni at codespeak.net Wed Oct 20 10:54:21 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Wed, 20 Oct 2010 10:54:21 +0200 (CEST)
Subject: [pypy-svn] r78110 - pypy/branch/debug-test_import/py/_process
Message-ID: <20101020085421.B36C0282B90@codespeak.net>
Author: antocuni
Date: Wed Oct 20 10:54:20 2010
New Revision: 78110
Modified:
pypy/branch/debug-test_import/py/_process/cmdexec.py
Log:
bah, a typo :-(
Modified: pypy/branch/debug-test_import/py/_process/cmdexec.py
==============================================================================
--- pypy/branch/debug-test_import/py/_process/cmdexec.py (original)
+++ pypy/branch/debug-test_import/py/_process/cmdexec.py Wed Oct 20 10:54:20 2010
@@ -30,8 +30,8 @@
err = unicode(err, process.stderr.encoding or default_encoding)
except UnicodeDecodeError:
print 'encoding:', process.stdout.encoding or default_encoding
- print 'stdout:', repr(stdout)
- print 'stderr:', repr(stderr)
+ print 'stdout:', repr(out)
+ print 'stderr:', repr(err)
raise
status = process.poll()
From arigo at codespeak.net Wed Oct 20 11:37:31 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 11:37:31 +0200 (CEST)
Subject: [pypy-svn] r78111 - pypy/trunk/lib_pypy/_ctypes
Message-ID: <20101020093731.5C95A36E3FB@codespeak.net>
Author: arigo
Date: Wed Oct 20 11:37:29 2010
New Revision: 78111
Modified:
pypy/trunk/lib_pypy/_ctypes/structure.py
pypy/trunk/lib_pypy/_ctypes/union.py
Log:
My best fix to ctypes ever (which I hope is correct):
kill most of union.py, and just do it by importing from
structure.py.
Modified: pypy/trunk/lib_pypy/_ctypes/structure.py
==============================================================================
--- pypy/trunk/lib_pypy/_ctypes/structure.py (original)
+++ pypy/trunk/lib_pypy/_ctypes/structure.py Wed Oct 20 11:37:29 2010
@@ -7,7 +7,7 @@
def round_up(size, alignment):
return (size + alignment - 1) & -alignment
-def size_alignment_pos(fields):
+def size_alignment_pos(fields, is_union=False):
import ctypes
size = 0
alignment = 1
@@ -15,14 +15,19 @@
for fieldname, ctype in fields:
fieldsize = ctypes.sizeof(ctype)
fieldalignment = ctypes.alignment(ctype)
- size = round_up(size, fieldalignment)
alignment = max(alignment, fieldalignment)
- pos.append(size)
- size += fieldsize
+ if is_union:
+ pos.append(0)
+ size = max(size, fieldsize)
+ else:
+ size = round_up(size, fieldalignment)
+ pos.append(size)
+ size += fieldsize
size = round_up(size, alignment)
return size, alignment, pos
-def names_and_fields(_fields_, superclass, zero_offset=False, anon=None):
+def names_and_fields(_fields_, superclass, zero_offset=False, anon=None,
+ is_union=False):
if isinstance(_fields_, tuple):
_fields_ = list(_fields_)
for _, tp in _fields_:
@@ -36,7 +41,7 @@
rawfields = [(name, ctype._ffishape)
for name, ctype in all_fields]
if not zero_offset:
- _, _, pos = size_alignment_pos(all_fields)
+ _, _, pos = size_alignment_pos(all_fields, is_union)
else:
pos = [0] * len(all_fields)
fields = {}
@@ -73,8 +78,8 @@
# ________________________________________________________________
-def _set_shape(tp, rawfields):
- tp._ffistruct = _rawffi.Structure(rawfields)
+def _set_shape(tp, rawfields, is_union=False):
+ tp._ffistruct = _rawffi.Structure(rawfields, is_union)
tp._ffiargshape = tp._ffishape = (tp._ffistruct, 1)
tp._fficompositesize = tp._ffistruct.size
@@ -92,13 +97,14 @@
raise AttributeError("Structure or union cannot contain itself")
self._names, rawfields, self._fieldtypes = names_and_fields(
value, self.__bases__[0], False,
- self.__dict__.get('_anonymous_', None))
+ self.__dict__.get('_anonymous_', None), self._is_union)
_CDataMeta.__setattr__(self, '_fields_', value)
- _set_shape(self, rawfields)
+ _set_shape(self, rawfields, self._is_union)
return
_CDataMeta.__setattr__(self, name, value)
-class StructureMeta(_CDataMeta):
+class StructOrUnionMeta(_CDataMeta):
+
def __new__(self, name, cls, typedict):
res = type.__new__(self, name, cls, typedict)
if '_fields_' in typedict:
@@ -109,8 +115,8 @@
raise AttributeError("Anonymous field not found")
res._names, rawfields, res._fieldtypes = names_and_fields(
typedict['_fields_'], cls[0], False,
- typedict.get('_anonymous_', None))
- _set_shape(res, rawfields)
+ typedict.get('_anonymous_', None), self._is_union)
+ _set_shape(res, rawfields, self._is_union)
return res
@@ -150,8 +156,8 @@
res.__dict__['_index'] = -1
return res
-class Structure(_CData):
- __metaclass__ = StructureMeta
+class StructOrUnion(_CData):
+ __metaclass__ = StructOrUnionMeta
def __new__(cls, *args, **kwds):
if not hasattr(cls, '_ffistruct'):
@@ -213,3 +219,10 @@
def _get_buffer_value(self):
return self._buffer.buffer
+
+
+class StructureMeta(StructOrUnionMeta):
+ _is_union = False
+
+class Structure(StructOrUnion):
+ __metaclass__ = StructureMeta
Modified: pypy/trunk/lib_pypy/_ctypes/union.py
==============================================================================
--- pypy/trunk/lib_pypy/_ctypes/union.py (original)
+++ pypy/trunk/lib_pypy/_ctypes/union.py Wed Oct 20 11:37:29 2010
@@ -1,118 +1,7 @@
+from _ctypes import structure
+class UnionMeta(structure.StructOrUnionMeta):
+ _is_union = True
-import _rawffi
-from _ctypes.basics import _CData, _CDataMeta, store_reference, keepalive_key
-from _ctypes.basics import ensure_objects
-from _ctypes.structure import round_up, names_and_fields, struct_getattr,\
- struct_setattr
-
-
-def _set_shape(tp):
- size = tp._sizeofinstances()
- alignment = tp._alignmentofinstances()
- tp._ffiopaque = _rawffi.Structure((size, alignment)) # opaque
- tp._ffiargshape = tp._ffishape = (tp._ffiopaque, 1)
- tp._fficompositesize = tp._ffiopaque.size
- # we need to create an array of size one for each
- # of our elements
- tp._ffiarrays = {}
- for name, field in tp._fieldtypes.iteritems():
- tp._ffiarrays[name] = _rawffi.Array(field.ctype._ffishape)
-
-class UnionMeta(_CDataMeta):
- def __new__(self, name, cls, typedict):
- res = type.__new__(self, name, cls, typedict)
- if '_fields_' in typedict:
- res._names, rawfields, res._fieldtypes = names_and_fields(
- typedict['_fields_'], cls[0], True,
- typedict.get('_anonymous_', None))
- _set_shape(res)
-
- def __init__(self): # don't allow arguments by now
- if not hasattr(self, '_ffiarrays'):
- raise TypeError("Cannot instantiate union, has no type")
- # malloc size
- size = self.__class__._sizeofinstances()
- self.__dict__['_objects'] = {}
- self.__dict__['_buffer'] = self._ffiopaque(autofree=True)
- res.__init__ = __init__
- return res
-
- def _sizeofinstances(self):
- if not hasattr(self, '_size_'):
- self._size_ = max([field.size for field in
- self._fieldtypes.values()] + [0])
- return self._size_
-
- def _alignmentofinstances(self):
- from ctypes import alignment
- if not hasattr(self, '_alignment_'):
- self._alignment_ = max([alignment(field.ctype) for field in
- self._fieldtypes.values()] + [1])
- return self._alignment_
-
- __getattr__ = struct_getattr
-
- def __setattr__(self, name, value):
- if name == '_fields_':
- if self.__dict__.get('_fields_', None):
- raise AttributeError("_fields_ is final")
- if self in [v for k, v in value]:
- raise AttributeError("Union cannot contain itself")
- self._names, rawfields, self._fieldtypes = names_and_fields(
- value, self.__bases__[0], True,
- self.__dict__.get('_anonymous_', None))
- _CDataMeta.__setattr__(self, '_fields_', value)
- _set_shape(self)
- _CDataMeta.__setattr__(self, name, value)
-
- def _CData_output(self, resarray, base=None, index=-1):
- res = self.__new__(self)
- ffiopaque = self._ffiopaque.fromaddress(resarray.buffer)
- res.__dict__['_buffer'] = ffiopaque
- res.__dict__['_base'] = base
- res.__dict__['_index'] = index
- return res
-
- def _CData_retval(self, resbuffer):
- res = self.__new__(self)
- res.__dict__['_buffer'] = resbuffer
- res.__dict__['_base'] = None
- res.__dict__['_index'] = -1
- return res
-
-
-class Union(_CData):
+class Union(structure.StructOrUnion):
__metaclass__ = UnionMeta
-
- def __getattr__(self, name):
- try:
- field = self._fieldtypes[name]
- except KeyError:
- raise AttributeError(name)
- fieldtype = field.ctype
- val = self._ffiarrays[name].fromaddress(self._buffer.buffer, 1)
- offset = field.num
- return fieldtype._CData_output(val, self, offset)
-
- def __setattr__(self, name, value):
- try:
- field = self._fieldtypes[name]
- except KeyError:
- raise AttributeError(name)
- fieldtype = field.ctype
- cobj = fieldtype.from_param(value)
- if ensure_objects(cobj) is not None:
- key = keepalive_key(field.num)
- store_reference(self, key, cobj._objects)
- arg = cobj._get_buffer_value()
- if fieldtype._fficompositesize is not None:
- from ctypes import memmove
- dest = self._buffer.buffer
- memmove(dest, arg, fieldtype._fficompositesize)
- else:
- buf = self._ffiarrays[name].fromaddress(self._buffer.buffer, 1)
- buf[0] = arg
-
- def _get_buffer_value(self):
- return self._buffer.buffer
From antocuni at codespeak.net Wed Oct 20 11:40:41 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Wed, 20 Oct 2010 11:40:41 +0200 (CEST)
Subject: [pypy-svn] r78112 -
pypy/branch/debug-test_import/pypy/module/imp/test
Message-ID: <20101020094041.E234A282B90@codespeak.net>
Author: antocuni
Date: Wed Oct 20 11:40:40 2010
New Revision: 78112
Modified:
pypy/branch/debug-test_import/pypy/module/imp/test/test_import.py
Log:
make sure that LANG is unset when calling py.py through py.process.cmdexec. This is a workaround for pylib bug #130:
http://bitbucket.org/hpk42/py-trunk/issue/130/pyprocesscmdexec-fails-if-the-out-err-contains-non-ascii
Modified: pypy/branch/debug-test_import/pypy/module/imp/test/test_import.py
==============================================================================
--- pypy/branch/debug-test_import/pypy/module/imp/test/test_import.py (original)
+++ pypy/branch/debug-test_import/pypy/module/imp/test/test_import.py Wed Oct 20 11:40:40 2010
@@ -790,6 +790,7 @@
extrapath = udir.ensure("pythonpath", dir=1)
extrapath.join("urllib.py").write("print 42\n")
old = os.environ.get('PYTHONPATH', None)
+ oldlang = os.environ.pop('LANG', None)
try:
os.environ['PYTHONPATH'] = str(extrapath)
output = py.process.cmdexec('''"%s" "%s" -c "import urllib"''' %
@@ -798,6 +799,8 @@
finally:
if old:
os.environ['PYTHONPATH'] = old
+ if oldlang:
+ os.environ['LANG'] = oldlang
class AppTestImportHooks(object):
def test_meta_path(self):
From arigo at codespeak.net Wed Oct 20 11:44:05 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 11:44:05 +0200 (CEST)
Subject: [pypy-svn] r78113 - in pypy/trunk/pypy: . annotation
jit/backend/llsupport jit/backend/test jit/backend/x86
jit/backend/x86/test jit/codewriter jit/metainterp
jit/metainterp/test module/_socket/test module/cpyext/test
module/thread rlib rlib/test rpython rpython/lltypesystem
rpython/lltypesystem/test rpython/memory rpython/memory/gc
rpython/memory/gctransform rpython/memory/test rpython/test
tool tool/test translator/c/src translator/c/test
Message-ID: <20101020094405.B9D9D36E3FB@codespeak.net>
Author: arigo
Date: Wed Oct 20 11:44:02 2010
New Revision: 78113
Added:
pypy/trunk/pypy/tool/leakfinder.py
- copied unchanged from r78005, pypy/branch/leak-finder/pypy/tool/leakfinder.py
pypy/trunk/pypy/tool/test/test_leakfinder.py
- copied unchanged from r78005, pypy/branch/leak-finder/pypy/tool/test/test_leakfinder.py
pypy/trunk/pypy/translator/c/src/debug_alloc.h
- copied unchanged from r78005, pypy/branch/leak-finder/pypy/translator/c/src/debug_alloc.h
Modified:
pypy/trunk/pypy/annotation/builtin.py
pypy/trunk/pypy/conftest.py
pypy/trunk/pypy/jit/backend/llsupport/gc.py
pypy/trunk/pypy/jit/backend/llsupport/llmodel.py
pypy/trunk/pypy/jit/backend/test/runner_test.py
pypy/trunk/pypy/jit/backend/x86/assembler.py
pypy/trunk/pypy/jit/backend/x86/regalloc.py
pypy/trunk/pypy/jit/backend/x86/runner.py
pypy/trunk/pypy/jit/backend/x86/test/test_assembler.py
pypy/trunk/pypy/jit/codewriter/jtransform.py
pypy/trunk/pypy/jit/metainterp/test/test_basic.py
pypy/trunk/pypy/jit/metainterp/virtualref.py
pypy/trunk/pypy/module/_socket/test/test_sock_app.py
pypy/trunk/pypy/module/cpyext/test/test_cpyext.py
pypy/trunk/pypy/module/thread/ll_thread.py
pypy/trunk/pypy/rlib/rgc.py
pypy/trunk/pypy/rlib/test/test_rdynload.py
pypy/trunk/pypy/rlib/test/test_rzlib.py
pypy/trunk/pypy/rpython/llinterp.py
pypy/trunk/pypy/rpython/lltypesystem/llmemory.py
pypy/trunk/pypy/rpython/lltypesystem/lloperation.py
pypy/trunk/pypy/rpython/lltypesystem/lltype.py
pypy/trunk/pypy/rpython/lltypesystem/rclass.py
pypy/trunk/pypy/rpython/lltypesystem/rffi.py
pypy/trunk/pypy/rpython/lltypesystem/test/test_ll2ctypes.py
pypy/trunk/pypy/rpython/lltypesystem/test/test_llmemory.py
pypy/trunk/pypy/rpython/lltypesystem/test/test_lltype.py
pypy/trunk/pypy/rpython/lltypesystem/test/test_rffi.py
pypy/trunk/pypy/rpython/memory/gc/minimarkpage.py
pypy/trunk/pypy/rpython/memory/gctransform/asmgcroot.py
pypy/trunk/pypy/rpython/memory/gctransform/transform.py
pypy/trunk/pypy/rpython/memory/gcwrapper.py
pypy/trunk/pypy/rpython/memory/support.py
pypy/trunk/pypy/rpython/memory/test/test_gc.py
pypy/trunk/pypy/rpython/rbuiltin.py
pypy/trunk/pypy/rpython/test/test_llinterp.py
pypy/trunk/pypy/rpython/test/test_nongc.py
pypy/trunk/pypy/rpython/test/test_rptr.py
pypy/trunk/pypy/translator/c/src/g_include.h
pypy/trunk/pypy/translator/c/src/main.h
pypy/trunk/pypy/translator/c/test/test_genc.py
pypy/trunk/pypy/translator/c/test/test_lltyped.py
Log:
Merge branch/leak-finder. Adds a way for direct tests to detect
when a malloc(flavor='raw') has no corresponding free(). After
translation, in 'make debug' builds, records the raw-flavored
mallocs and frees and print at the end a summary of the still-
allocated objects. It is normal to have a few (but not a lot).
Note that you don't get tracking if you specify malloc(immortal=True)
or malloc(track_allocation=False).
Modified: pypy/trunk/pypy/annotation/builtin.py
==============================================================================
--- pypy/trunk/pypy/annotation/builtin.py (original)
+++ pypy/trunk/pypy/annotation/builtin.py Wed Oct 20 11:44:02 2010
@@ -423,7 +423,7 @@
from pypy.annotation.model import SomePtr
from pypy.rpython.lltypesystem import lltype
-def malloc(s_T, s_n=None, s_flavor=None, s_zero=None):
+def malloc(s_T, s_n=None, s_flavor=None, s_zero=None, s_track_allocation=None):
assert (s_n is None or s_n.knowntype == int
or issubclass(s_n.knowntype, pypy.rlib.rarithmetic.base_int))
assert s_T.is_constant()
@@ -438,13 +438,15 @@
r = SomePtr(lltype.typeOf(p))
else:
assert s_flavor.is_constant()
+ assert s_track_allocation is None or s_track_allocation.is_constant()
# not sure how to call malloc() for the example 'p' in the
# presence of s_extraargs
r = SomePtr(lltype.Ptr(s_T.const))
return r
-def free(s_p, s_flavor):
+def free(s_p, s_flavor, s_track_allocation=None):
assert s_flavor.is_constant()
+ assert s_track_allocation is None or s_track_allocation.is_constant()
# same problem as in malloc(): some flavors are not easy to
# malloc-by-example
#T = s_p.ll_ptrtype.TO
Modified: pypy/trunk/pypy/conftest.py
==============================================================================
--- pypy/trunk/pypy/conftest.py (original)
+++ pypy/trunk/pypy/conftest.py Wed Oct 20 11:44:02 2010
@@ -7,6 +7,7 @@
from inspect import isclass, getmro
from pypy.tool.udir import udir
from pypy.tool.autopath import pypydir
+from pypy.tool import leakfinder
# pytest settings
pytest_plugins = "resultlog",
@@ -354,7 +355,14 @@
def runtest(self):
try:
- super(IntTestFunction, self).runtest()
+ leakfinder.start_tracking_allocations()
+ try:
+ super(IntTestFunction, self).runtest()
+ finally:
+ if leakfinder.TRACK_ALLOCATIONS:
+ leaks = leakfinder.stop_tracking_allocations(False)
+ else:
+ leaks = None # stop_tracking_allocations() already called
except OperationError, e:
check_keyboard_interrupt(e)
raise
@@ -373,6 +381,8 @@
_pygame_imported = True
assert option.view, ("should not invoke Pygame "
"if conftest.option.view is False")
+ if leaks: # check for leaks, but only if the test passed so far
+ raise leakfinder.MallocMismatch(leaks)
class AppTestFunction(PyPyTestFunction):
def _prunetraceback(self, traceback):
Modified: pypy/trunk/pypy/jit/backend/llsupport/gc.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llsupport/gc.py (original)
+++ pypy/trunk/pypy/jit/backend/llsupport/gc.py Wed Oct 20 11:44:02 2010
@@ -158,7 +158,7 @@
# used to avoid too many duplications in the GCREF_LISTs.
self.hashtable = lltype.malloc(self.HASHTABLE,
self.HASHTABLE_SIZE+1,
- flavor='raw')
+ flavor='raw', track_allocation=False)
dummy = lltype.direct_ptradd(lltype.direct_arrayitems(self.hashtable),
self.HASHTABLE_SIZE)
dummy = llmemory.cast_ptr_to_adr(dummy)
@@ -252,14 +252,15 @@
def _enlarge_gcmap(self):
newlength = 250 + self._gcmap_maxlength * 2
- newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw')
+ newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw',
+ track_allocation=False)
oldgcmap = self._gcmap
for i in range(self._gcmap_curlength):
newgcmap[i] = oldgcmap[i]
self._gcmap = newgcmap
self._gcmap_maxlength = newlength
if oldgcmap:
- lltype.free(oldgcmap, flavor='raw')
+ lltype.free(oldgcmap, flavor='raw', track_allocation=False)
def get_basic_shape(self, is_64_bit=False):
# XXX: Should this code even really know about stack frame layout of
@@ -308,7 +309,8 @@
# them inside bigger arrays) and we never try to share them.
length = len(shape)
compressed = lltype.malloc(self.CALLSHAPE_ARRAY, length,
- flavor='raw')
+ flavor='raw',
+ track_allocation=False) # memory leak
for i in range(length):
compressed[length-1-i] = rffi.cast(rffi.UCHAR, shape[i])
return llmemory.cast_ptr_to_adr(compressed)
Modified: pypy/trunk/pypy/jit/backend/llsupport/llmodel.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llsupport/llmodel.py (original)
+++ pypy/trunk/pypy/jit/backend/llsupport/llmodel.py Wed Oct 20 11:44:02 2010
@@ -83,7 +83,8 @@
# read back by the machine code reading at the address given by
# pos_exception() and pos_exc_value().
_exception_emulator = lltype.malloc(rffi.CArray(lltype.Signed), 2,
- zero=True, flavor='raw')
+ zero=True, flavor='raw',
+ immortal=True)
self._exception_emulator = _exception_emulator
def _store_exception(lle):
Modified: pypy/trunk/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/trunk/pypy/jit/backend/test/runner_test.py Wed Oct 20 11:44:02 2010
@@ -1330,6 +1330,7 @@
descr=fd)
res = self.execute_operation(get_op, [s_box], 'int', descr=fd)
assert res.getint() == 32
+ lltype.free(s, flavor='raw')
def test_new_with_vtable(self):
cpu = self.cpu
Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/assembler.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/assembler.py Wed Oct 20 11:44:02 2010
@@ -249,7 +249,8 @@
def _build_float_constants(self):
# 44 bytes: 32 bytes for the data, and up to 12 bytes for alignment
- addr = lltype.malloc(rffi.CArray(lltype.Char), 44, flavor='raw')
+ addr = lltype.malloc(rffi.CArray(lltype.Char), 44, flavor='raw',
+ track_allocation=False)
if not we_are_translated():
self._keepalive_malloced_float_consts = addr
float_constants = rffi.cast(lltype.Signed, addr)
@@ -399,7 +400,8 @@
funcname = "" % len(self.loop_run_counters)
# invent the counter, so we don't get too confused
if self._debug:
- struct = lltype.malloc(DEBUG_COUNTER, flavor='raw')
+ struct = lltype.malloc(DEBUG_COUNTER, flavor='raw',
+ track_allocation=False) # known to leak
struct.i = 0
self.loop_run_counters.append((len(self.loop_run_counters), struct))
return funcname
Modified: pypy/trunk/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/regalloc.py Wed Oct 20 11:44:02 2010
@@ -70,8 +70,9 @@
def _get_new_array(self):
n = self.BASE_CONSTANT_SIZE
+ # known to leak
self.cur_array = lltype.malloc(rffi.CArray(lltype.Float), n,
- flavor='raw')
+ flavor='raw', track_allocation=False)
self.cur_array_free = n
_get_new_array._dont_inline_ = True
Modified: pypy/trunk/pypy/jit/backend/x86/runner.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/runner.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/runner.py Wed Oct 20 11:44:02 2010
@@ -113,7 +113,8 @@
return CPU386.cast_adr_to_int(adr)
all_null_registers = lltype.malloc(rffi.LONGP.TO, 24,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
def force(self, addr_of_force_index):
TP = rffi.CArrayPtr(lltype.Signed)
Modified: pypy/trunk/pypy/jit/backend/x86/test/test_assembler.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/test/test_assembler.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/test/test_assembler.py Wed Oct 20 11:44:02 2010
@@ -81,7 +81,8 @@
# also test rebuild_faillocs_from_descr(), which should not
# reproduce the holes at all
- bytecode = lltype.malloc(rffi.UCHARP.TO, len(mc.content), flavor='raw')
+ bytecode = lltype.malloc(rffi.UCHARP.TO, len(mc.content), flavor='raw',
+ immortal=True)
for i in range(len(mc.content)):
assert 0 <= mc.content[i] <= 255
bytecode[i] = rffi.cast(rffi.UCHAR, mc.content[i])
@@ -115,7 +116,8 @@
assert withfloats
value = random.random() - 0.5
# make sure it fits into 64 bits
- tmp = lltype.malloc(rffi.LONGP.TO, 2, flavor='raw')
+ tmp = lltype.malloc(rffi.LONGP.TO, 2, flavor='raw',
+ track_allocation=False)
rffi.cast(rffi.DOUBLEP, tmp)[0] = value
return rffi.cast(rffi.DOUBLEP, tmp)[0], tmp[0], tmp[1]
@@ -152,10 +154,12 @@
# prepare the expected target arrays, the descr_bytecode,
# the 'registers' and the 'stack' arrays according to 'content'
- xmmregisters = lltype.malloc(rffi.LONGP.TO, 16+ACTUAL_CPU.NUM_REGS+1, flavor='raw')
+ xmmregisters = lltype.malloc(rffi.LONGP.TO, 16+ACTUAL_CPU.NUM_REGS+1,
+ flavor='raw', immortal=True)
registers = rffi.ptradd(xmmregisters, 16)
stacklen = baseloc + 10
- stack = lltype.malloc(rffi.LONGP.TO, stacklen, flavor='raw')
+ stack = lltype.malloc(rffi.LONGP.TO, stacklen, flavor='raw',
+ immortal=True)
expected_ints = [0] * len(content)
expected_ptrs = [lltype.nullptr(llmemory.GCREF.TO)] * len(content)
expected_floats = [0.0] * len(content)
@@ -221,7 +225,7 @@
descr_bytecode.append(0x00)
descr_bytecode.append(0xCC) # end marker
descr_bytes = lltype.malloc(rffi.UCHARP.TO, len(descr_bytecode),
- flavor='raw')
+ flavor='raw', immortal=True)
for i in range(len(descr_bytecode)):
assert 0 <= descr_bytecode[i] <= 255
descr_bytes[i] = rffi.cast(rffi.UCHAR, descr_bytecode[i])
Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/trunk/pypy/jit/codewriter/jtransform.py Wed Oct 20 11:44:02 2010
@@ -433,7 +433,8 @@
op.result)
def rewrite_op_free(self, op):
- assert op.args[1].value == 'raw'
+ flags = op.args[1].value
+ assert flags['flavor'] == 'raw'
ARRAY = op.args[0].concretetype.TO
return self._do_builtin_call(op, 'raw_free', [op.args[0]],
extra = (ARRAY,), extrakey = ARRAY)
Modified: pypy/trunk/pypy/jit/metainterp/test/test_basic.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/test_basic.py (original)
+++ pypy/trunk/pypy/jit/metainterp/test/test_basic.py Wed Oct 20 11:44:02 2010
@@ -1780,7 +1780,7 @@
c = bool(p1)
d = not bool(p2)
return 1000*a + 100*b + 10*c + d
- prebuilt = [lltype.malloc(TP, flavor='raw')] * 2
+ prebuilt = [lltype.malloc(TP, flavor='raw', immortal=True)] * 2
expected = f(0, 1)
assert self.interp_operations(f, [0, 1]) == expected
Modified: pypy/trunk/pypy/jit/metainterp/virtualref.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/virtualref.py (original)
+++ pypy/trunk/pypy/jit/metainterp/virtualref.py Wed Oct 20 11:44:02 2010
@@ -16,7 +16,8 @@
('virtualref_index', lltype.Signed),
('forced', rclass.OBJECTPTR))
self.jit_virtual_ref_vtable = lltype.malloc(rclass.OBJECT_VTABLE,
- zero=True, flavor='raw')
+ zero=True, flavor='raw',
+ immortal=True)
self.jit_virtual_ref_vtable.name = rclass.alloc_array_name(
'jit_virtual_ref')
# build some constants
Modified: pypy/trunk/pypy/module/_socket/test/test_sock_app.py
==============================================================================
--- pypy/trunk/pypy/module/_socket/test/test_sock_app.py (original)
+++ pypy/trunk/pypy/module/_socket/test/test_sock_app.py Wed Oct 20 11:44:02 2010
@@ -254,6 +254,7 @@
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
fd = s.fileno()
w_obj = rsocket.make_address(c_addr, addrlen).as_object(fd, space)
+ lltype.free(c_addr_ll, flavor='raw')
assert space.is_true(space.eq(w_obj, space.newtuple([
space.wrap('lo'),
space.wrap(socket.ntohs(8)),
Modified: pypy/trunk/pypy/module/cpyext/test/test_cpyext.py
==============================================================================
--- pypy/trunk/pypy/module/cpyext/test/test_cpyext.py (original)
+++ pypy/trunk/pypy/module/cpyext/test/test_cpyext.py Wed Oct 20 11:44:02 2010
@@ -16,6 +16,7 @@
from pypy.module.cpyext.pyobject import Py_DecRef, InvalidPointerException
from pypy.translator.goal import autopath
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
@api.cpython_api([], api.PyObject)
def PyPy_Crash1(space):
@@ -78,7 +79,6 @@
self.frozen_refcounts[w_obj] = obj.c_ob_refcnt
#state.print_refcounts()
self.frozen_ll2callocations = set(ll2ctypes.ALLOCATED.values())
- lltype.start_tracking_allocations()
class LeakCheckingTest(object):
def check_and_print_leaks(self):
@@ -126,17 +126,8 @@
for w_obj in lost_objects_w:
print >>sys.stderr, "Lost object %r" % (w_obj, )
leaking = True
- for llvalue in set(ll2ctypes.ALLOCATED.values()) - self.frozen_ll2callocations:
- if getattr(llvalue, "_traceback", None): # this means that the allocation should be tracked
- leaking = True
- print >>sys.stderr, "Did not deallocate %r (ll2ctypes)" % (llvalue, )
- print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines())
- for llvalue in lltype.ALLOCATED.keys():
- leaking = True
- print >>sys.stderr, "Did not deallocate %r (llvalue)" % (llvalue, )
- print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines())
-
- lltype.stop_tracking_allocations()
+ # the actual low-level leak checking is done by pypy.tool.leakfinder,
+ # enabled automatically by pypy.conftest.
return leaking
class AppTestCpythonExtensionBase(LeakCheckingTest):
Modified: pypy/trunk/pypy/module/thread/ll_thread.py
==============================================================================
--- pypy/trunk/pypy/module/thread/ll_thread.py (original)
+++ pypy/trunk/pypy/module/thread/ll_thread.py Wed Oct 20 11:44:02 2010
@@ -111,7 +111,7 @@
c_thread_releaselock(self._lock)
def __del__(self):
- lltype.free(self._lock, flavor='raw')
+ lltype.free(self._lock, flavor='raw', track_allocation=False)
# ____________________________________________________________
#
@@ -128,10 +128,13 @@
null_ll_lock = lltype.nullptr(TLOCKP.TO)
def allocate_ll_lock():
- ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw')
+ # track_allocation=False here; be careful to lltype.free() it. The
+ # reason it is set to False is that we get it from all app-level
+ # lock objects, as well as from the GIL, which exists at shutdown.
+ ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw', track_allocation=False)
res = c_thread_lock_init(ll_lock)
if res == -1:
- lltype.free(ll_lock, flavor='raw')
+ lltype.free(ll_lock, flavor='raw', track_allocation=False)
raise error("out of resources")
return ll_lock
Modified: pypy/trunk/pypy/rlib/rgc.py
==============================================================================
--- pypy/trunk/pypy/rlib/rgc.py (original)
+++ pypy/trunk/pypy/rlib/rgc.py Wed Oct 20 11:44:02 2010
@@ -170,7 +170,14 @@
return hop.genop('gc_set_max_heap_size', [v_nbytes],
resulttype=lltype.Void)
-def can_move(p): # NB. must not be called with NULL pointers
+def can_move(p):
+ """Check if the GC object 'p' is at an address that can move.
+ Must not be called with None. With non-moving GCs, it is always False.
+ With some moving GCs like the SemiSpace GC, it is always True.
+ With other moving GCs like the MiniMark GC, it can be True for some
+ time, then False for the same object, when we are sure that it won't
+ move any more.
+ """
return True
class CanMoveEntry(ExtRegistryEntry):
Modified: pypy/trunk/pypy/rlib/test/test_rdynload.py
==============================================================================
--- pypy/trunk/pypy/rlib/test/test_rdynload.py (original)
+++ pypy/trunk/pypy/rlib/test/test_rdynload.py Wed Oct 20 11:44:02 2010
@@ -5,11 +5,18 @@
class TestDLOperations:
def test_dlopen(self):
- py.test.raises(DLOpenError, "dlopen(rffi.str2charp('xxxxxxxxxxxx'))")
- assert dlopen(rffi.str2charp(get_libc_name()))
+ s = rffi.str2charp('xxxxxxxxxxxx')
+ py.test.raises(DLOpenError, "dlopen(s)")
+ rffi.free_charp(s)
+ #
+ s = rffi.str2charp(get_libc_name())
+ assert dlopen(s)
+ rffi.free_charp(s)
def test_dlsym(self):
- lib = dlopen(rffi.str2charp(get_libc_name()))
+ s = rffi.str2charp(get_libc_name())
+ lib = dlopen(s)
+ rffi.free_charp(s)
handle = rffi.cast(lltype.Ptr(lltype.FuncType([lltype.Signed],
lltype.Signed)), dlsym(lib, 'abs'))
assert 1 == handle(1)
Modified: pypy/trunk/pypy/rlib/test/test_rzlib.py
==============================================================================
--- pypy/trunk/pypy/rlib/test/test_rzlib.py (original)
+++ pypy/trunk/pypy/rlib/test/test_rzlib.py Wed Oct 20 11:44:02 2010
@@ -189,6 +189,8 @@
assert unused3 == len('more_garbage')
assert data3 == ''
+ rzlib.deflateEnd(stream)
+
def test_decompress_max_length():
"""
@@ -205,6 +207,8 @@
assert finished2 is True
assert unused2 == 0
+ rzlib.deflateEnd(stream)
+
def test_cornercases():
"""
@@ -215,6 +219,7 @@
bytes += rzlib.compress(stream, "")
bytes += rzlib.compress(stream, "", rzlib.Z_FINISH)
assert zlib.decompress(bytes) == ""
+ rzlib.deflateEnd(stream)
stream = rzlib.inflateInit()
data, finished, unused = rzlib.decompress(stream, "")
@@ -228,3 +233,4 @@
assert finished is False
assert unused > 0
buf = buf[-unused:]
+ rzlib.deflateEnd(stream)
Modified: pypy/trunk/pypy/rpython/llinterp.py
==============================================================================
--- pypy/trunk/pypy/rpython/llinterp.py (original)
+++ pypy/trunk/pypy/rpython/llinterp.py Wed Oct 20 11:44:02 2010
@@ -48,8 +48,7 @@
current_interpreter = None
- def __init__(self, typer, tracing=True, exc_data_ptr=None,
- malloc_check=True):
+ def __init__(self, typer, tracing=True, exc_data_ptr=None):
self.bindings = {}
self.typer = typer
# 'heap' is module or object that provides malloc, etc for lltype ops
@@ -57,9 +56,7 @@
self.exc_data_ptr = exc_data_ptr
self.frame_stack = []
self.tracer = None
- self.malloc_check = malloc_check
self.frame_class = LLFrame
- self.mallocs = {}
if tracing:
self.tracer = Tracer()
@@ -163,24 +160,6 @@
return self.exc_data_ptr
return None
- def remember_malloc(self, ptr, llframe):
- # err....
- self.mallocs[ptr._obj] = llframe
-
- def remember_free(self, ptr):
- try:
- del self.mallocs[ptr._obj]
- except KeyError:
- self._rehash_mallocs()
- del self.mallocs[ptr._obj]
-
- def _rehash_mallocs(self):
- # rehashing is needed because some objects' hash may change
- # when being turned to
- items = self.mallocs.items()
- self.mallocs = {}
- self.mallocs.update(items)
-
def _store_exception(self, exc):
raise PleaseOverwriteStoreException("You just invoked ll2ctypes callback without overwriting _store_exception on llinterpreter")
@@ -726,13 +705,13 @@
def op_malloc(self, obj, flags):
flavor = flags['flavor']
zero = flags.get('zero', False)
+ track_allocation = flags.get('track_allocation', True)
if flavor == "stack":
result = self.heap.malloc(obj, zero=zero, flavor='raw')
self.alloca_objects.append(result)
return result
- ptr = self.heap.malloc(obj, zero=zero, flavor=flavor)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_malloc(ptr, self)
+ ptr = self.heap.malloc(obj, zero=zero, flavor=flavor,
+ track_allocation=track_allocation)
return ptr
def op_malloc_varsize(self, obj, flags, size):
@@ -741,8 +720,6 @@
assert flavor in ('gc', 'raw')
try:
ptr = self.heap.malloc(obj, size, zero=zero, flavor=flavor)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_malloc(ptr, self)
return ptr
except MemoryError:
self.make_llexception()
@@ -759,11 +736,10 @@
zero = flags.get('zero', False)
return self.heap.malloc_nonmovable(TYPE, size, zero=zero)
- def op_free(self, obj, flavor):
- assert isinstance(flavor, str)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_free(obj)
- self.heap.free(obj, flavor=flavor)
+ def op_free(self, obj, flags):
+ assert flags['flavor'] == 'raw'
+ track_allocation = flags.get('track_allocation', True)
+ self.heap.free(obj, flavor='raw', track_allocation=track_allocation)
def op_shrink_array(self, obj, smallersize):
return self.heap.shrink_array(obj, smallersize)
@@ -1037,6 +1013,13 @@
def op_stack_malloc(self, size): # mmh
raise NotImplementedError("backend only")
+ def op_track_alloc_start(self, addr):
+ # we don't do tracking at this level
+ checkadr(addr)
+
+ def op_track_alloc_stop(self, addr):
+ checkadr(addr)
+
# ____________________________________________________________
# Overflow-detecting variants
Modified: pypy/trunk/pypy/rpython/lltypesystem/llmemory.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/llmemory.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/llmemory.py Wed Oct 20 11:44:02 2010
@@ -105,11 +105,13 @@
if (isinstance(self.TYPE, lltype.ContainerType)
and self.TYPE._gckind == 'gc'):
assert self.repeat == 1
- p = lltype.malloc(self.TYPE, flavor='raw', zero=zero)
+ p = lltype.malloc(self.TYPE, flavor='raw', zero=zero,
+ track_allocation=False)
return cast_ptr_to_adr(p)
else:
T = lltype.FixedSizeArray(self.TYPE, self.repeat)
- p = lltype.malloc(T, flavor='raw', zero=zero)
+ p = lltype.malloc(T, flavor='raw', zero=zero,
+ track_allocation=False)
array_adr = cast_ptr_to_adr(p)
return array_adr + ArrayItemsOffset(T)
@@ -288,7 +290,8 @@
count = 0
p = lltype.malloc(parenttype or self.TYPE, count,
immortal = self.TYPE._gckind == 'raw',
- zero = zero)
+ zero = zero,
+ track_allocation = False)
return cast_ptr_to_adr(p)
def raw_memcopy(self, srcadr, dstadr):
Modified: pypy/trunk/pypy/rpython/lltypesystem/lloperation.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/lloperation.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/lloperation.py Wed Oct 20 11:44:02 2010
@@ -403,6 +403,8 @@
'raw_load': LLOp(sideeffects=False),
'raw_store': LLOp(),
'stack_malloc': LLOp(), # mmh
+ 'track_alloc_start': LLOp(),
+ 'track_alloc_stop': LLOp(),
'adr_add': LLOp(canfold=True),
'adr_sub': LLOp(canfold=True),
'adr_delta': LLOp(canfold=True),
Modified: pypy/trunk/pypy/rpython/lltypesystem/lltype.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/lltype.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/lltype.py Wed Oct 20 11:44:02 2010
@@ -1,7 +1,3 @@
-import StringIO
-import traceback
-import sys
-
import py
from pypy.rlib.rarithmetic import (r_int, r_uint, intmask, r_singlefloat,
r_ulonglong, r_longlong, base_int,
@@ -10,25 +6,13 @@
from pypy.tool.uid import Hashable
from pypy.tool.tls import tlsobject
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
from types import NoneType
from sys import maxint
import weakref
TLS = tlsobject()
-# Track allocations to detect memory leaks
-# Don't track 'gc' and immortal mallocs
-TRACK_ALLOCATIONS = False
-ALLOCATED = identity_dict()
-
-def start_tracking_allocations():
- global TRACK_ALLOCATIONS
- TRACK_ALLOCATIONS = True
- ALLOCATED.clear()
-
-def stop_tracking_allocations():
- global TRACK_ALLOCATIONS
- TRACK_ALLOCATIONS = False
class _uninitialized(object):
def __init__(self, TYPE):
@@ -1382,41 +1366,21 @@
__slots__ = ('_TYPE',
'_parent_type', '_parent_index', '_keepparent',
'_wrparent',
- '__weakref__', '_traceback',
- '__storage')
+ '__weakref__',
+ '_storage')
- def __init__(self, TYPE, track_allocation=None):
+ def __init__(self, TYPE):
self._wrparent = None
self._TYPE = TYPE
self._storage = True # means "use default storage", as opposed to:
# None - container was freed
# - using ctypes
# (see ll2ctypes.py)
- if track_allocation is not False and TRACK_ALLOCATIONS:
- self._traceback = self._get_traceback()
- ALLOCATED[self] = None
- else:
- self._traceback = None
-
- def _get_traceback(self):
- frame = sys._getframe().f_back.f_back.f_back.f_back
- sio = StringIO.StringIO()
- traceback.print_stack(frame, file=sio)
- return sio.getvalue()
def _free(self):
self._check() # no double-frees
self._storage = None
- def _storage_get(self):
- return self.__storage
-
- def _storage_set(self, value):
- self.__storage = value
- if value is not True and self in ALLOCATED:
- del ALLOCATED[self]
- _storage = property(_storage_get, _storage_set)
-
def _was_freed(self):
if self._storage is None:
return True
@@ -1495,12 +1459,12 @@
__slots__ = ('_hash_cache_', '_compilation_info')
- def __new__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None, track_allocation=None):
+ def __new__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None):
my_variety = _struct_variety(TYPE._names)
return object.__new__(my_variety)
- def __init__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None, track_allocation=None):
- _parentable.__init__(self, TYPE, track_allocation)
+ def __init__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None):
+ _parentable.__init__(self, TYPE)
if n is not None and TYPE._arrayfld is None:
raise TypeError("%r is not variable-sized" % (TYPE,))
if n is None and TYPE._arrayfld is not None:
@@ -1508,8 +1472,7 @@
first, FIRSTTYPE = TYPE._first_struct()
for fld, typ in TYPE._flds.items():
if fld == TYPE._arrayfld:
- value = _array(typ, n, initialization=initialization, parent=self, parentindex=fld,
- track_allocation=track_allocation)
+ value = _array(typ, n, initialization=initialization, parent=self, parentindex=fld)
else:
value = typ._allocate(initialization=initialization, parent=self, parentindex=fld)
setattr(self, fld, value)
@@ -1570,12 +1533,12 @@
__slots__ = ('items',)
- def __init__(self, TYPE, n, initialization=None, parent=None, parentindex=None, track_allocation=None):
+ def __init__(self, TYPE, n, initialization=None, parent=None, parentindex=None):
if not isinstance(n, int):
raise TypeError, "array length must be an int"
if n < 0:
raise ValueError, "negative array length"
- _parentable.__init__(self, TYPE, track_allocation)
+ _parentable.__init__(self, TYPE)
try:
myrange = range(n)
except OverflowError:
@@ -1642,7 +1605,7 @@
_cache = weakref.WeakKeyDictionary() # parentarray -> {subarrays}
def __init__(self, TYPE, parent, baseoffset_or_fieldname):
- _parentable.__init__(self, TYPE, track_allocation=False)
+ _parentable.__init__(self, TYPE)
self._setparentstructure(parent, baseoffset_or_fieldname)
# Keep the parent array alive, we share the same allocation.
# Don't do it if we are inside a GC object, though -- it's someone
@@ -1650,6 +1613,13 @@
if typeOf(top_container(parent))._gckind == 'raw':
self._keepparent = parent
+ def __str__(self):
+ parent = self._wrparent()
+ if parent is None:
+ return '_subarray at %s in already freed' % (self._parent_index,)
+ return '_subarray at %r in %s' % (self._parent_index,
+ parent._TYPE)
+
def __repr__(self):
parent = self._wrparent()
if parent is None:
@@ -1863,8 +1833,9 @@
return id(self.value)
-def malloc(T, n=None, flavor='gc', immortal=False, zero=False):
- assert flavor != 'cpy'
+def malloc(T, n=None, flavor='gc', immortal=False, zero=False,
+ track_allocation=True):
+ assert flavor in ('gc', 'raw')
if zero or immortal:
initialization = 'example'
elif flavor == 'raw':
@@ -1872,9 +1843,9 @@
else:
initialization = 'malloc'
if isinstance(T, Struct):
- o = _struct(T, n, initialization=initialization, track_allocation=flavor == "raw" and not immortal)
+ o = _struct(T, n, initialization=initialization)
elif isinstance(T, Array):
- o = _array(T, n, initialization=initialization, track_allocation=flavor == "raw" and not immortal)
+ o = _array(T, n, initialization=initialization)
elif isinstance(T, OpaqueType):
assert n is None
o = _opaque(T, initialization=initialization)
@@ -1882,15 +1853,19 @@
raise TypeError, "malloc for Structs and Arrays only"
if T._gckind != 'gc' and not immortal and flavor.startswith('gc'):
raise TypeError, "gc flavor malloc of a non-GC non-immortal structure"
+ if flavor == "raw" and not immortal and track_allocation:
+ leakfinder.remember_malloc(o, framedepth=2)
solid = immortal or not flavor.startswith('gc') # immortal or non-gc case
return _ptr(Ptr(T), o, solid)
-def free(p, flavor):
+def free(p, flavor, track_allocation=True):
if flavor.startswith('gc'):
raise TypeError, "gc flavor free"
T = typeOf(p)
if not isinstance(T, Ptr) or p._togckind() != 'raw':
raise TypeError, "free(): only for pointers to non-gc containers"
+ if track_allocation:
+ leakfinder.remember_free(p._obj0)
p._obj0._free()
def functionptr(TYPE, name, **attrs):
Modified: pypy/trunk/pypy/rpython/lltypesystem/rclass.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/rclass.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/rclass.py Wed Oct 20 11:44:02 2010
@@ -420,7 +420,7 @@
return cast_pointer(self.lowleveltype, result)
def create_instance(self):
- return malloc(self.object_type, flavor=self.gcflavor)
+ return malloc(self.object_type, flavor=self.gcflavor, immortal=True)
def initialize_prebuilt_data(self, value, classdef, result):
if self.classdef is not None:
Modified: pypy/trunk/pypy/rpython/lltypesystem/rffi.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/rffi.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/rffi.py Wed Oct 20 11:44:02 2010
@@ -644,10 +644,18 @@
"""
Either free a non-moving buffer or keep the original storage alive.
"""
- if rgc.can_move(data):
+ # We cannot rely on rgc.can_move(data) here, because its result
+ # might have changed since get_nonmovingbuffer(). Instead we check
+ # if 'buf' points inside 'data'. This is only possible if we
+ # followed the 2nd case in get_nonmovingbuffer(); in the first case,
+ # 'buf' points to its own raw-malloced memory.
+ data = llstrtype(data)
+ data_start = cast_ptr_to_adr(data) + \
+ offsetof(STRTYPE, 'chars') + itemoffsetof(STRTYPE.chars, 0)
+ followed_2nd_path = (buf == cast(TYPEP, data_start))
+ keepalive_until_here(data)
+ if not followed_2nd_path:
lltype.free(buf, flavor='raw')
- else:
- keepalive_until_here(data)
# int -> (char*, str)
def alloc_buffer(count):
Modified: pypy/trunk/pypy/rpython/lltypesystem/test/test_ll2ctypes.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/test/test_ll2ctypes.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/test/test_ll2ctypes.py Wed Oct 20 11:44:02 2010
@@ -765,6 +765,7 @@
assert abs(float(b[1]) - 1.1) < 1E-6
assert isinstance(b[2], rffi.r_singlefloat)
assert abs(float(b[2]) - 2.2) < 1E-6
+ lltype.free(a, flavor='raw')
def test_different_signatures(self):
if sys.platform=='win32':
@@ -879,6 +880,7 @@
qsort(rffi.cast(rffi.VOIDP, a), 5, rffi.sizeof(rffi.INT), compare)
for i in range(5):
assert a[i] == i + 1
+ lltype.free(a, flavor='raw')
def test_array_type_bug(self):
A = lltype.Array(lltype.Signed)
Modified: pypy/trunk/pypy/rpython/lltypesystem/test/test_llmemory.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/test/test_llmemory.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/test/test_llmemory.py Wed Oct 20 11:44:02 2010
@@ -324,12 +324,14 @@
p_t = lltype.malloc(T)
assert p_t.s == lltype.nullptr(S)
# raw malloc does not
- p_raw_t = lltype.malloc(T, flavor="raw")
- py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.s")
+ U = lltype.Struct("U", ('x', lltype.Signed))
+ p_raw_t = lltype.malloc(U, flavor="raw")
+ py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.x")
+ lltype.free(p_raw_t, flavor="raw")
# this sort of raw_malloc too
- p_raw_t = cast_adr_to_ptr(raw_malloc(sizeof(T)), lltype.Ptr(T))
- py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.s")
-
+ p_raw_t = cast_adr_to_ptr(raw_malloc(sizeof(U)), lltype.Ptr(U))
+ py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.x")
+
def test_raw_malloc_signed_bunch():
adr = raw_malloc(sizeof(lltype.Signed) * 50)
@@ -601,7 +603,8 @@
a = lltype.malloc(A, flavor='raw')
src = cast_ptr_to_adr(a) + itemoffsetof(A, 0)
raw_memclear(src, sizeof(lltype.Signed) * 0)
-
+ lltype.free(a, flavor="raw")
+
def test_nonneg():
S1 = lltype.GcStruct('S1', ('x', lltype.Float))
A1 = lltype.GcArray(lltype.Float)
Modified: pypy/trunk/pypy/rpython/lltypesystem/test/test_lltype.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/test/test_lltype.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/test/test_lltype.py Wed Oct 20 11:44:02 2010
@@ -2,6 +2,7 @@
from pypy.rpython.lltypesystem.lltype import *
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
def isweak(p, T):
try:
@@ -804,22 +805,20 @@
class TestTrackAllocation:
- def setup_method(self, func):
- start_tracking_allocations()
-
- def teardown_method(self, func):
- assert not lltype.ALLOCATED, "Memory was not correctly freed"
- stop_tracking_allocations()
+ def test_automatic_tracking(self):
+ # calls to start_tracking_allocations/stop_tracking_allocations
+ # should occur automatically from pypy/conftest.py. Check that.
+ assert leakfinder.TRACK_ALLOCATIONS
def test_track_allocation(self):
"""A malloc'd buffer fills the ALLOCATED dictionary"""
- assert lltype.TRACK_ALLOCATIONS
- assert not lltype.ALLOCATED
+ assert leakfinder.TRACK_ALLOCATIONS
+ assert not leakfinder.ALLOCATED
buf = malloc(Array(Signed), 1, flavor="raw")
- assert len(lltype.ALLOCATED) == 1
- assert lltype.ALLOCATED.keys() == [buf._obj]
+ assert len(leakfinder.ALLOCATED) == 1
+ assert leakfinder.ALLOCATED.keys() == [buf._obj]
free(buf, flavor="raw")
- assert not lltype.ALLOCATED
+ assert not leakfinder.ALLOCATED
def test_str_from_buffer(self):
"""gc-managed memory does not need to be freed"""
@@ -828,16 +827,22 @@
for i in range(size): raw_buf[i] = 'a'
rstr = rffi.str_from_buffer(raw_buf, gc_buf, size, size)
rffi.keep_buffer_alive_until_here(raw_buf, gc_buf)
- assert not lltype.ALLOCATED
+ assert not leakfinder.ALLOCATED
def test_leak_traceback(self):
"""Test info stored for allocated items"""
buf = malloc(Array(Signed), 1, flavor="raw")
- traceback = lltype.ALLOCATED.keys()[0]._traceback
+ traceback = leakfinder.ALLOCATED.values()[0]
lines = traceback.splitlines()
assert 'malloc(' in lines[-1] and 'flavor="raw")' in lines[-1]
- # XXX The traceback should not be too long
+ # The traceback should not be too long
print traceback
free(buf, flavor="raw")
+
+ def test_no_tracking(self):
+ p1 = malloc(Array(Signed), 1, flavor='raw', track_allocation=False)
+ p2 = malloc(Array(Signed), 1, flavor='raw', track_allocation=False)
+ free(p2, flavor='raw', track_allocation=False)
+ # p1 is not freed
Modified: pypy/trunk/pypy/rpython/lltypesystem/test/test_rffi.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/test/test_rffi.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/test/test_rffi.py Wed Oct 20 11:44:02 2010
@@ -9,7 +9,7 @@
from pypy.rpython.lltypesystem.rstr import STR
from pypy.rpython.lltypesystem import lltype
from pypy.tool.udir import udir
-from pypy.rpython.test.test_llinterp import interpret, MallocMismatch
+from pypy.rpython.test.test_llinterp import interpret
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.annotation.annrpython import RPythonAnnotator
from pypy.rpython.rtyper import RPythonTyper
Modified: pypy/trunk/pypy/rpython/memory/gc/minimarkpage.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/gc/minimarkpage.py (original)
+++ pypy/trunk/pypy/rpython/memory/gc/minimarkpage.py Wed Oct 20 11:44:02 2010
@@ -100,11 +100,14 @@
# allocation of the given size.
length = small_request_threshold / WORD + 1
self.page_for_size = lltype.malloc(rffi.CArray(PAGE_PTR), length,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
self.full_page_for_size = lltype.malloc(rffi.CArray(PAGE_PTR), length,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
self.nblocks_for_size = lltype.malloc(rffi.CArray(lltype.Signed),
- length, flavor='raw')
+ length, flavor='raw',
+ immortal=True)
self.hdrsize = llmemory.raw_malloc_usage(llmemory.sizeof(PAGE_HEADER))
assert page_size > self.hdrsize
self.nblocks_for_size[0] = 0 # unused
@@ -114,11 +117,13 @@
self.max_pages_per_arena = arena_size // page_size
self.arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
self.max_pages_per_arena,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
# this is used in mass_free() only
self.old_arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
self.max_pages_per_arena,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
#
# the arena currently consumed; it must have at least one page
# available, or be NULL. The arena object that we point to is
@@ -281,7 +286,7 @@
npages = (arena_end - firstpage) // self.page_size
#
# Allocate an ARENA object and initialize it
- arena = lltype.malloc(ARENA, flavor='raw')
+ arena = lltype.malloc(ARENA, flavor='raw', track_allocation=False)
arena.base = arena_base
arena.nfreepages = 0 # they are all uninitialized pages
arena.totalpages = npages
@@ -332,7 +337,7 @@
#
# The whole arena is empty. Free it.
llarena.arena_free(arena.base)
- lltype.free(arena, flavor='raw')
+ lltype.free(arena, flavor='raw', track_allocation=False)
#
else:
# Insert 'arena' in the correct arenas_lists[n]
Modified: pypy/trunk/pypy/rpython/memory/gctransform/asmgcroot.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/gctransform/asmgcroot.py (original)
+++ pypy/trunk/pypy/rpython/memory/gctransform/asmgcroot.py Wed Oct 20 11:44:02 2010
@@ -75,7 +75,8 @@
key = (TYPE, num)
if key not in sradict:
CONTAINER = lltype.FixedSizeArray(TYPE, 1)
- p = lltype.malloc(CONTAINER, flavor='raw', zero=True)
+ p = lltype.malloc(CONTAINER, flavor='raw', zero=True,
+ immortal=True)
sradict[key] = Constant(p, lltype.Ptr(CONTAINER))
sra.append(sradict[key])
#
Modified: pypy/trunk/pypy/rpython/memory/gctransform/transform.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/gctransform/transform.py (original)
+++ pypy/trunk/pypy/rpython/memory/gctransform/transform.py Wed Oct 20 11:44:02 2010
@@ -430,7 +430,8 @@
return self.parenttransformer.gct_malloc_varsize(hop)
def gct_free(self, hop):
- flavor = hop.spaceop.args[1].value
+ flags = hop.spaceop.args[1].value
+ flavor = flags['flavor']
assert flavor == 'raw'
return self.parenttransformer.gct_free(hop)
@@ -532,6 +533,8 @@
resulttype=llmemory.Address)
if flags.get('zero'):
hop.genop("raw_memclear", [v_raw, c_size])
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_start", [v_raw])
return v_raw
def gct_fv_stack_malloc(self, hop, flags, TYPE, c_size):
@@ -602,15 +605,20 @@
[self.raw_malloc_varsize_ptr, v_length,
c_const_size, c_item_size, c_offset_to_length],
resulttype=llmemory.Address)
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_start", [v_raw])
return v_raw
def gct_free(self, hop):
op = hop.spaceop
- flavor = op.args[1].value
+ flags = op.args[1].value
+ flavor = flags['flavor']
v = op.args[0]
assert flavor != 'cpy', "cannot free CPython objects directly"
if flavor == 'raw':
v = hop.genop("cast_ptr_to_adr", [v], resulttype=llmemory.Address)
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_stop", [v])
hop.genop('raw_free', [v])
else:
assert False, "%s has no support for free with flavor %r" % (self, flavor)
Modified: pypy/trunk/pypy/rpython/memory/gcwrapper.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/gcwrapper.py (original)
+++ pypy/trunk/pypy/rpython/memory/gcwrapper.py Wed Oct 20 11:44:02 2010
@@ -42,7 +42,8 @@
#
# Interface for the llinterp
#
- def malloc(self, TYPE, n=None, flavor='gc', zero=False):
+ def malloc(self, TYPE, n=None, flavor='gc', zero=False,
+ track_allocation=True):
if flavor == 'gc':
typeid = self.get_type_id(TYPE)
addr = self.gc.malloc(typeid, n, zero=zero)
@@ -51,7 +52,8 @@
gctypelayout.zero_gc_pointers(result)
return result
else:
- return lltype.malloc(TYPE, n, flavor=flavor, zero=zero)
+ return lltype.malloc(TYPE, n, flavor=flavor, zero=zero,
+ track_allocation=track_allocation)
def malloc_nonmovable(self, TYPE, n=None, zero=False):
typeid = self.get_type_id(TYPE)
@@ -69,9 +71,10 @@
return self.gc.shrink_array(addr, smallersize)
return False
- def free(self, TYPE, flavor='gc'):
+ def free(self, TYPE, flavor='gc', track_allocation=True):
assert flavor != 'gc'
- return lltype.free(TYPE, flavor=flavor)
+ return lltype.free(TYPE, flavor=flavor,
+ track_allocation=track_allocation)
def setfield(self, obj, fieldname, fieldvalue):
STRUCT = lltype.typeOf(obj).TO
Modified: pypy/trunk/pypy/rpython/memory/support.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/support.py (original)
+++ pypy/trunk/pypy/rpython/memory/support.py Wed Oct 20 11:44:02 2010
@@ -30,7 +30,8 @@
# we zero-initialize the chunks to make the translation
# backends happy, but we don't need to do it at run-time.
zero = not we_are_translated()
- return lltype.malloc(CHUNK, flavor="raw", zero=zero)
+ return lltype.malloc(CHUNK, flavor="raw", zero=zero,
+ track_allocation=False)
result = self.free_list
self.free_list = result.next
@@ -44,7 +45,7 @@
# Don't cache the old chunks but free them immediately.
# Helps debugging, and avoids that old chunks full of
# addresses left behind by a test end up in genc...
- lltype.free(chunk, flavor="raw")
+ lltype.free(chunk, flavor="raw", track_allocation=False)
unused_chunks = FreeList()
cache[chunk_size] = unused_chunks, null_chunk
Modified: pypy/trunk/pypy/rpython/memory/test/test_gc.py
==============================================================================
--- pypy/trunk/pypy/rpython/memory/test/test_gc.py (original)
+++ pypy/trunk/pypy/rpython/memory/test/test_gc.py Wed Oct 20 11:44:02 2010
@@ -368,6 +368,14 @@
res = self.interpret(f, [4, 42])
assert res == 12
+ def test_print_leak(self):
+ def f(n):
+ for i in range(n):
+ print i
+ return 42
+ res = self.interpret(f, [10])
+ assert res == 42
+
def test_weakref_across_minor_collection(self):
import weakref
class A:
Modified: pypy/trunk/pypy/rpython/rbuiltin.py
==============================================================================
--- pypy/trunk/pypy/rpython/rbuiltin.py (original)
+++ pypy/trunk/pypy/rpython/rbuiltin.py Wed Oct 20 11:44:02 2010
@@ -345,17 +345,22 @@
BUILTIN_TYPER[object.__init__] = rtype_object__init__
# annotation of low-level types
-def rtype_malloc(hop, i_flavor=None, i_zero=None):
+def rtype_malloc(hop, i_flavor=None, i_zero=None, i_track_allocation=None):
assert hop.args_s[0].is_constant()
vlist = [hop.inputarg(lltype.Void, arg=0)]
opname = 'malloc'
- v_flavor, v_zero = parse_kwds(hop, (i_flavor, lltype.Void), (i_zero, None))
+ v_flavor, v_zero, v_track_allocation = parse_kwds(hop,
+ (i_flavor, lltype.Void),
+ (i_zero, None),
+ (i_track_allocation, None))
flags = {'flavor': 'gc'}
if v_flavor is not None:
flags['flavor'] = v_flavor.value
if i_zero is not None:
flags['zero'] = v_zero.value
+ if i_track_allocation is not None:
+ flags['track_allocation'] = v_track_allocation.value
vlist.append(hop.inputconst(lltype.Void, flags))
if hop.nb_args == 2:
@@ -366,10 +371,19 @@
hop.exception_is_here()
return hop.genop(opname, vlist, resulttype = hop.r_result.lowleveltype)
-def rtype_free(hop, i_flavor):
- assert i_flavor == 1
+def rtype_free(hop, i_flavor, i_track_allocation=None):
+ vlist = [hop.inputarg(hop.args_r[0], arg=0)]
+ v_flavor, v_track_allocation = parse_kwds(hop,
+ (i_flavor, lltype.Void),
+ (i_track_allocation, None))
+ #
+ assert v_flavor is not None and v_flavor.value == 'raw'
+ flags = {'flavor': 'raw'}
+ if i_track_allocation is not None:
+ flags['track_allocation'] = v_track_allocation.value
+ vlist.append(hop.inputconst(lltype.Void, flags))
+ #
hop.exception_cannot_occur()
- vlist = hop.inputargs(hop.args_r[0], lltype.Void)
hop.genop('free', vlist)
def rtype_const_result(hop):
@@ -584,8 +598,9 @@
vinst, = hop.inputargs(hop.args_r[0])
flavor = hop.args_r[0].gcflavor
assert flavor != 'gc'
- cflavor = hop.inputconst(lltype.Void, flavor)
- return hop.genop('free', [vinst, cflavor])
+ flags = {'flavor': flavor}
+ cflags = hop.inputconst(lltype.Void, flags)
+ return hop.genop('free', [vinst, cflags])
BUILTIN_TYPER[objectmodel.free_non_gc_object] = rtype_free_non_gc_object
Modified: pypy/trunk/pypy/rpython/test/test_llinterp.py
==============================================================================
--- pypy/trunk/pypy/rpython/test/test_llinterp.py (original)
+++ pypy/trunk/pypy/rpython/test/test_llinterp.py Wed Oct 20 11:44:02 2010
@@ -12,13 +12,11 @@
from pypy.annotation.model import lltype_to_annotation
from pypy.rlib.rarithmetic import r_uint, ovfcheck
from pypy.rpython.ootypesystem import ootype
+from pypy.tool import leakfinder
from pypy import conftest
# switch on logging of interp to show more info on failing tests
-class MallocMismatch(Exception):
- pass
-
def setup_module(mod):
mod.logstate = py.log._getstate()
py.log.setconsumer("llinterp", py.log.STDOUT)
@@ -72,7 +70,7 @@
def get_interpreter(func, values, view='auto', viewbefore='auto', policy=None,
someobjects=False, type_system="lltype", backendopt=False,
- config=None, malloc_check=True, **extraconfigopts):
+ config=None, **extraconfigopts):
extra_key = [(key, value) for key, value in extraconfigopts.iteritems()]
extra_key.sort()
extra_key = tuple(extra_key)
@@ -97,7 +95,7 @@
viewbefore, policy, type_system=type_system,
backendopt=backendopt, config=config,
**extraconfigopts)
- interp = LLInterpreter(typer, malloc_check=malloc_check)
+ interp = LLInterpreter(typer)
_tcache[key] = (t, interp, graph)
# keep the cache small
_lastinterpreted.append(key)
@@ -115,10 +113,17 @@
interp, graph = get_interpreter(func, values, view, viewbefore, policy,
someobjects, type_system=type_system,
backendopt=backendopt, config=config,
- malloc_check=malloc_check, **kwargs)
- result = interp.eval_graph(graph, values)
- if malloc_check and interp.mallocs:
- raise MallocMismatch(interp.mallocs)
+ **kwargs)
+ if not malloc_check:
+ result = interp.eval_graph(graph, values)
+ else:
+ prev = leakfinder.start_tracking_allocations()
+ try:
+ result = interp.eval_graph(graph, values)
+ finally:
+ leaks = leakfinder.stop_tracking_allocations(False, prev)
+ if leaks:
+ raise leakfinder.MallocMismatch(leaks)
return result
def interpret_raises(exc, func, values, view='auto', viewbefore='auto',
@@ -418,6 +423,7 @@
assert result
def test_stack_malloc():
+ py.test.skip("stack-flavored mallocs no longer supported")
class A(object):
pass
def f():
@@ -430,6 +436,7 @@
assert result == 1
def test_invalid_stack_access():
+ py.test.skip("stack-flavored mallocs no longer supported")
class A(object):
pass
globala = A()
@@ -605,7 +612,7 @@
if x:
free(t, flavor='raw')
interpret(f, [1])
- py.test.raises(MallocMismatch, "interpret(f, [0])")
+ py.test.raises(leakfinder.MallocMismatch, "interpret(f, [0])")
def f():
t1 = malloc(T, flavor='raw')
Modified: pypy/trunk/pypy/rpython/test/test_nongc.py
==============================================================================
--- pypy/trunk/pypy/rpython/test/test_nongc.py (original)
+++ pypy/trunk/pypy/rpython/test/test_nongc.py Wed Oct 20 11:44:02 2010
@@ -79,7 +79,7 @@
py.test.raises(TypeError,rtyper.specialize) # results in an invalid cast
def test_isinstance():
- class A:
+ class A(object):
_alloc_flavor_ = "raw"
class B(A):
pass
@@ -95,7 +95,24 @@
o = B()
else:
o = C()
- return 100*isinstance(o, A)+10*isinstance(o, B)+1*isinstance(o ,C)
+ res = 100*isinstance(o, A) + 10*isinstance(o, B) + 1*isinstance(o, C)
+ if i == 0:
+ pass
+ elif i == 1:
+ assert isinstance(o, A)
+ free_non_gc_object(o)
+ elif i == 2:
+ assert isinstance(o, B)
+ free_non_gc_object(o)
+ else:
+ assert isinstance(o, C)
+ free_non_gc_object(o)
+ return res
+
+ assert f(1) == 100
+ assert f(2) == 110
+ assert f(3) == 111
+ assert f(0) == 0
a = RPythonAnnotator()
#does not raise:
@@ -131,10 +148,14 @@
d = b
elif i == 2:
e = c
- return (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
+ res = (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
0x0008*(a is e) | 0x0010*(b is c) | 0x0020*(b is d) |
0x0040*(b is e) | 0x0080*(c is d) | 0x0100*(c is e) |
0x0200*(d is e))
+ free_non_gc_object(a)
+ free_non_gc_object(b)
+ free_non_gc_object(c)
+ return res
a = RPythonAnnotator()
#does not raise:
s = a.build_types(f, [int])
@@ -169,10 +190,13 @@
d = b
elif i == 2:
e = c
- return (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
+ res = (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
0x0008*(a is e) | 0x0010*(b is c) | 0x0020*(b is d) |
0x0040*(b is e) | 0x0080*(c is d) | 0x0100*(c is e) |
0x0200*(d is e))
+ free_non_gc_object(a)
+ free_non_gc_object(b)
+ return res
a = RPythonAnnotator()
#does not raise:
s = a.build_types(f, [int])
Modified: pypy/trunk/pypy/rpython/test/test_rptr.py
==============================================================================
--- pypy/trunk/pypy/rpython/test/test_rptr.py (original)
+++ pypy/trunk/pypy/rpython/test/test_rptr.py Wed Oct 20 11:44:02 2010
@@ -212,10 +212,31 @@
S = Struct('S', ('x', Signed))
def fn(n):
- p = malloc(S, flavor='whatever')
+ p = malloc(S, flavor='raw')
p.x = n
result = p.x
- free(p, flavor='whatever')
+ free(p, flavor='raw')
+ return n
+
+ res = interpret(fn, [23])
+ assert res == 23
+
+ S = Struct('S', ('x', Signed))
+ def fn(n):
+ p = malloc(S, flavor='raw', track_allocation=False)
+ p.x = n
+ result = p.x
+ return n
+
+ res = interpret(fn, [23])
+ assert res == 23
+
+ S = Struct('S', ('x', Signed))
+ def fn(n):
+ p = malloc(S, flavor='raw', track_allocation=False)
+ p.x = n
+ result = p.x
+ free(p, flavor='raw', track_allocation=False)
return n
res = interpret(fn, [23])
Modified: pypy/trunk/pypy/translator/c/src/g_include.h
==============================================================================
--- pypy/trunk/pypy/translator/c/src/g_include.h (original)
+++ pypy/trunk/pypy/translator/c/src/g_include.h Wed Oct 20 11:44:02 2010
@@ -53,6 +53,7 @@
# include "src/rtyper.h"
# include "src/debug_print.h"
# include "src/debug_traceback.h"
+# include "src/debug_alloc.h"
#ifndef AVR
# include "src/ll_os.h"
# include "src/ll_strtod.h"
Modified: pypy/trunk/pypy/translator/c/src/main.h
==============================================================================
--- pypy/trunk/pypy/translator/c/src/main.h (original)
+++ pypy/trunk/pypy/translator/c/src/main.h Wed Oct 20 11:44:02 2010
@@ -53,10 +53,16 @@
}
exitcode = STANDALONE_ENTRY_POINT(list);
+
+#ifdef RPY_ASSERT
+ pypy_debug_alloc_results();
+#endif
+
if (RPyExceptionOccurred()) {
/* print the RPython traceback */
pypy_debug_catch_fatal_exception();
}
+
return exitcode;
memory_out:
Modified: pypy/trunk/pypy/translator/c/test/test_genc.py
==============================================================================
--- pypy/trunk/pypy/translator/c/test/test_genc.py (original)
+++ pypy/trunk/pypy/translator/c/test/test_genc.py Wed Oct 20 11:44:02 2010
@@ -426,6 +426,7 @@
if py.test.config.option.view:
t.view()
assert ' BarStruct ' in t.driver.cbuilder.c_source_filename.read()
+ free(foo, flavor="raw")
def test_recursive_llhelper():
from pypy.rpython.annlowlevel import llhelper
@@ -473,7 +474,7 @@
return f(s)
a_f = A(f, "f")
a_g = A(g, "g")
- t = lltype.malloc(STRUCT, flavor="raw")
+ t = lltype.malloc(STRUCT, flavor="raw", immortal=True)
t.bar = llhelper(FTPTR, a_f.make_func())
fn = compile(chooser, [bool])
assert fn(True)
Modified: pypy/trunk/pypy/translator/c/test/test_lltyped.py
==============================================================================
--- pypy/trunk/pypy/translator/c/test/test_lltyped.py (original)
+++ pypy/trunk/pypy/translator/c/test/test_lltyped.py Wed Oct 20 11:44:02 2010
@@ -401,6 +401,7 @@
for i in range(n):
p = malloc(S, flavor='raw', zero=True)
if p.x != 0 or p.y != 0:
+ free(p, flavor='raw')
return -1
p.x = i
p.y = i
@@ -418,14 +419,16 @@
def f(n):
for length in range(n-1, -1, -1):
p = malloc(S, length, flavor='raw', zero=True)
- if p.x != 0:
- return -1
- p.x = n
- for j in range(length):
- if p.y[j] != 0:
- return -3
- p.y[j] = n^j
- free(p, flavor='raw')
+ try:
+ if p.x != 0:
+ return -1
+ p.x = n
+ for j in range(length):
+ if p.y[j] != 0:
+ return -3
+ p.y[j] = n^j
+ finally:
+ free(p, flavor='raw')
return 42
fn = self.getcompiled(f, [int])
@@ -655,7 +658,7 @@
def test_prebuilt_ll2ctypes_array(self):
from pypy.rpython.lltypesystem import rffi, ll2ctypes
A = rffi.CArray(Char)
- a = malloc(A, 6, flavor='raw')
+ a = malloc(A, 6, flavor='raw', immortal=True)
a[0] = 'a'
a[1] = 'b'
a[2] = 'c'
@@ -676,7 +679,7 @@
def test_ll2ctypes_array_from_c(self):
from pypy.rpython.lltypesystem import rffi, ll2ctypes
A = rffi.CArray(Char)
- a = malloc(A, 6, flavor='raw')
+ a = malloc(A, 6, flavor='raw', immortal=True)
a[0] = 'a'
a[1] = 'b'
a[2] = 'c'
From arigo at codespeak.net Wed Oct 20 11:44:19 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 11:44:19 +0200 (CEST)
Subject: [pypy-svn] r78114 - pypy/branch/leak-finder
Message-ID: <20101020094419.98B83282B90@codespeak.net>
Author: arigo
Date: Wed Oct 20 11:44:18 2010
New Revision: 78114
Removed:
pypy/branch/leak-finder/
Log:
Remove merged branch.
From arigo at codespeak.net Wed Oct 20 11:46:06 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 11:46:06 +0200 (CEST)
Subject: [pypy-svn] r78115 - in pypy/branch/jit-sign-descr/pypy/jit/backend:
llsupport test x86
Message-ID: <20101020094606.C467036E3FB@codespeak.net>
Author: arigo
Date: Wed Oct 20 11:46:04 2010
New Revision: 78115
Modified:
pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/descr.py
pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/llmodel.py
pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/symbolic.py
pypy/branch/jit-sign-descr/pypy/jit/backend/test/runner_test.py
pypy/branch/jit-sign-descr/pypy/jit/backend/x86/assembler.py
pypy/branch/jit-sign-descr/pypy/jit/backend/x86/regalloc.py
pypy/branch/jit-sign-descr/pypy/jit/backend/x86/regloc.py
pypy/branch/jit-sign-descr/pypy/jit/backend/x86/rx86.py
Log:
Finish this.
Modified: pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/descr.py
==============================================================================
--- pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/descr.py (original)
+++ pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/descr.py Wed Oct 20 11:46:04 2010
@@ -425,7 +425,7 @@
#
if TYPE is lltype.Float:
setattr(Descr, floatattrname, True)
- elif rffi.cast(TYPE, -1) == -1:
+ elif TYPE is not lltype.Bool and rffi.cast(TYPE, -1) == -1:
setattr(Descr, signedattrname, True)
#
_cache[nameprefix, TYPE] = Descr
Modified: pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/llmodel.py
==============================================================================
--- pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/llmodel.py (original)
+++ pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/llmodel.py Wed Oct 20 11:46:04 2010
@@ -211,7 +211,8 @@
assert isinstance(fielddescr, BaseFieldDescr)
ofs = fielddescr.offset
size = fielddescr.get_field_size(self.translate_support_code)
- return ofs, size
+ sign = fielddescr.is_field_signed()
+ return ofs, size, sign
unpack_fielddescr_size._always_inline_ = True
def arraydescrof(self, A):
@@ -226,7 +227,8 @@
assert isinstance(arraydescr, BaseArrayDescr)
ofs = arraydescr.get_base_size(self.translate_support_code)
size = arraydescr.get_item_size(self.translate_support_code)
- return ofs, size
+ sign = arraydescr.is_item_signed()
+ return ofs, size, sign
unpack_arraydescr_size._always_inline_ = True
def calldescrof(self, FUNC, ARGS, RESULT, extrainfo=None):
@@ -256,15 +258,21 @@
@specialize.argtype(2)
def bh_getarrayitem_gc_i(self, arraydescr, gcref, itemindex):
- ofs, size = self.unpack_arraydescr_size(arraydescr)
+ ofs, size, sign = self.unpack_arraydescr_size(arraydescr)
# --- start of GC unsafe code (no GC operation!) ---
items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for STYPE, UTYPE, itemsize in unroll_basic_sizes:
if size == itemsize:
- items = rffi.cast(rffi.CArrayPtr(TYPE), items)
- val = items[itemindex]
+ if sign:
+ items = rffi.cast(rffi.CArrayPtr(STYPE), items)
+ val = items[itemindex]
+ val = rffi.cast(lltype.Signed, val)
+ else:
+ items = rffi.cast(rffi.CArrayPtr(UTYPE), items)
+ val = items[itemindex]
+ val = rffi.cast(lltype.Signed, val)
# --- end of GC unsafe code ---
- return rffi.cast(lltype.Signed, val)
+ return val
else:
raise NotImplementedError("size = %d" % size)
@@ -289,10 +297,10 @@
@specialize.argtype(2)
def bh_setarrayitem_gc_i(self, arraydescr, gcref, itemindex, newvalue):
- ofs, size = self.unpack_arraydescr_size(arraydescr)
+ ofs, size, sign = self.unpack_arraydescr_size(arraydescr)
# --- start of GC unsafe code (no GC operation!) ---
items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for TYPE, _, itemsize in unroll_basic_sizes:
if size == itemsize:
items = rffi.cast(rffi.CArrayPtr(TYPE), items)
items[itemindex] = rffi.cast(TYPE, newvalue)
@@ -343,14 +351,22 @@
@specialize.argtype(1)
def _base_do_getfield_i(self, struct, fielddescr):
- ofs, size = self.unpack_fielddescr_size(fielddescr)
+ ofs, size, sign = self.unpack_fielddescr_size(fielddescr)
# --- start of GC unsafe code (no GC operation!) ---
fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for STYPE, UTYPE, itemsize in unroll_basic_sizes:
if size == itemsize:
- val = rffi.cast(rffi.CArrayPtr(TYPE), fieldptr)[0]
+ # Note that in the common case where size==sizeof(Signed),
+ # both cases of what follows are doing the same thing.
+ # But gcc is clever enough to figure this out :-)
+ if sign:
+ val = rffi.cast(rffi.CArrayPtr(STYPE), fieldptr)[0]
+ val = rffi.cast(lltype.Signed, val)
+ else:
+ val = rffi.cast(rffi.CArrayPtr(UTYPE), fieldptr)[0]
+ val = rffi.cast(lltype.Signed, val)
# --- end of GC unsafe code ---
- return rffi.cast(lltype.Signed, val)
+ return val
else:
raise NotImplementedError("size = %d" % size)
@@ -382,10 +398,10 @@
@specialize.argtype(1)
def _base_do_setfield_i(self, struct, fielddescr, newvalue):
- ofs, size = self.unpack_fielddescr_size(fielddescr)
+ ofs, size, sign = self.unpack_fielddescr_size(fielddescr)
# --- start of GC unsafe code (no GC operation!) ---
fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for TYPE, _, itemsize in unroll_basic_sizes:
if size == itemsize:
fieldptr = rffi.cast(rffi.CArrayPtr(TYPE), fieldptr)
fieldptr[0] = rffi.cast(TYPE, newvalue)
Modified: pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/symbolic.py
==============================================================================
--- pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/symbolic.py (original)
+++ pypy/branch/jit-sign-descr/pypy/jit/backend/llsupport/symbolic.py Wed Oct 20 11:46:04 2010
@@ -69,8 +69,9 @@
SIZEOF_INT = get_size(rffi.INT, False)
SIZEOF_FLOAT = get_size(lltype.Float, False)
-unroll_basic_sizes = unrolling_iterable([(lltype.Signed, WORD),
- (lltype.Char, SIZEOF_CHAR),
- (rffi.SHORT, SIZEOF_SHORT),
- (rffi.INT, SIZEOF_INT)])
+unroll_basic_sizes = unrolling_iterable([
+ (lltype.Signed, lltype.Unsigned, WORD),
+ (rffi.SIGNEDCHAR, lltype.Char, SIZEOF_CHAR),
+ (rffi.SHORT, rffi.USHORT, SIZEOF_SHORT),
+ (rffi.INT, rffi.UINT, SIZEOF_INT)])
# does not contain Float ^^^ which must be special-cased
Modified: pypy/branch/jit-sign-descr/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/branch/jit-sign-descr/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/branch/jit-sign-descr/pypy/jit/backend/test/runner_test.py Wed Oct 20 11:46:04 2010
@@ -2067,7 +2067,7 @@
res = self.execute_operation(rop.GETFIELD_GC, [BoxPtr(s_gcref)],
'int', descr=descrfld_x)
assert res.value == expected, (
- "%r: got %r, expected %r" % (RESTYPE, x, expected))
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
def test_short_result_of_getarrayitem_direct(self):
# Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
Modified: pypy/branch/jit-sign-descr/pypy/jit/backend/x86/assembler.py
==============================================================================
--- pypy/branch/jit-sign-descr/pypy/jit/backend/x86/assembler.py (original)
+++ pypy/branch/jit-sign-descr/pypy/jit/backend/x86/assembler.py Wed Oct 20 11:46:04 2010
@@ -8,7 +8,8 @@
from pypy.rpython.annlowlevel import llhelper
from pypy.tool.uid import fixid
from pypy.jit.backend.x86.regalloc import (RegAlloc, X86RegisterManager,
- X86XMMRegisterManager, get_ebp_ofs)
+ X86XMMRegisterManager, get_ebp_ofs,
+ _get_scale)
from pypy.jit.backend.x86.arch import (FRAME_FIXED_SIZE, FORCE_INDEX_OFS, WORD,
IS_X86_32, IS_X86_64)
@@ -22,7 +23,8 @@
X86_64_SCRATCH_REG,
X86_64_XMM_SCRATCH_REG,
RegLoc, StackLoc, ConstFloatLoc,
- ImmedLoc, AddressLoc, imm)
+ ImmedLoc, AddressLoc, imm,
+ imm0, imm1)
from pypy.rlib.objectmodel import we_are_translated, specialize
from pypy.jit.backend.x86 import rx86, regloc, codebuf
@@ -440,7 +442,7 @@
# self.mc.PUSH(eax)
# adr = rffi.cast(lltype.Signed, self.loop_run_counters[-1][1])
# self.mc.MOV(eax, heap(adr))
- # self.mc.ADD(eax, imm(1))
+ # self.mc.ADD(eax, imm1)
# self.mc.MOV(heap(adr), eax)
# self.mc.POP(eax)
return operations
@@ -711,7 +713,7 @@
self.regalloc_perform_with_guard(None, guard_op, faillocs, arglocs,
resloc, current_depths)
- def load_effective_addr(self, sizereg, baseofs, scale, result, frm=imm(0)):
+ def load_effective_addr(self, sizereg, baseofs, scale, result, frm=imm0):
self.mc.LEA(result, addr_add(frm, sizereg, baseofs, scale))
def _unaryop(asmop):
@@ -973,28 +975,28 @@
def genop_guard_int_is_true(self, op, guard_op, guard_token, arglocs, resloc):
guard_opnum = guard_op.getopnum()
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
if guard_opnum == rop.GUARD_TRUE:
return self.implement_guard(guard_token, 'Z')
else:
return self.implement_guard(guard_token, 'NZ')
def genop_int_is_true(self, op, arglocs, resloc):
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
rl = resloc.lowest8bits()
self.mc.SET_ir(rx86.Conditions['NE'], rl.value)
self.mc.MOVZX8(resloc, rl)
def genop_guard_int_is_zero(self, op, guard_op, guard_token, arglocs, resloc):
guard_opnum = guard_op.getopnum()
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
if guard_opnum == rop.GUARD_TRUE:
return self.implement_guard(guard_token, 'NZ')
else:
return self.implement_guard(guard_token, 'Z')
def genop_int_is_zero(self, op, arglocs, resloc):
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
rl = resloc.lowest8bits()
self.mc.SET_ir(rx86.Conditions['E'], rl.value)
self.mc.MOVZX8(resloc, rl)
@@ -1050,50 +1052,66 @@
assert result_loc is eax
self.call(self.malloc_unicode_func_addr, arglocs, eax)
- def genop_getfield_gc(self, op, arglocs, resloc):
- base_loc, ofs_loc, size_loc = arglocs
- assert isinstance(size_loc, ImmedLoc)
+ # ----------
+
+ def load_from_mem(self, resloc, source_addr, size_loc, sign_loc):
assert isinstance(resloc, RegLoc)
size = size_loc.value
-
- source_addr = AddressLoc(base_loc, ofs_loc)
+ sign = sign_loc.value
if resloc.is_xmm:
self.mc.MOVSD(resloc, source_addr)
+ elif size == WORD:
+ self.mc.MOV(resloc, source_addr)
+ elif size == 1:
+ if sign:
+ self.mc.MOVSX8(resloc, source_addr)
+ else:
+ self.mc.MOVZX8(resloc, source_addr)
+ elif size == 2:
+ if sign:
+ self.mc.MOVSX16(resloc, source_addr)
+ else:
+ self.mc.MOVZX16(resloc, source_addr)
+ elif IS_X86_64 and size == 4:
+ if sign:
+ self.mc.MOVSX32(resloc, source_addr)
+ else:
+ self.mc.MOV32(resloc, source_addr) # zero-extending
+ else:
+ not_implemented("load_from_mem size = %d" % size)
+
+ def save_into_mem(self, dest_addr, value_loc, size_loc):
+ size = size_loc.value
+ if isinstance(value_loc, RegLoc) and value_loc.is_xmm:
+ self.mc.MOVSD(dest_addr, value_loc)
elif size == 1:
- self.mc.MOVZX8(resloc, source_addr)
+ self.mc.MOV8(dest_addr, value_loc.lowest8bits())
elif size == 2:
- self.mc.MOVZX16(resloc, source_addr)
+ self.mc.MOV16(dest_addr, value_loc)
elif size == 4:
- # MOV32 is zero-extending on 64-bit, so this is okay
- self.mc.MOV32(resloc, source_addr)
+ self.mc.MOV32(dest_addr, value_loc)
elif IS_X86_64 and size == 8:
- self.mc.MOV(resloc, source_addr)
+ self.mc.MOV(dest_addr, value_loc)
else:
- raise NotImplementedError("getfield size = %d" % size)
+ not_implemented("save_into_mem size = %d" % size)
+
+ def genop_getfield_gc(self, op, arglocs, resloc):
+ base_loc, ofs_loc, size_loc, sign_loc = arglocs
+ assert isinstance(size_loc, ImmedLoc)
+ source_addr = AddressLoc(base_loc, ofs_loc)
+ self.load_from_mem(resloc, source_addr, size_loc, sign_loc)
genop_getfield_raw = genop_getfield_gc
genop_getfield_raw_pure = genop_getfield_gc
genop_getfield_gc_pure = genop_getfield_gc
def genop_getarrayitem_gc(self, op, arglocs, resloc):
- base_loc, ofs_loc, scale, ofs = arglocs
+ base_loc, ofs_loc, size_loc, ofs, sign_loc = arglocs
assert isinstance(ofs, ImmedLoc)
- assert isinstance(scale, ImmedLoc)
- src_addr = addr_add(base_loc, ofs_loc, ofs.value, scale.value)
- if op.result.type == FLOAT:
- self.mc.MOVSD(resloc, src_addr)
- else:
- if scale.value == 0:
- self.mc.MOVZX8(resloc, src_addr)
- elif scale.value == 1:
- self.mc.MOVZX16(resloc, src_addr)
- elif scale.value == 2:
- self.mc.MOV32(resloc, src_addr)
- elif IS_X86_64 and scale.value == 3:
- self.mc.MOV(resloc, src_addr)
- else:
- print "[asmgen]getarrayitem unsupported size: %d" % scale.value
- raise NotImplementedError()
+ assert isinstance(size_loc, ImmedLoc)
+ scale = _get_scale(size_loc.value)
+ src_addr = addr_add(base_loc, ofs_loc, ofs.value, scale)
+ self.load_from_mem(resloc, src_addr, size_loc, sign_loc)
genop_getarrayitem_gc_pure = genop_getarrayitem_gc
genop_getarrayitem_raw = genop_getarrayitem_gc
@@ -1101,40 +1119,16 @@
def genop_discard_setfield_gc(self, op, arglocs):
base_loc, ofs_loc, size_loc, value_loc = arglocs
assert isinstance(size_loc, ImmedLoc)
- size = size_loc.value
dest_addr = AddressLoc(base_loc, ofs_loc)
- if isinstance(value_loc, RegLoc) and value_loc.is_xmm:
- self.mc.MOVSD(dest_addr, value_loc)
- elif IS_X86_64 and size == 8:
- self.mc.MOV(dest_addr, value_loc)
- elif size == 4:
- self.mc.MOV32(dest_addr, value_loc)
- elif size == 2:
- self.mc.MOV16(dest_addr, value_loc)
- elif size == 1:
- self.mc.MOV8(dest_addr, value_loc.lowest8bits())
- else:
- print "[asmgen]setfield addr size %d" % size
- raise NotImplementedError("Addr size %d" % size)
+ self.save_into_mem(dest_addr, value_loc, size_loc)
def genop_discard_setarrayitem_gc(self, op, arglocs):
- base_loc, ofs_loc, value_loc, scale_loc, baseofs = arglocs
+ base_loc, ofs_loc, value_loc, size_loc, baseofs = arglocs
assert isinstance(baseofs, ImmedLoc)
- assert isinstance(scale_loc, ImmedLoc)
- dest_addr = AddressLoc(base_loc, ofs_loc, scale_loc.value, baseofs.value)
- if op.getarg(2).type == FLOAT:
- self.mc.MOVSD(dest_addr, value_loc)
- else:
- if IS_X86_64 and scale_loc.value == 3:
- self.mc.MOV(dest_addr, value_loc)
- elif scale_loc.value == 2:
- self.mc.MOV32(dest_addr, value_loc)
- elif scale_loc.value == 1:
- self.mc.MOV16(dest_addr, value_loc)
- elif scale_loc.value == 0:
- self.mc.MOV8(dest_addr, value_loc.lowest8bits())
- else:
- raise NotImplementedError("scale = %d" % scale_loc.value)
+ assert isinstance(size_loc, ImmedLoc)
+ scale = _get_scale(size_loc.value)
+ dest_addr = AddressLoc(base_loc, ofs_loc, scale, baseofs.value)
+ self.save_into_mem(dest_addr, value_loc, size_loc)
def genop_discard_strsetitem(self, op, arglocs):
base_loc, ofs_loc, val_loc = arglocs
@@ -1201,7 +1195,7 @@
def genop_guard_guard_no_exception(self, ign_1, guard_op, guard_token,
locs, ign_2):
- self.mc.CMP(heap(self.cpu.pos_exception()), imm(0))
+ self.mc.CMP(heap(self.cpu.pos_exception()), imm0)
return self.implement_guard(guard_token, 'NZ')
def genop_guard_guard_exception(self, ign_1, guard_op, guard_token,
@@ -1213,8 +1207,8 @@
addr = self.implement_guard(guard_token, 'NE')
if resloc is not None:
self.mc.MOV(resloc, heap(self.cpu.pos_exc_value()))
- self.mc.MOV(heap(self.cpu.pos_exception()), imm(0))
- self.mc.MOV(heap(self.cpu.pos_exc_value()), imm(0))
+ self.mc.MOV(heap(self.cpu.pos_exception()), imm0)
+ self.mc.MOV(heap(self.cpu.pos_exc_value()), imm0)
return addr
def _gen_guard_overflow(self, guard_op, guard_token):
@@ -1224,8 +1218,8 @@
elif guard_opnum == rop.GUARD_OVERFLOW:
return self.implement_guard(guard_token, 'NO')
else:
- print "int_xxx_ovf followed by", guard_op.getopname()
- raise AssertionError
+ not_implemented("int_xxx_ovf followed by %s" %
+ guard_op.getopname())
def genop_guard_int_add_ovf(self, op, guard_op, guard_token, arglocs, result_loc):
self.genop_int_add(op, arglocs, result_loc)
@@ -1288,7 +1282,7 @@
def genop_guard_guard_nonnull_class(self, ign_1, guard_op,
guard_token, locs, ign_2):
self.mc.ensure_bytes_available(256)
- self.mc.CMP(locs[0], imm(1))
+ self.mc.CMP(locs[0], imm1)
# Patched below
self.mc.J_il8(rx86.Conditions['B'], 0)
jb_location = self.mc.get_relative_pos()
@@ -1637,25 +1631,34 @@
sizeloc = arglocs[0]
assert isinstance(sizeloc, ImmedLoc)
size = sizeloc.value
+ signloc = arglocs[1]
if isinstance(op.getarg(0), Const):
x = imm(op.getarg(0).getint())
else:
- x = arglocs[1]
+ x = arglocs[2]
if x is eax:
tmp = ecx
else:
tmp = eax
- self._emit_call(x, arglocs, 2, tmp=tmp)
+ self._emit_call(x, arglocs, 3, tmp=tmp)
+
+ if IS_X86_32 and isinstance(resloc, StackLoc) and resloc.width == 8:
+ self.mc.FSTP_b(resloc.value) # float return
+ elif size == WORD:
+ assert resloc is eax or resloc is xmm0 # a full word
+ elif size == 0:
+ pass # void return
+ else:
+ # use the code in load_from_mem to do the zero- or sign-extension
+ assert resloc is eax
+ if size == 1:
+ srcloc = eax.lowest8bits()
+ else:
+ srcloc = eax
+ self.load_from_mem(eax, srcloc, sizeloc, signloc)
- if isinstance(resloc, StackLoc) and resloc.width == 8 and IS_X86_32:
- self.mc.FSTP_b(resloc.value)
- elif size == 1:
- self.mc.AND_ri(eax.value, 0xff)
- elif size == 2:
- self.mc.AND_ri(eax.value, 0xffff)
-
def genop_guard_call_may_force(self, op, guard_op, guard_token,
arglocs, result_loc):
faildescr = guard_op.getdescr()
@@ -1808,20 +1811,16 @@
self.mc.LEA_rb(resloc.value, FORCE_INDEX_OFS)
def not_implemented_op_discard(self, op, arglocs):
- msg = "not implemented operation: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation: %s" % op.getopname())
def not_implemented_op(self, op, arglocs, resloc):
- msg = "not implemented operation with res: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation with res: %s" %
+ op.getopname())
def not_implemented_op_guard(self, op, guard_op,
failaddr, arglocs, resloc):
- msg = "not implemented operation (guard): %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation (guard): %s" %
+ op.getopname())
def mark_gc_roots(self):
gcrootmap = self.cpu.gc_ll_descr.gcrootmap
@@ -1905,3 +1904,7 @@
def heap(addr):
return AddressLoc(ImmedLoc(addr), ImmedLoc(0), 0, 0)
+
+def not_implemented(msg):
+ os.write(2, '[x86/asm] %s\n' % msg)
+ raise NotImplementedError(msg)
Modified: pypy/branch/jit-sign-descr/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/branch/jit-sign-descr/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/branch/jit-sign-descr/pypy/jit/backend/x86/regalloc.py Wed Oct 20 11:46:04 2010
@@ -2,6 +2,7 @@
""" Register allocation scheme.
"""
+import os
from pypy.jit.metainterp.history import (Box, Const, ConstInt, ConstPtr,
ResOperation, BoxPtr,
LoopToken, INT, REF, FLOAT)
@@ -40,12 +41,10 @@
return imm(c.value)
elif isinstance(c, ConstPtr):
if we_are_translated() and c.value and rgc.can_move(c.value):
- print "convert_to_imm: ConstPtr needs special care"
- raise AssertionError
+ not_implemented("convert_to_imm: ConstPtr needs special care")
return imm(rffi.cast(lltype.Signed, c.value))
else:
- print "convert_to_imm: got a %s" % c
- raise AssertionError
+ not_implemented("convert_to_imm: got a %s" % c)
class X86_64_RegisterManager(X86RegisterManager):
# r11 omitted because it's used as scratch
@@ -359,8 +358,8 @@
if op.is_ovf():
if (operations[i + 1].getopnum() != rop.GUARD_NO_OVERFLOW and
operations[i + 1].getopnum() != rop.GUARD_OVERFLOW):
- print "int_xxx_ovf not followed by guard_(no)_overflow"
- raise AssertionError
+ not_implemented("int_xxx_ovf not followed by "
+ "guard_(no)_overflow")
return True
return False
if (operations[i + 1].getopnum() != rop.GUARD_TRUE and
@@ -412,8 +411,8 @@
arg = op.getarg(j)
if isinstance(arg, Box):
if arg not in start_live:
- print "Bogus arg in operation %d at %d" % (op.getopnum(), i)
- raise AssertionError
+ not_implemented("Bogus arg in operation %d at %d" %
+ (op.getopnum(), i))
longevity[arg] = (start_live[arg], i)
if op.is_guard():
for arg in op.getfailargs():
@@ -421,8 +420,8 @@
continue
assert isinstance(arg, Box)
if arg not in start_live:
- print "Bogus arg in guard %d at %d" % (op.getopnum(), i)
- raise AssertionError
+ not_implemented("Bogus arg in guard %d at %d" %
+ (op.getopnum(), i))
longevity[arg] = (start_live[arg], i)
for arg in inputargs:
if arg not in longevity:
@@ -667,7 +666,13 @@
assert isinstance(calldescr, BaseCallDescr)
assert len(calldescr.arg_classes) == op.numargs() - 1
size = calldescr.get_result_size(self.translate_support_code)
- self._call(op, [imm(size)] + [self.loc(op.getarg(i)) for i in range(op.numargs())],
+ sign = calldescr.is_result_signed()
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self._call(op, [imm(size), sign_loc] +
+ [self.loc(op.getarg(i)) for i in range(op.numargs())],
guard_not_forced_op=guard_not_forced_op)
def consider_call(self, op):
@@ -688,7 +693,7 @@
self.rm._sync_var(op.getarg(vable_index))
vable = self.fm.loc(op.getarg(vable_index))
else:
- vable = imm(0)
+ vable = imm0
self._call(op, [imm(size), vable] +
[self.loc(op.getarg(i)) for i in range(op.numargs())],
guard_not_forced_op=guard_op)
@@ -814,8 +819,9 @@
arglocs.append(self.loc(op.getarg(0)))
return self._call(op, arglocs)
# boehm GC (XXX kill the following code at some point)
- scale_of_field, basesize, ofs_length, _ = (
+ itemsize, basesize, ofs_length, _, _ = (
self._unpack_arraydescr(op.getdescr()))
+ scale_of_field = _get_scale(itemsize)
return self._malloc_varsize(basesize, ofs_length, scale_of_field,
op.getarg(0), op.result)
@@ -825,21 +831,19 @@
ofs = arraydescr.get_base_size(self.translate_support_code)
size = arraydescr.get_item_size(self.translate_support_code)
ptr = arraydescr.is_array_of_pointers()
- scale = 0
- while (1 << scale) < size:
- scale += 1
- assert (1 << scale) == size
- return scale, ofs, ofs_length, ptr
+ sign = arraydescr.is_item_signed()
+ return size, ofs, ofs_length, ptr, sign
def _unpack_fielddescr(self, fielddescr):
assert isinstance(fielddescr, BaseFieldDescr)
ofs = fielddescr.offset
size = fielddescr.get_field_size(self.translate_support_code)
ptr = fielddescr.is_pointer_field()
- return imm(ofs), imm(size), ptr
+ sign = fielddescr.is_field_signed()
+ return imm(ofs), imm(size), ptr, sign
def consider_setfield_gc(self, op):
- ofs_loc, size_loc, ptr = self._unpack_fielddescr(op.getdescr())
+ ofs_loc, size_loc, _, _ = self._unpack_fielddescr(op.getdescr())
assert isinstance(size_loc, ImmedLoc)
if size_loc.value == 1:
need_lower_byte = True
@@ -866,10 +870,10 @@
consider_unicodesetitem = consider_strsetitem
def consider_setarrayitem_gc(self, op):
- scale, ofs, _, ptr = self._unpack_arraydescr(op.getdescr())
+ itemsize, ofs, _, _, _ = self._unpack_arraydescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
- if scale == 0:
+ if itemsize == 1:
need_lower_byte = True
else:
need_lower_byte = False
@@ -878,30 +882,39 @@
ofs_loc = self.rm.make_sure_var_in_reg(op.getarg(1), args)
self.possibly_free_vars(args)
self.PerformDiscard(op, [base_loc, ofs_loc, value_loc,
- imm(scale), imm(ofs)])
+ imm(itemsize), imm(ofs)])
consider_setarrayitem_raw = consider_setarrayitem_gc
def consider_getfield_gc(self, op):
- ofs_loc, size_loc, _ = self._unpack_fielddescr(op.getdescr())
+ ofs_loc, size_loc, _, sign = self._unpack_fielddescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
self.rm.possibly_free_vars(args)
result_loc = self.force_allocate_reg(op.result)
- self.Perform(op, [base_loc, ofs_loc, size_loc], result_loc)
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self.Perform(op, [base_loc, ofs_loc, size_loc, sign_loc], result_loc)
consider_getfield_raw = consider_getfield_gc
consider_getfield_raw_pure = consider_getfield_gc
consider_getfield_gc_pure = consider_getfield_gc
def consider_getarrayitem_gc(self, op):
- scale, ofs, _, _ = self._unpack_arraydescr(op.getdescr())
+ itemsize, ofs, _, _, sign = self._unpack_arraydescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
ofs_loc = self.rm.make_sure_var_in_reg(op.getarg(1), args)
self.rm.possibly_free_vars_for_op(op)
result_loc = self.force_allocate_reg(op.result)
- self.Perform(op, [base_loc, ofs_loc, imm(scale), imm(ofs)], result_loc)
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self.Perform(op, [base_loc, ofs_loc, imm(itemsize), imm(ofs),
+ sign_loc], result_loc)
consider_getarrayitem_raw = consider_getarrayitem_gc
consider_getarrayitem_gc_pure = consider_getarrayitem_gc
@@ -1090,15 +1103,11 @@
self.Perform(op, [], loc)
def not_implemented_op(self, op):
- msg = "[regalloc] Not implemented operation: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation: %s" % op.getopname())
def not_implemented_op_with_guard(self, op, guard_op):
- msg = "[regalloc] Not implemented operation with guard: %s" % (
- op.getopname(),)
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation with guard: %s" % (
+ op.getopname(),))
oplist = [RegAlloc.not_implemented_op] * rop._LAST
oplist_with_guard = [RegAlloc.not_implemented_op_with_guard] * rop._LAST
@@ -1132,3 +1141,14 @@
# Returns (ebp-20), (ebp-24), (ebp-28)...
# i.e. the n'th word beyond the fixed frame size.
return -WORD * (FRAME_FIXED_SIZE + position)
+
+def _get_scale(size):
+ assert size == 1 or size == 2 or size == 4 or size == 8
+ if size < 4:
+ return size - 1 # 1, 2 => 0, 1
+ else:
+ return (size >> 2) + 1 # 4, 8 => 2, 3
+
+def not_implemented(msg):
+ os.write(2, '[x86/regalloc] %s\n' % msg)
+ raise NotImplementedError(msg)
Modified: pypy/branch/jit-sign-descr/pypy/jit/backend/x86/regloc.py
==============================================================================
--- pypy/branch/jit-sign-descr/pypy/jit/backend/x86/regloc.py (original)
+++ pypy/branch/jit-sign-descr/pypy/jit/backend/x86/regloc.py Wed Oct 20 11:46:04 2010
@@ -442,8 +442,11 @@
MOV8 = _binaryop('MOV8')
MOV16 = _16_bit_binaryop('MOV')
MOVZX8 = _binaryop('MOVZX8')
+ MOVSX8 = _binaryop('MOVSX8')
MOVZX16 = _binaryop('MOVZX16')
+ MOVSX16 = _binaryop('MOVSX16')
MOV32 = _binaryop('MOV32')
+ MOVSX32 = _binaryop('MOVSX32')
XCHG = _binaryop('XCHG')
PUSH = _unaryop('PUSH')
@@ -473,6 +476,9 @@
else:
return ImmedLoc(x)
+imm0 = imm(0)
+imm1 = imm(1)
+
all_extra_instructions = [name for name in LocationCodeBuilder.__dict__
if name[0].isupper()]
all_extra_instructions.sort()
Modified: pypy/branch/jit-sign-descr/pypy/jit/backend/x86/rx86.py
==============================================================================
--- pypy/branch/jit-sign-descr/pypy/jit/backend/x86/rx86.py (original)
+++ pypy/branch/jit-sign-descr/pypy/jit/backend/x86/rx86.py Wed Oct 20 11:46:04 2010
@@ -642,7 +642,10 @@
define_modrm_modes('MOV8_*i', [rex_w, '\xC6', orbyte(0<<3)], [immediate(2, 'b')], regtype='BYTE')
define_modrm_modes('MOVZX8_r*', [rex_w, '\x0F\xB6', register(1, 8)], regtype='BYTE')
+define_modrm_modes('MOVSX8_r*', [rex_w, '\x0F\xBE', register(1, 8)], regtype='BYTE')
define_modrm_modes('MOVZX16_r*', [rex_w, '\x0F\xB7', register(1, 8)])
+define_modrm_modes('MOVSX16_r*', [rex_w, '\x0F\xBF', register(1, 8)])
+define_modrm_modes('MOVSX32_r*', [rex_w, '\x63', register(1, 8)])
define_modrm_modes('MOVSD_x*', ['\xF2', rex_nw, '\x0F\x10', register(1,8)], regtype='XMM')
define_modrm_modes('MOVSD_*x', ['\xF2', rex_nw, '\x0F\x11', register(2,8)], regtype='XMM')
From arigo at codespeak.net Wed Oct 20 11:53:59 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 11:53:59 +0200 (CEST)
Subject: [pypy-svn] r78116 - pypy/trunk/pypy/module/pypyjit/test
Message-ID: <20101020095359.21B4F282BDA@codespeak.net>
Author: arigo
Date: Wed Oct 20 11:53:57 2010
New Revision: 78116
Modified:
pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
Log:
Baaah. This was a CALL_METHOD instead of a CALL_FUNCTION.
Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Wed Oct 20 11:53:57 2010
@@ -560,13 +560,13 @@
def test_blockstack_virtualizable(self):
self.run_source('''
- import pypyjit
+ from pypyjit import residual_call
def main():
i = 0
while i < 100:
try:
- pypyjit.residual_call(len, [])
+ residual_call(len, [])
except:
pass
i += 1
From antocuni at codespeak.net Wed Oct 20 12:01:04 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Wed, 20 Oct 2010 12:01:04 +0200 (CEST)
Subject: [pypy-svn] r78117 - pypy/trunk/pypy/module/imp/test
Message-ID: <20101020100104.2AC2936E3FB@codespeak.net>
Author: antocuni
Date: Wed Oct 20 12:01:02 2010
New Revision: 78117
Modified:
pypy/trunk/pypy/module/imp/test/test_import.py
Log:
make sure that LANG is unset when calling py.py through py.process.cmdexec.
This is a workaround for pylib bug #130:
http://bitbucket.org/hpk42/py-trunk/issue/130/pyprocesscmdexec-fails-if-the-out-err-contains-non-ascii
Modified: pypy/trunk/pypy/module/imp/test/test_import.py
==============================================================================
--- pypy/trunk/pypy/module/imp/test/test_import.py (original)
+++ pypy/trunk/pypy/module/imp/test/test_import.py Wed Oct 20 12:01:02 2010
@@ -790,6 +790,7 @@
extrapath = udir.ensure("pythonpath", dir=1)
extrapath.join("urllib.py").write("print 42\n")
old = os.environ.get('PYTHONPATH', None)
+ oldlang = os.environ.pop('LANG', None)
try:
os.environ['PYTHONPATH'] = str(extrapath)
output = py.process.cmdexec('''"%s" "%s" -c "import urllib"''' %
@@ -798,6 +799,8 @@
finally:
if old:
os.environ['PYTHONPATH'] = old
+ if oldlang:
+ os.environ['LANG'] = oldlang
class AppTestImportHooks(object):
def test_meta_path(self):
From antocuni at codespeak.net Wed Oct 20 12:01:17 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Wed, 20 Oct 2010 12:01:17 +0200 (CEST)
Subject: [pypy-svn] r78118 - pypy/branch/debug-test_import
Message-ID: <20101020100117.72B73282BDA@codespeak.net>
Author: antocuni
Date: Wed Oct 20 12:01:16 2010
New Revision: 78118
Removed:
pypy/branch/debug-test_import/
Log:
remove merged branch
From arigo at codespeak.net Wed Oct 20 13:03:15 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 13:03:15 +0200 (CEST)
Subject: [pypy-svn] r78119 -
pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests
Message-ID: <20101020110315.3BC0536E3FB@codespeak.net>
Author: arigo
Date: Wed Oct 20 13:03:13 2010
New Revision: 78119
Removed:
pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/test_find.py
Log:
Revert r52839: this (unmodified) test file from ctypes fails
on some platform (in this case tannit64) because it does not
have the opengl libraries.
From antocuni at codespeak.net Wed Oct 20 13:04:03 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Wed, 20 Oct 2010 13:04:03 +0200 (CEST)
Subject: [pypy-svn] r78120 - in pypy/trunk/pypy: rlib rpython
Message-ID: <20101020110403.CCBB5282BDA@codespeak.net>
Author: antocuni
Date: Wed Oct 20 13:04:02 2010
New Revision: 78120
Modified:
pypy/trunk/pypy/rlib/clibffi.py
pypy/trunk/pypy/rpython/llinterp.py
Log:
(antocuni, arigo): don't track these mallocs, they are automatically freed in the __del__
Modified: pypy/trunk/pypy/rlib/clibffi.py
==============================================================================
--- pypy/trunk/pypy/rlib/clibffi.py (original)
+++ pypy/trunk/pypy/rlib/clibffi.py Wed Oct 20 13:04:02 2010
@@ -385,10 +385,12 @@
self.restype = restype
self.flags = flags
argnum = len(argtypes)
- self.ll_argtypes = lltype.malloc(FFI_TYPE_PP.TO, argnum, flavor='raw')
+ self.ll_argtypes = lltype.malloc(FFI_TYPE_PP.TO, argnum, flavor='raw',
+ track_allocation=False) # freed by the __del__
for i in range(argnum):
self.ll_argtypes[i] = argtypes[i]
- self.ll_cif = lltype.malloc(FFI_CIFP.TO, flavor='raw')
+ self.ll_cif = lltype.malloc(FFI_CIFP.TO, flavor='raw',
+ track_allocation=False) # freed by the __del__
if _WIN32 and (flags & FUNCFLAG_CDECL == 0):
cc = FFI_STDCALL
Modified: pypy/trunk/pypy/rpython/llinterp.py
==============================================================================
--- pypy/trunk/pypy/rpython/llinterp.py (original)
+++ pypy/trunk/pypy/rpython/llinterp.py Wed Oct 20 13:04:02 2010
@@ -717,9 +717,11 @@
def op_malloc_varsize(self, obj, flags, size):
flavor = flags['flavor']
zero = flags.get('zero', False)
+ track_allocation = flags.get('track_allocation', True)
assert flavor in ('gc', 'raw')
try:
- ptr = self.heap.malloc(obj, size, zero=zero, flavor=flavor)
+ ptr = self.heap.malloc(obj, size, zero=zero, flavor=flavor,
+ track_allocation=track_allocation)
return ptr
except MemoryError:
self.make_llexception()
From afa at codespeak.net Wed Oct 20 13:21:11 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 13:21:11 +0200 (CEST)
Subject: [pypy-svn] r78121 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101020112111.1312E36E3FF@codespeak.net>
Author: afa
Date: Wed Oct 20 13:21:09 2010
New Revision: 78121
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
Log:
Add FileIO.closefd attribute
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Wed Oct 20 13:21:09 2010
@@ -1,6 +1,6 @@
from pypy.module._io.interp_iobase import W_RawIOBase
from pypy.interpreter.typedef import (
- TypeDef, interp_attrproperty_w, GetSetProperty)
+ TypeDef, interp_attrproperty, interp_attrproperty_w, GetSetProperty)
from pypy.interpreter.gateway import interp2app, unwrap_spec, Arguments
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
from pypy.interpreter.error import OperationError, wrap_oserror, wrap_oserror2
@@ -366,6 +366,7 @@
fileno = interp2app(W_FileIO.fileno_w),
isatty = interp2app(W_FileIO.isatty_w),
name = interp_attrproperty_w('w_name', cls=W_FileIO),
+ closefd = interp_attrproperty('closefd', cls=W_FileIO),
mode = GetSetProperty(W_FileIO.descr_get_mode),
)
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py Wed Oct 20 13:21:09 2010
@@ -18,15 +18,18 @@
f = _io.FileIO(self.tmpfile, 'a')
assert f.name.endswith('tmpfile')
assert f.mode == 'wb'
+ assert f.closefd is True
f.close()
def test_open_fd(self):
import _io
os = self.posix
fd = os.open(self.tmpfile, os.O_RDONLY, 0666)
- f = _io.FileIO(fd, "rb")
+ f = _io.FileIO(fd, "rb", closefd=False)
assert f.fileno() == fd
+ assert f.closefd is False
f.close()
+ os.close(fd)
def test_open_directory(self):
import _io
From afa at codespeak.net Wed Oct 20 13:27:35 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 13:27:35 +0200 (CEST)
Subject: [pypy-svn] r78122 - in pypy/branch/fast-forward/pypy/objspace/std:
. test
Message-ID: <20101020112735.9C964282BDA@codespeak.net>
Author: afa
Date: Wed Oct 20 13:27:34 2010
New Revision: 78122
Modified:
pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py
Log:
Implement bytearray.__iadd__, bytearray item/slice assignment.
Modified: pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py Wed Oct 20 13:27:34 2010
@@ -359,15 +359,20 @@
def list_append__Bytearray_ANY(space, w_bytearray, w_item):
from pypy.objspace.std.bytearraytype import getbytevalue
w_bytearray.data.append(getbytevalue(space, w_item))
- return space.w_None
def list_extend__Bytearray_Bytearray(space, w_bytearray, w_other):
w_bytearray.data += w_other.data
- return space.w_None
def list_extend__Bytearray_ANY(space, w_bytearray, w_other):
w_bytearray.data += [c for c in space.str_w(w_other)]
- return space.w_None
+
+def inplace_add__Bytearray_Bytearray(space, w_bytearray1, w_bytearray2):
+ list_extend__Bytearray_Bytearray(space, w_bytearray1, w_bytearray2)
+ return w_bytearray1
+
+def inplace_add__Bytearray_ANY(space, w_bytearray1, w_iterable2):
+ list_extend__Bytearray_ANY(space, w_bytearray1, w_iterable2)
+ return w_bytearray1
def delslice__Bytearray_ANY_ANY(space, w_bytearray, w_start, w_stop):
length = len(w_bytearray.data)
@@ -376,5 +381,24 @@
return
del w_bytearray.data[start:stop]
+def setitem__Bytearray_ANY_ANY(space, w_bytearray, w_index, w_item):
+ from pypy.objspace.std.bytearraytype import getbytevalue
+ idx = space.getindex_w(w_index, space.w_IndexError, "bytearray index")
+ try:
+ w_bytearray.data[idx] = getbytevalue(space, w_item)
+ except IndexError:
+ raise OperationError(space.w_IndexError,
+ space.wrap("bytearray index out of range"))
+
+def setitem__Bytearray_Slice_ANY(space, w_bytearray, w_slice, w_other):
+ oldsize = len(w_bytearray.data)
+ start, stop, step, slicelength = w_slice.indices4(space, oldsize)
+ if step != 1:
+ raise OperationError(space.w_NotImplementedError,
+ space.wrap("fixme: only step=1 for the moment"))
+ assert start >= 0
+ assert stop >= 0
+ w_bytearray.data[start:stop] = [c for c in space.str_w(w_other)]
+
from pypy.objspace.std import bytearraytype
register_all(vars(), bytearraytype)
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py Wed Oct 20 13:27:34 2010
@@ -143,6 +143,12 @@
b.append(ord('e'))
assert b == 'abcde'
+ def test_iadd(self):
+ b = bytearray('abc')
+ b += 'def'
+ assert b == 'abcdef'
+ assert isinstance(b, bytearray)
+
def test_extend(self):
b = bytearray('abc')
b.extend(bytearray('def'))
@@ -155,3 +161,17 @@
assert b == 'abcdei'
del b[:3]
assert b == 'dei'
+
+ def test_setitem(self):
+ b = bytearray('abcdefghi')
+ b[1] = 'B'
+ assert b == 'aBcdefghi'
+
+ def test_setitem_slice(self):
+ b = bytearray('abcdefghi')
+ b[0:3] = 'ABC'
+ assert b == 'ABCdefghi'
+ b[3:3] = '...'
+ assert b == 'ABC...defghi'
+ b[3:6] = '()'
+ assert b == 'ABC()defghi'
From afa at codespeak.net Wed Oct 20 13:32:47 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 13:32:47 +0200 (CEST)
Subject: [pypy-svn] r78123 - pypy/branch/fast-forward/pypy/module/_io
Message-ID: <20101020113247.92D12282BDA@codespeak.net>
Author: afa
Date: Wed Oct 20 13:32:46 2010
New Revision: 78123
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_io.py
Log:
Add default constructors to all io classes
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_io.py Wed Oct 20 13:32:46 2010
@@ -45,36 +45,42 @@
pass
W_BytesIO.typedef = TypeDef(
'BytesIO', W_BufferedIOBase.typedef,
+ __new__ = generic_new_descr(W_BytesIO),
)
class W_BufferedReader(W_BufferedIOBase):
pass
W_BufferedReader.typedef = TypeDef(
'BufferedReader', W_BufferedIOBase.typedef,
+ __new__ = generic_new_descr(W_BufferedReader),
)
class W_BufferedWriter(W_BufferedIOBase):
pass
W_BufferedWriter.typedef = TypeDef(
'BufferedWriter', W_BufferedIOBase.typedef,
+ __new__ = generic_new_descr(W_BufferedWriter),
)
class W_BufferedRWPair(W_BufferedIOBase):
pass
W_BufferedRWPair.typedef = TypeDef(
'BufferedRWPair', W_BufferedIOBase.typedef,
+ __new__ = generic_new_descr(W_BufferedRWPair),
)
class W_BufferedRandom(W_BufferedIOBase):
pass
W_BufferedRandom.typedef = TypeDef(
'BufferedRandom', W_BufferedIOBase.typedef,
+ __new__ = generic_new_descr(W_BufferedRandom),
)
class W_TextIOWrapper(W_TextIOBase):
pass
W_TextIOWrapper.typedef = TypeDef(
'TextIOWrapper', W_TextIOBase.typedef,
+ __new__ = generic_new_descr(W_TextIOWrapper),
)
@unwrap_spec(ObjSpace, Arguments)
From arigo at codespeak.net Wed Oct 20 13:33:38 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 13:33:38 +0200 (CEST)
Subject: [pypy-svn] r78124 - in pypy/trunk/pypy/jit/backend: llsupport
llsupport/test test x86
Message-ID: <20101020113338.8E6C9282BDA@codespeak.net>
Author: arigo
Date: Wed Oct 20 13:33:36 2010
New Revision: 78124
Modified:
pypy/trunk/pypy/jit/backend/llsupport/descr.py
pypy/trunk/pypy/jit/backend/llsupport/ffisupport.py
pypy/trunk/pypy/jit/backend/llsupport/llmodel.py
pypy/trunk/pypy/jit/backend/llsupport/symbolic.py
pypy/trunk/pypy/jit/backend/llsupport/test/test_descr.py
pypy/trunk/pypy/jit/backend/llsupport/test/test_ffisupport.py
pypy/trunk/pypy/jit/backend/test/runner_test.py
pypy/trunk/pypy/jit/backend/x86/assembler.py
pypy/trunk/pypy/jit/backend/x86/regalloc.py
pypy/trunk/pypy/jit/backend/x86/regloc.py
pypy/trunk/pypy/jit/backend/x86/rx86.py
Log:
Merge branch/jit-sign-descr, minus the change to rlib/libffi that will
be redone next. Implement support for signed shorter-than-LONG integers
in all descrs of the JIT backend, namely for getfield(), getarrayitem()
and call().
Modified: pypy/trunk/pypy/jit/backend/llsupport/descr.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llsupport/descr.py (original)
+++ pypy/trunk/pypy/jit/backend/llsupport/descr.py Wed Oct 20 13:33:36 2010
@@ -82,6 +82,7 @@
_is_pointer_field = False # unless overridden by GcPtrFieldDescr
_is_float_field = False # unless overridden by FloatFieldDescr
+ _is_field_signed = False # unless overridden by XxxFieldDescr
def is_pointer_field(self):
return self._is_pointer_field
@@ -89,6 +90,9 @@
def is_float_field(self):
return self._is_float_field
+ def is_field_signed(self):
+ return self._is_field_signed
+
def repr_of_descr(self):
return '<%s %s %s>' % (self._clsname, self.name, self.offset)
@@ -105,7 +109,7 @@
def getFieldDescrClass(TYPE):
return getDescrClass(TYPE, BaseFieldDescr, GcPtrFieldDescr,
NonGcPtrFieldDescr, 'Field', 'get_field_size',
- '_is_float_field')
+ '_is_float_field', '_is_field_signed')
def get_field_descr(gccache, STRUCT, fieldname):
cache = gccache._cache_field
@@ -144,6 +148,7 @@
_is_array_of_pointers = False # unless overridden by GcPtrArrayDescr
_is_array_of_floats = False # unless overridden by FloatArrayDescr
+ _is_item_signed = False # unless overridden by XxxArrayDescr
def is_array_of_pointers(self):
return self._is_array_of_pointers
@@ -151,6 +156,9 @@
def is_array_of_floats(self):
return self._is_array_of_floats
+ def is_item_signed(self):
+ return self._is_item_signed
+
def repr_of_descr(self):
return '<%s>' % self._clsname
@@ -186,12 +194,12 @@
def getArrayDescrClass(ARRAY):
return getDescrClass(ARRAY.OF, BaseArrayDescr, GcPtrArrayDescr,
NonGcPtrArrayDescr, 'Array', 'get_item_size',
- '_is_array_of_floats')
+ '_is_array_of_floats', '_is_item_signed')
def getArrayNoLengthDescrClass(ARRAY):
return getDescrClass(ARRAY.OF, BaseArrayNoLengthDescr, GcPtrArrayNoLengthDescr,
NonGcPtrArrayNoLengthDescr, 'ArrayNoLength', 'get_item_size',
- '_is_array_of_floats')
+ '_is_array_of_floats', '_is_item_signed')
def get_array_descr(gccache, ARRAY):
cache = gccache._cache_array
@@ -242,6 +250,9 @@
def get_result_size(self, translate_support_code):
raise NotImplementedError
+ def is_result_signed(self):
+ return False # unless overridden
+
def create_call_stub(self, rtyper, RESULT):
def process(c):
arg = 'args_%s[%d]' % (c, seen[c])
@@ -307,6 +318,10 @@
_return_type = history.INT
call_stub = staticmethod(lambda func, args_i, args_r, args_f: 0)
+ _is_result_signed = False # can be overridden in XxxCallDescr
+ def is_result_signed(self):
+ return self._is_result_signed
+
class DynamicIntCallDescr(BaseIntCallDescr):
"""
calldescr that works for every integer type, by explicitly passing it the
@@ -314,12 +329,17 @@
"""
_clsname = 'DynamicIntCallDescr'
- def __init__(self, arg_classes, result_size, extrainfo=None):
+ def __init__(self, arg_classes, result_size, result_sign, extrainfo=None):
BaseIntCallDescr.__init__(self, arg_classes, extrainfo)
- self._result_size = result_size
+ assert isinstance(result_sign, bool)
+ self._result_size = chr(result_size)
+ self._result_sign = result_sign
def get_result_size(self, translate_support_code):
- return self._result_size
+ return ord(self._result_size)
+
+ def is_result_signed(self):
+ return self._result_sign
class NonGcPtrCallDescr(BaseIntCallDescr):
@@ -356,7 +376,8 @@
return FloatCallDescr
return getDescrClass(RESULT, BaseIntCallDescr, GcPtrCallDescr,
NonGcPtrCallDescr, 'Call', 'get_result_size',
- Ellipsis) # <= floatattrname should not be used here
+ Ellipsis, # <= floatattrname should not be used here
+ '_is_result_signed')
def get_call_descr(gccache, ARGS, RESULT, extrainfo=None):
arg_classes = []
@@ -383,7 +404,8 @@
# ____________________________________________________________
def getDescrClass(TYPE, BaseDescr, GcPtrDescr, NonGcPtrDescr,
- nameprefix, methodname, floatattrname, _cache={}):
+ nameprefix, methodname, floatattrname, signedattrname,
+ _cache={}):
if isinstance(TYPE, lltype.Ptr):
if TYPE.TO._gckind == 'gc':
return GcPtrDescr
@@ -403,6 +425,8 @@
#
if TYPE is lltype.Float:
setattr(Descr, floatattrname, True)
+ elif TYPE is not lltype.Bool and rffi.cast(TYPE, -1) == -1:
+ setattr(Descr, signedattrname, True)
#
_cache[nameprefix, TYPE] = Descr
return Descr
Modified: pypy/trunk/pypy/jit/backend/llsupport/ffisupport.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llsupport/ffisupport.py (original)
+++ pypy/trunk/pypy/jit/backend/llsupport/ffisupport.py Wed Oct 20 13:33:36 2010
@@ -14,7 +14,8 @@
arg_classes = ''.join(argkinds)
if reskind == history.INT:
size = intmask(ffi_result.c_size)
- return DynamicIntCallDescr(arg_classes, size, extrainfo)
+ signed = is_ffi_type_signed(ffi_result)
+ return DynamicIntCallDescr(arg_classes, size, signed, extrainfo)
elif reskind == history.REF:
return NonGcPtrCallDescr(arg_classes, extrainfo)
elif reskind == history.FLOAT:
@@ -26,10 +27,15 @@
def get_ffi_type_kind(ffi_type):
from pypy.rlib.libffi import types
kind = types.getkind(ffi_type)
- if kind == 'i':
+ if kind == 'i' or kind == 'u':
return history.INT
elif kind == 'f':
return history.FLOAT
elif kind == 'v':
return history.VOID
- assert False, "Unsuported kind '%s'" % kind
+ assert False, "Unsupported kind '%s'" % kind
+
+def is_ffi_type_signed(ffi_type):
+ from pypy.rlib.libffi import types
+ kind = types.getkind(ffi_type)
+ return kind != 'u'
Modified: pypy/trunk/pypy/jit/backend/llsupport/llmodel.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llsupport/llmodel.py (original)
+++ pypy/trunk/pypy/jit/backend/llsupport/llmodel.py Wed Oct 20 13:33:36 2010
@@ -212,7 +212,8 @@
assert isinstance(fielddescr, BaseFieldDescr)
ofs = fielddescr.offset
size = fielddescr.get_field_size(self.translate_support_code)
- return ofs, size
+ sign = fielddescr.is_field_signed()
+ return ofs, size, sign
unpack_fielddescr_size._always_inline_ = True
def arraydescrof(self, A):
@@ -227,7 +228,8 @@
assert isinstance(arraydescr, BaseArrayDescr)
ofs = arraydescr.get_base_size(self.translate_support_code)
size = arraydescr.get_item_size(self.translate_support_code)
- return ofs, size
+ sign = arraydescr.is_item_signed()
+ return ofs, size, sign
unpack_arraydescr_size._always_inline_ = True
def calldescrof(self, FUNC, ARGS, RESULT, extrainfo=None):
@@ -257,15 +259,21 @@
@specialize.argtype(2)
def bh_getarrayitem_gc_i(self, arraydescr, gcref, itemindex):
- ofs, size = self.unpack_arraydescr_size(arraydescr)
+ ofs, size, sign = self.unpack_arraydescr_size(arraydescr)
# --- start of GC unsafe code (no GC operation!) ---
items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for STYPE, UTYPE, itemsize in unroll_basic_sizes:
if size == itemsize:
- items = rffi.cast(rffi.CArrayPtr(TYPE), items)
- val = items[itemindex]
+ if sign:
+ items = rffi.cast(rffi.CArrayPtr(STYPE), items)
+ val = items[itemindex]
+ val = rffi.cast(lltype.Signed, val)
+ else:
+ items = rffi.cast(rffi.CArrayPtr(UTYPE), items)
+ val = items[itemindex]
+ val = rffi.cast(lltype.Signed, val)
# --- end of GC unsafe code ---
- return rffi.cast(lltype.Signed, val)
+ return val
else:
raise NotImplementedError("size = %d" % size)
@@ -290,10 +298,10 @@
@specialize.argtype(2)
def bh_setarrayitem_gc_i(self, arraydescr, gcref, itemindex, newvalue):
- ofs, size = self.unpack_arraydescr_size(arraydescr)
+ ofs, size, sign = self.unpack_arraydescr_size(arraydescr)
# --- start of GC unsafe code (no GC operation!) ---
items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for TYPE, _, itemsize in unroll_basic_sizes:
if size == itemsize:
items = rffi.cast(rffi.CArrayPtr(TYPE), items)
items[itemindex] = rffi.cast(TYPE, newvalue)
@@ -344,14 +352,22 @@
@specialize.argtype(1)
def _base_do_getfield_i(self, struct, fielddescr):
- ofs, size = self.unpack_fielddescr_size(fielddescr)
+ ofs, size, sign = self.unpack_fielddescr_size(fielddescr)
# --- start of GC unsafe code (no GC operation!) ---
fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for STYPE, UTYPE, itemsize in unroll_basic_sizes:
if size == itemsize:
- val = rffi.cast(rffi.CArrayPtr(TYPE), fieldptr)[0]
+ # Note that in the common case where size==sizeof(Signed),
+ # both cases of what follows are doing the same thing.
+ # But gcc is clever enough to figure this out :-)
+ if sign:
+ val = rffi.cast(rffi.CArrayPtr(STYPE), fieldptr)[0]
+ val = rffi.cast(lltype.Signed, val)
+ else:
+ val = rffi.cast(rffi.CArrayPtr(UTYPE), fieldptr)[0]
+ val = rffi.cast(lltype.Signed, val)
# --- end of GC unsafe code ---
- return rffi.cast(lltype.Signed, val)
+ return val
else:
raise NotImplementedError("size = %d" % size)
@@ -383,10 +399,10 @@
@specialize.argtype(1)
def _base_do_setfield_i(self, struct, fielddescr, newvalue):
- ofs, size = self.unpack_fielddescr_size(fielddescr)
+ ofs, size, sign = self.unpack_fielddescr_size(fielddescr)
# --- start of GC unsafe code (no GC operation!) ---
fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for TYPE, _, itemsize in unroll_basic_sizes:
if size == itemsize:
fieldptr = rffi.cast(rffi.CArrayPtr(TYPE), fieldptr)
fieldptr[0] = rffi.cast(TYPE, newvalue)
Modified: pypy/trunk/pypy/jit/backend/llsupport/symbolic.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llsupport/symbolic.py (original)
+++ pypy/trunk/pypy/jit/backend/llsupport/symbolic.py Wed Oct 20 13:33:36 2010
@@ -69,8 +69,9 @@
SIZEOF_INT = get_size(rffi.INT, False)
SIZEOF_FLOAT = get_size(lltype.Float, False)
-unroll_basic_sizes = unrolling_iterable([(lltype.Signed, WORD),
- (lltype.Char, SIZEOF_CHAR),
- (rffi.SHORT, SIZEOF_SHORT),
- (rffi.INT, SIZEOF_INT)])
+unroll_basic_sizes = unrolling_iterable([
+ (lltype.Signed, lltype.Unsigned, WORD),
+ (rffi.SIGNEDCHAR, lltype.Char, SIZEOF_CHAR),
+ (rffi.SHORT, rffi.USHORT, SIZEOF_SHORT),
+ (rffi.INT, rffi.UINT, SIZEOF_INT)])
# does not contain Float ^^^ which must be special-cased
Modified: pypy/trunk/pypy/jit/backend/llsupport/test/test_descr.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llsupport/test/test_descr.py (original)
+++ pypy/trunk/pypy/jit/backend/llsupport/test/test_descr.py Wed Oct 20 13:33:36 2010
@@ -83,6 +83,18 @@
assert descr_f.is_float_field()
+def test_get_field_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ descr_x = get_field_descr(c2, S, 'x')
+ assert descr_x.is_field_signed() == signed
+
+
def test_get_array_descr():
U = lltype.Struct('U')
T = lltype.GcStruct('T')
@@ -164,6 +176,25 @@
assert descr.get_base_size(False) == 0
assert descr.get_ofs_length(False) == -1
+
+def test_get_array_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ A = lltype.GcArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ arraydescr = get_array_descr(c2, A)
+ assert arraydescr.is_item_signed() == signed
+ #
+ RA = rffi.CArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ arraydescr = get_array_descr(c2, RA)
+ assert arraydescr.is_item_signed() == signed
+
+
def test_get_call_descr_not_translated():
c0 = GcCache(False)
descr1 = get_call_descr(c0, [lltype.Char, lltype.Signed], lltype.Char)
@@ -219,6 +250,17 @@
extrainfo = descr3.get_extra_info()
assert extrainfo is None
+def test_get_call_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ A = lltype.GcArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ descr1 = get_call_descr(c2, [], RESTYPE)
+ assert descr1.is_result_signed() == signed
+
def test_repr_of_descr():
c0 = GcCache(False)
Modified: pypy/trunk/pypy/jit/backend/llsupport/test/test_ffisupport.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llsupport/test/test_ffisupport.py (original)
+++ pypy/trunk/pypy/jit/backend/llsupport/test/test_ffisupport.py Wed Oct 20 13:33:36 2010
@@ -12,6 +12,12 @@
descr = get_call_descr_dynamic([], types.sint8)
assert isinstance(descr, DynamicIntCallDescr)
assert descr.get_result_size(False) == 1
+ assert descr.is_result_signed() == True
+
+ descr = get_call_descr_dynamic([], types.uint8)
+ assert isinstance(descr, DynamicIntCallDescr)
+ assert descr.get_result_size(False) == 1
+ assert descr.is_result_signed() == False
descr = get_call_descr_dynamic([], types.float)
assert descr is None # single floats are not supported so far
Modified: pypy/trunk/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/trunk/pypy/jit/backend/test/runner_test.py Wed Oct 20 13:33:36 2010
@@ -15,6 +15,7 @@
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.llinterp import LLException
from pypy.jit.codewriter import heaptracker
+from pypy.rlib.rarithmetic import intmask
class Runner(object):
@@ -2028,6 +2029,196 @@
assert self.cpu.get_latest_value_float(0) == 13.5
assert called
+ def test_short_result_of_getfield_direct(self):
+ # Test that a getfield that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ descrfld_x = cpu.fielddescrof(S, 'x')
+ s = lltype.malloc(S)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ s.x = rffi.cast(RESTYPE, value)
+ x = cpu.bh_getfield_gc_i(lltype.cast_opaque_ptr(llmemory.GCREF, s),
+ descrfld_x)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_getfield_compiled(self):
+ # Test that a getfield that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ descrfld_x = cpu.fielddescrof(S, 'x')
+ s = lltype.malloc(S)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ s.x = rffi.cast(RESTYPE, value)
+ s_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
+ res = self.execute_operation(rop.GETFIELD_GC, [BoxPtr(s_gcref)],
+ 'int', descr=descrfld_x)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
+ def test_short_result_of_getarrayitem_direct(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = lltype.GcArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ x = cpu.bh_getarrayitem_gc_i(
+ descrarray, lltype.cast_opaque_ptr(llmemory.GCREF, a), 3)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_getarrayitem_compiled(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = lltype.GcArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, a)
+ res = self.execute_operation(rop.GETARRAYITEM_GC,
+ [BoxPtr(a_gcref), BoxInt(3)],
+ 'int', descr=descrarray)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
+ def test_short_result_of_getarrayitem_raw_direct(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = rffi.CArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5, flavor='raw')
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_rawint = heaptracker.adr2int(llmemory.cast_ptr_to_adr(a))
+ x = cpu.bh_getarrayitem_raw_i(descrarray, a_rawint, 3)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+ lltype.free(a, flavor='raw')
+
+ def test_short_result_of_getarrayitem_raw_compiled(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = rffi.CArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5, flavor='raw')
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_rawint = heaptracker.adr2int(llmemory.cast_ptr_to_adr(a))
+ res = self.execute_operation(rop.GETARRAYITEM_RAW,
+ [BoxInt(a_rawint), BoxInt(3)],
+ 'int', descr=descrarray)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+ lltype.free(a, flavor='raw')
+
+ def test_short_result_of_call_direct(self):
+ # Test that calling a function that returns a CHAR, SHORT or INT,
+ # signed or unsigned, properly gets zero-extended or sign-extended.
+ from pypy.translator.tool.cbuild import ExternalCompilationInfo
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ # Tested with a function that intentionally does not cast the
+ # result to RESTYPE, but makes sure that we return the whole
+ # value in eax or rax.
+ eci = ExternalCompilationInfo(separate_module_sources=["""
+ long fn_test_result_of_call(long x)
+ {
+ return x + 1;
+ }
+ """])
+ f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed],
+ RESTYPE, compilation_info=eci, _nowrapper=True)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1))
+ assert intmask(f(value)) == expected
+ #
+ FUNC = self.FuncType([lltype.Signed], RESTYPE)
+ FPTR = self.Ptr(FUNC)
+ calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+ x = self.cpu.bh_call_i(self.get_funcbox(self.cpu, f).value,
+ calldescr, [value], None, None)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_call_compiled(self):
+ # Test that calling a function that returns a CHAR, SHORT or INT,
+ # signed or unsigned, properly gets zero-extended or sign-extended.
+ from pypy.translator.tool.cbuild import ExternalCompilationInfo
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ # Tested with a function that intentionally does not cast the
+ # result to RESTYPE, but makes sure that we return the whole
+ # value in eax or rax.
+ eci = ExternalCompilationInfo(separate_module_sources=["""
+ long fn_test_result_of_call(long x)
+ {
+ return x + 1;
+ }
+ """])
+ f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed],
+ RESTYPE, compilation_info=eci, _nowrapper=True)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1))
+ assert intmask(f(value)) == expected
+ #
+ FUNC = self.FuncType([lltype.Signed], RESTYPE)
+ FPTR = self.Ptr(FUNC)
+ calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+ funcbox = self.get_funcbox(self.cpu, f)
+ res = self.execute_operation(rop.CALL, [funcbox, BoxInt(value)],
+ 'int', descr=calldescr)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
class OOtypeBackendTest(BaseBackendTest):
Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/assembler.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/assembler.py Wed Oct 20 13:33:36 2010
@@ -8,7 +8,8 @@
from pypy.rpython.annlowlevel import llhelper
from pypy.tool.uid import fixid
from pypy.jit.backend.x86.regalloc import (RegAlloc, X86RegisterManager,
- X86XMMRegisterManager, get_ebp_ofs)
+ X86XMMRegisterManager, get_ebp_ofs,
+ _get_scale)
from pypy.jit.backend.x86.arch import (FRAME_FIXED_SIZE, FORCE_INDEX_OFS, WORD,
IS_X86_32, IS_X86_64)
@@ -22,7 +23,8 @@
X86_64_SCRATCH_REG,
X86_64_XMM_SCRATCH_REG,
RegLoc, StackLoc, ConstFloatLoc,
- ImmedLoc, AddressLoc, imm)
+ ImmedLoc, AddressLoc, imm,
+ imm0, imm1)
from pypy.rlib.objectmodel import we_are_translated, specialize
from pypy.jit.backend.x86 import rx86, regloc, codebuf
@@ -442,7 +444,7 @@
# self.mc.PUSH(eax)
# adr = rffi.cast(lltype.Signed, self.loop_run_counters[-1][1])
# self.mc.MOV(eax, heap(adr))
- # self.mc.ADD(eax, imm(1))
+ # self.mc.ADD(eax, imm1)
# self.mc.MOV(heap(adr), eax)
# self.mc.POP(eax)
return operations
@@ -713,7 +715,7 @@
self.regalloc_perform_with_guard(None, guard_op, faillocs, arglocs,
resloc, current_depths)
- def load_effective_addr(self, sizereg, baseofs, scale, result, frm=imm(0)):
+ def load_effective_addr(self, sizereg, baseofs, scale, result, frm=imm0):
self.mc.LEA(result, addr_add(frm, sizereg, baseofs, scale))
def _unaryop(asmop):
@@ -975,28 +977,28 @@
def genop_guard_int_is_true(self, op, guard_op, guard_token, arglocs, resloc):
guard_opnum = guard_op.getopnum()
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
if guard_opnum == rop.GUARD_TRUE:
return self.implement_guard(guard_token, 'Z')
else:
return self.implement_guard(guard_token, 'NZ')
def genop_int_is_true(self, op, arglocs, resloc):
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
rl = resloc.lowest8bits()
self.mc.SET_ir(rx86.Conditions['NE'], rl.value)
self.mc.MOVZX8(resloc, rl)
def genop_guard_int_is_zero(self, op, guard_op, guard_token, arglocs, resloc):
guard_opnum = guard_op.getopnum()
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
if guard_opnum == rop.GUARD_TRUE:
return self.implement_guard(guard_token, 'NZ')
else:
return self.implement_guard(guard_token, 'Z')
def genop_int_is_zero(self, op, arglocs, resloc):
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
rl = resloc.lowest8bits()
self.mc.SET_ir(rx86.Conditions['E'], rl.value)
self.mc.MOVZX8(resloc, rl)
@@ -1052,50 +1054,66 @@
assert result_loc is eax
self.call(self.malloc_unicode_func_addr, arglocs, eax)
- def genop_getfield_gc(self, op, arglocs, resloc):
- base_loc, ofs_loc, size_loc = arglocs
- assert isinstance(size_loc, ImmedLoc)
+ # ----------
+
+ def load_from_mem(self, resloc, source_addr, size_loc, sign_loc):
assert isinstance(resloc, RegLoc)
size = size_loc.value
-
- source_addr = AddressLoc(base_loc, ofs_loc)
+ sign = sign_loc.value
if resloc.is_xmm:
self.mc.MOVSD(resloc, source_addr)
+ elif size == WORD:
+ self.mc.MOV(resloc, source_addr)
+ elif size == 1:
+ if sign:
+ self.mc.MOVSX8(resloc, source_addr)
+ else:
+ self.mc.MOVZX8(resloc, source_addr)
+ elif size == 2:
+ if sign:
+ self.mc.MOVSX16(resloc, source_addr)
+ else:
+ self.mc.MOVZX16(resloc, source_addr)
+ elif IS_X86_64 and size == 4:
+ if sign:
+ self.mc.MOVSX32(resloc, source_addr)
+ else:
+ self.mc.MOV32(resloc, source_addr) # zero-extending
+ else:
+ not_implemented("load_from_mem size = %d" % size)
+
+ def save_into_mem(self, dest_addr, value_loc, size_loc):
+ size = size_loc.value
+ if isinstance(value_loc, RegLoc) and value_loc.is_xmm:
+ self.mc.MOVSD(dest_addr, value_loc)
elif size == 1:
- self.mc.MOVZX8(resloc, source_addr)
+ self.mc.MOV8(dest_addr, value_loc.lowest8bits())
elif size == 2:
- self.mc.MOVZX16(resloc, source_addr)
+ self.mc.MOV16(dest_addr, value_loc)
elif size == 4:
- # MOV32 is zero-extending on 64-bit, so this is okay
- self.mc.MOV32(resloc, source_addr)
+ self.mc.MOV32(dest_addr, value_loc)
elif IS_X86_64 and size == 8:
- self.mc.MOV(resloc, source_addr)
+ self.mc.MOV(dest_addr, value_loc)
else:
- raise NotImplementedError("getfield size = %d" % size)
+ not_implemented("save_into_mem size = %d" % size)
+
+ def genop_getfield_gc(self, op, arglocs, resloc):
+ base_loc, ofs_loc, size_loc, sign_loc = arglocs
+ assert isinstance(size_loc, ImmedLoc)
+ source_addr = AddressLoc(base_loc, ofs_loc)
+ self.load_from_mem(resloc, source_addr, size_loc, sign_loc)
genop_getfield_raw = genop_getfield_gc
genop_getfield_raw_pure = genop_getfield_gc
genop_getfield_gc_pure = genop_getfield_gc
def genop_getarrayitem_gc(self, op, arglocs, resloc):
- base_loc, ofs_loc, scale, ofs = arglocs
+ base_loc, ofs_loc, size_loc, ofs, sign_loc = arglocs
assert isinstance(ofs, ImmedLoc)
- assert isinstance(scale, ImmedLoc)
- src_addr = addr_add(base_loc, ofs_loc, ofs.value, scale.value)
- if op.result.type == FLOAT:
- self.mc.MOVSD(resloc, src_addr)
- else:
- if scale.value == 0:
- self.mc.MOVZX8(resloc, src_addr)
- elif scale.value == 1:
- self.mc.MOVZX16(resloc, src_addr)
- elif scale.value == 2:
- self.mc.MOV32(resloc, src_addr)
- elif IS_X86_64 and scale.value == 3:
- self.mc.MOV(resloc, src_addr)
- else:
- print "[asmgen]getarrayitem unsupported size: %d" % scale.value
- raise NotImplementedError()
+ assert isinstance(size_loc, ImmedLoc)
+ scale = _get_scale(size_loc.value)
+ src_addr = addr_add(base_loc, ofs_loc, ofs.value, scale)
+ self.load_from_mem(resloc, src_addr, size_loc, sign_loc)
genop_getarrayitem_gc_pure = genop_getarrayitem_gc
genop_getarrayitem_raw = genop_getarrayitem_gc
@@ -1103,40 +1121,16 @@
def genop_discard_setfield_gc(self, op, arglocs):
base_loc, ofs_loc, size_loc, value_loc = arglocs
assert isinstance(size_loc, ImmedLoc)
- size = size_loc.value
dest_addr = AddressLoc(base_loc, ofs_loc)
- if isinstance(value_loc, RegLoc) and value_loc.is_xmm:
- self.mc.MOVSD(dest_addr, value_loc)
- elif IS_X86_64 and size == 8:
- self.mc.MOV(dest_addr, value_loc)
- elif size == 4:
- self.mc.MOV32(dest_addr, value_loc)
- elif size == 2:
- self.mc.MOV16(dest_addr, value_loc)
- elif size == 1:
- self.mc.MOV8(dest_addr, value_loc.lowest8bits())
- else:
- print "[asmgen]setfield addr size %d" % size
- raise NotImplementedError("Addr size %d" % size)
+ self.save_into_mem(dest_addr, value_loc, size_loc)
def genop_discard_setarrayitem_gc(self, op, arglocs):
- base_loc, ofs_loc, value_loc, scale_loc, baseofs = arglocs
+ base_loc, ofs_loc, value_loc, size_loc, baseofs = arglocs
assert isinstance(baseofs, ImmedLoc)
- assert isinstance(scale_loc, ImmedLoc)
- dest_addr = AddressLoc(base_loc, ofs_loc, scale_loc.value, baseofs.value)
- if op.getarg(2).type == FLOAT:
- self.mc.MOVSD(dest_addr, value_loc)
- else:
- if IS_X86_64 and scale_loc.value == 3:
- self.mc.MOV(dest_addr, value_loc)
- elif scale_loc.value == 2:
- self.mc.MOV32(dest_addr, value_loc)
- elif scale_loc.value == 1:
- self.mc.MOV16(dest_addr, value_loc)
- elif scale_loc.value == 0:
- self.mc.MOV8(dest_addr, value_loc.lowest8bits())
- else:
- raise NotImplementedError("scale = %d" % scale_loc.value)
+ assert isinstance(size_loc, ImmedLoc)
+ scale = _get_scale(size_loc.value)
+ dest_addr = AddressLoc(base_loc, ofs_loc, scale, baseofs.value)
+ self.save_into_mem(dest_addr, value_loc, size_loc)
def genop_discard_strsetitem(self, op, arglocs):
base_loc, ofs_loc, val_loc = arglocs
@@ -1203,7 +1197,7 @@
def genop_guard_guard_no_exception(self, ign_1, guard_op, guard_token,
locs, ign_2):
- self.mc.CMP(heap(self.cpu.pos_exception()), imm(0))
+ self.mc.CMP(heap(self.cpu.pos_exception()), imm0)
return self.implement_guard(guard_token, 'NZ')
def genop_guard_guard_exception(self, ign_1, guard_op, guard_token,
@@ -1215,8 +1209,8 @@
addr = self.implement_guard(guard_token, 'NE')
if resloc is not None:
self.mc.MOV(resloc, heap(self.cpu.pos_exc_value()))
- self.mc.MOV(heap(self.cpu.pos_exception()), imm(0))
- self.mc.MOV(heap(self.cpu.pos_exc_value()), imm(0))
+ self.mc.MOV(heap(self.cpu.pos_exception()), imm0)
+ self.mc.MOV(heap(self.cpu.pos_exc_value()), imm0)
return addr
def _gen_guard_overflow(self, guard_op, guard_token):
@@ -1226,8 +1220,8 @@
elif guard_opnum == rop.GUARD_OVERFLOW:
return self.implement_guard(guard_token, 'NO')
else:
- print "int_xxx_ovf followed by", guard_op.getopname()
- raise AssertionError
+ not_implemented("int_xxx_ovf followed by %s" %
+ guard_op.getopname())
def genop_guard_int_add_ovf(self, op, guard_op, guard_token, arglocs, result_loc):
self.genop_int_add(op, arglocs, result_loc)
@@ -1290,7 +1284,7 @@
def genop_guard_guard_nonnull_class(self, ign_1, guard_op,
guard_token, locs, ign_2):
self.mc.ensure_bytes_available(256)
- self.mc.CMP(locs[0], imm(1))
+ self.mc.CMP(locs[0], imm1)
# Patched below
self.mc.J_il8(rx86.Conditions['B'], 0)
jb_location = self.mc.get_relative_pos()
@@ -1639,25 +1633,34 @@
sizeloc = arglocs[0]
assert isinstance(sizeloc, ImmedLoc)
size = sizeloc.value
+ signloc = arglocs[1]
if isinstance(op.getarg(0), Const):
x = imm(op.getarg(0).getint())
else:
- x = arglocs[1]
+ x = arglocs[2]
if x is eax:
tmp = ecx
else:
tmp = eax
- self._emit_call(x, arglocs, 2, tmp=tmp)
+ self._emit_call(x, arglocs, 3, tmp=tmp)
+
+ if IS_X86_32 and isinstance(resloc, StackLoc) and resloc.width == 8:
+ self.mc.FSTP_b(resloc.value) # float return
+ elif size == WORD:
+ assert resloc is eax or resloc is xmm0 # a full word
+ elif size == 0:
+ pass # void return
+ else:
+ # use the code in load_from_mem to do the zero- or sign-extension
+ assert resloc is eax
+ if size == 1:
+ srcloc = eax.lowest8bits()
+ else:
+ srcloc = eax
+ self.load_from_mem(eax, srcloc, sizeloc, signloc)
- if isinstance(resloc, StackLoc) and resloc.width == 8 and IS_X86_32:
- self.mc.FSTP_b(resloc.value)
- elif size == 1:
- self.mc.AND_ri(eax.value, 0xff)
- elif size == 2:
- self.mc.AND_ri(eax.value, 0xffff)
-
def genop_guard_call_may_force(self, op, guard_op, guard_token,
arglocs, result_loc):
faildescr = guard_op.getdescr()
@@ -1810,20 +1813,16 @@
self.mc.LEA_rb(resloc.value, FORCE_INDEX_OFS)
def not_implemented_op_discard(self, op, arglocs):
- msg = "not implemented operation: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation: %s" % op.getopname())
def not_implemented_op(self, op, arglocs, resloc):
- msg = "not implemented operation with res: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation with res: %s" %
+ op.getopname())
def not_implemented_op_guard(self, op, guard_op,
failaddr, arglocs, resloc):
- msg = "not implemented operation (guard): %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation (guard): %s" %
+ op.getopname())
def mark_gc_roots(self):
gcrootmap = self.cpu.gc_ll_descr.gcrootmap
@@ -1907,3 +1906,7 @@
def heap(addr):
return AddressLoc(ImmedLoc(addr), ImmedLoc(0), 0, 0)
+
+def not_implemented(msg):
+ os.write(2, '[x86/asm] %s\n' % msg)
+ raise NotImplementedError(msg)
Modified: pypy/trunk/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/regalloc.py Wed Oct 20 13:33:36 2010
@@ -2,6 +2,7 @@
""" Register allocation scheme.
"""
+import os
from pypy.jit.metainterp.history import (Box, Const, ConstInt, ConstPtr,
ResOperation, BoxPtr,
LoopToken, INT, REF, FLOAT)
@@ -40,12 +41,10 @@
return imm(c.value)
elif isinstance(c, ConstPtr):
if we_are_translated() and c.value and rgc.can_move(c.value):
- print "convert_to_imm: ConstPtr needs special care"
- raise AssertionError
+ not_implemented("convert_to_imm: ConstPtr needs special care")
return imm(rffi.cast(lltype.Signed, c.value))
else:
- print "convert_to_imm: got a %s" % c
- raise AssertionError
+ not_implemented("convert_to_imm: got a %s" % c)
class X86_64_RegisterManager(X86RegisterManager):
# r11 omitted because it's used as scratch
@@ -360,8 +359,8 @@
if op.is_ovf():
if (operations[i + 1].getopnum() != rop.GUARD_NO_OVERFLOW and
operations[i + 1].getopnum() != rop.GUARD_OVERFLOW):
- print "int_xxx_ovf not followed by guard_(no)_overflow"
- raise AssertionError
+ not_implemented("int_xxx_ovf not followed by "
+ "guard_(no)_overflow")
return True
return False
if (operations[i + 1].getopnum() != rop.GUARD_TRUE and
@@ -413,8 +412,8 @@
arg = op.getarg(j)
if isinstance(arg, Box):
if arg not in start_live:
- print "Bogus arg in operation %d at %d" % (op.getopnum(), i)
- raise AssertionError
+ not_implemented("Bogus arg in operation %d at %d" %
+ (op.getopnum(), i))
longevity[arg] = (start_live[arg], i)
if op.is_guard():
for arg in op.getfailargs():
@@ -422,8 +421,8 @@
continue
assert isinstance(arg, Box)
if arg not in start_live:
- print "Bogus arg in guard %d at %d" % (op.getopnum(), i)
- raise AssertionError
+ not_implemented("Bogus arg in guard %d at %d" %
+ (op.getopnum(), i))
longevity[arg] = (start_live[arg], i)
for arg in inputargs:
if arg not in longevity:
@@ -668,7 +667,13 @@
assert isinstance(calldescr, BaseCallDescr)
assert len(calldescr.arg_classes) == op.numargs() - 1
size = calldescr.get_result_size(self.translate_support_code)
- self._call(op, [imm(size)] + [self.loc(op.getarg(i)) for i in range(op.numargs())],
+ sign = calldescr.is_result_signed()
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self._call(op, [imm(size), sign_loc] +
+ [self.loc(op.getarg(i)) for i in range(op.numargs())],
guard_not_forced_op=guard_not_forced_op)
def consider_call(self, op):
@@ -689,7 +694,7 @@
self.rm._sync_var(op.getarg(vable_index))
vable = self.fm.loc(op.getarg(vable_index))
else:
- vable = imm(0)
+ vable = imm0
self._call(op, [imm(size), vable] +
[self.loc(op.getarg(i)) for i in range(op.numargs())],
guard_not_forced_op=guard_op)
@@ -815,8 +820,9 @@
arglocs.append(self.loc(op.getarg(0)))
return self._call(op, arglocs)
# boehm GC (XXX kill the following code at some point)
- scale_of_field, basesize, ofs_length, _ = (
+ itemsize, basesize, ofs_length, _, _ = (
self._unpack_arraydescr(op.getdescr()))
+ scale_of_field = _get_scale(itemsize)
return self._malloc_varsize(basesize, ofs_length, scale_of_field,
op.getarg(0), op.result)
@@ -826,21 +832,19 @@
ofs = arraydescr.get_base_size(self.translate_support_code)
size = arraydescr.get_item_size(self.translate_support_code)
ptr = arraydescr.is_array_of_pointers()
- scale = 0
- while (1 << scale) < size:
- scale += 1
- assert (1 << scale) == size
- return scale, ofs, ofs_length, ptr
+ sign = arraydescr.is_item_signed()
+ return size, ofs, ofs_length, ptr, sign
def _unpack_fielddescr(self, fielddescr):
assert isinstance(fielddescr, BaseFieldDescr)
ofs = fielddescr.offset
size = fielddescr.get_field_size(self.translate_support_code)
ptr = fielddescr.is_pointer_field()
- return imm(ofs), imm(size), ptr
+ sign = fielddescr.is_field_signed()
+ return imm(ofs), imm(size), ptr, sign
def consider_setfield_gc(self, op):
- ofs_loc, size_loc, ptr = self._unpack_fielddescr(op.getdescr())
+ ofs_loc, size_loc, _, _ = self._unpack_fielddescr(op.getdescr())
assert isinstance(size_loc, ImmedLoc)
if size_loc.value == 1:
need_lower_byte = True
@@ -867,10 +871,10 @@
consider_unicodesetitem = consider_strsetitem
def consider_setarrayitem_gc(self, op):
- scale, ofs, _, ptr = self._unpack_arraydescr(op.getdescr())
+ itemsize, ofs, _, _, _ = self._unpack_arraydescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
- if scale == 0:
+ if itemsize == 1:
need_lower_byte = True
else:
need_lower_byte = False
@@ -879,30 +883,39 @@
ofs_loc = self.rm.make_sure_var_in_reg(op.getarg(1), args)
self.possibly_free_vars(args)
self.PerformDiscard(op, [base_loc, ofs_loc, value_loc,
- imm(scale), imm(ofs)])
+ imm(itemsize), imm(ofs)])
consider_setarrayitem_raw = consider_setarrayitem_gc
def consider_getfield_gc(self, op):
- ofs_loc, size_loc, _ = self._unpack_fielddescr(op.getdescr())
+ ofs_loc, size_loc, _, sign = self._unpack_fielddescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
self.rm.possibly_free_vars(args)
result_loc = self.force_allocate_reg(op.result)
- self.Perform(op, [base_loc, ofs_loc, size_loc], result_loc)
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self.Perform(op, [base_loc, ofs_loc, size_loc, sign_loc], result_loc)
consider_getfield_raw = consider_getfield_gc
consider_getfield_raw_pure = consider_getfield_gc
consider_getfield_gc_pure = consider_getfield_gc
def consider_getarrayitem_gc(self, op):
- scale, ofs, _, _ = self._unpack_arraydescr(op.getdescr())
+ itemsize, ofs, _, _, sign = self._unpack_arraydescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
ofs_loc = self.rm.make_sure_var_in_reg(op.getarg(1), args)
self.rm.possibly_free_vars_for_op(op)
result_loc = self.force_allocate_reg(op.result)
- self.Perform(op, [base_loc, ofs_loc, imm(scale), imm(ofs)], result_loc)
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self.Perform(op, [base_loc, ofs_loc, imm(itemsize), imm(ofs),
+ sign_loc], result_loc)
consider_getarrayitem_raw = consider_getarrayitem_gc
consider_getarrayitem_gc_pure = consider_getarrayitem_gc
@@ -1091,15 +1104,11 @@
self.Perform(op, [], loc)
def not_implemented_op(self, op):
- msg = "[regalloc] Not implemented operation: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation: %s" % op.getopname())
def not_implemented_op_with_guard(self, op, guard_op):
- msg = "[regalloc] Not implemented operation with guard: %s" % (
- op.getopname(),)
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation with guard: %s" % (
+ op.getopname(),))
oplist = [RegAlloc.not_implemented_op] * rop._LAST
oplist_with_guard = [RegAlloc.not_implemented_op_with_guard] * rop._LAST
@@ -1133,3 +1142,14 @@
# Returns (ebp-20), (ebp-24), (ebp-28)...
# i.e. the n'th word beyond the fixed frame size.
return -WORD * (FRAME_FIXED_SIZE + position)
+
+def _get_scale(size):
+ assert size == 1 or size == 2 or size == 4 or size == 8
+ if size < 4:
+ return size - 1 # 1, 2 => 0, 1
+ else:
+ return (size >> 2) + 1 # 4, 8 => 2, 3
+
+def not_implemented(msg):
+ os.write(2, '[x86/regalloc] %s\n' % msg)
+ raise NotImplementedError(msg)
Modified: pypy/trunk/pypy/jit/backend/x86/regloc.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/regloc.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/regloc.py Wed Oct 20 13:33:36 2010
@@ -442,8 +442,11 @@
MOV8 = _binaryop('MOV8')
MOV16 = _16_bit_binaryop('MOV')
MOVZX8 = _binaryop('MOVZX8')
+ MOVSX8 = _binaryop('MOVSX8')
MOVZX16 = _binaryop('MOVZX16')
+ MOVSX16 = _binaryop('MOVSX16')
MOV32 = _binaryop('MOV32')
+ MOVSX32 = _binaryop('MOVSX32')
XCHG = _binaryop('XCHG')
PUSH = _unaryop('PUSH')
@@ -473,6 +476,9 @@
else:
return ImmedLoc(x)
+imm0 = imm(0)
+imm1 = imm(1)
+
all_extra_instructions = [name for name in LocationCodeBuilder.__dict__
if name[0].isupper()]
all_extra_instructions.sort()
Modified: pypy/trunk/pypy/jit/backend/x86/rx86.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/rx86.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/rx86.py Wed Oct 20 13:33:36 2010
@@ -642,7 +642,10 @@
define_modrm_modes('MOV8_*i', [rex_w, '\xC6', orbyte(0<<3)], [immediate(2, 'b')], regtype='BYTE')
define_modrm_modes('MOVZX8_r*', [rex_w, '\x0F\xB6', register(1, 8)], regtype='BYTE')
+define_modrm_modes('MOVSX8_r*', [rex_w, '\x0F\xBE', register(1, 8)], regtype='BYTE')
define_modrm_modes('MOVZX16_r*', [rex_w, '\x0F\xB7', register(1, 8)])
+define_modrm_modes('MOVSX16_r*', [rex_w, '\x0F\xBF', register(1, 8)])
+define_modrm_modes('MOVSX32_r*', [rex_w, '\x63', register(1, 8)])
define_modrm_modes('MOVSD_x*', ['\xF2', rex_nw, '\x0F\x10', register(1,8)], regtype='XMM')
define_modrm_modes('MOVSD_*x', ['\xF2', rex_nw, '\x0F\x11', register(2,8)], regtype='XMM')
From arigo at codespeak.net Wed Oct 20 13:35:29 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 13:35:29 +0200 (CEST)
Subject: [pypy-svn] r78125 - in pypy/trunk/pypy: module/_ffi
module/_ffi/test rlib
Message-ID: <20101020113529.2A13D282BDA@codespeak.net>
Author: arigo
Date: Wed Oct 20 13:35:27 2010
New Revision: 78125
Modified:
pypy/trunk/pypy/module/_ffi/interp_ffi.py
pypy/trunk/pypy/module/_ffi/test/test__ffi.py
pypy/trunk/pypy/rlib/libffi.py
Log:
Adapt libffi.py and module/_ffi to get the same signed/unsigned
distinction. Add tests.
Modified: pypy/trunk/pypy/module/_ffi/interp_ffi.py
==============================================================================
--- pypy/trunk/pypy/module/_ffi/interp_ffi.py (original)
+++ pypy/trunk/pypy/module/_ffi/interp_ffi.py Wed Oct 20 13:35:27 2010
@@ -11,6 +11,7 @@
from pypy.rlib import jit
from pypy.rlib import libffi
from pypy.rlib.rdynload import DLOpenError
+from pypy.rlib.rarithmetic import intmask
class W_FFIType(Wrappable):
def __init__(self, name, ffitype):
@@ -74,6 +75,8 @@
kind = libffi.types.getkind(argtype)
if kind == 'i':
argchain.arg(space.int_w(w_arg))
+ elif kind == 'u':
+ argchain.arg(intmask(space.uint_w(w_arg)))
elif kind == 'f':
argchain.arg(space.float_w(w_arg))
else:
@@ -88,6 +91,9 @@
if reskind == 'i':
intres = self.func.call(argchain, rffi.LONG)
return space.wrap(intres)
+ elif reskind == 'u':
+ intres = self.func.call(argchain, rffi.ULONG)
+ return space.wrap(intres)
elif reskind == 'f':
floatres = self.func.call(argchain, rffi.DOUBLE)
return space.wrap(floatres)
Modified: pypy/trunk/pypy/module/_ffi/test/test__ffi.py
==============================================================================
--- pypy/trunk/pypy/module/_ffi/test/test__ffi.py (original)
+++ pypy/trunk/pypy/module/_ffi/test/test__ffi.py Wed Oct 20 13:35:27 2010
@@ -97,6 +97,36 @@
assert set_dummy(42) is None
assert get_dummy() == 42
+ def test_unsigned_long_args(self):
+ """
+ unsigned long sum_xy_ul(unsigned long x, unsigned long y)
+ {
+ return x+y;
+ }
+ """
+ import sys
+ from _ffi import CDLL, types
+ libfoo = CDLL(self.libfoo_name)
+ sum_xy = libfoo.getfunc('sum_xy_ul', [types.ulong, types.ulong],
+ types.ulong)
+ assert sum_xy(sys.maxint, 12) == sys.maxint+12
+ assert sum_xy(sys.maxint+1, 12) == sys.maxint+13
+
+ def test_unsigned_short_args(self):
+ """
+ unsigned short sum_xy_us(unsigned short x, unsigned short y)
+ {
+ return x+y;
+ }
+ """
+ import sys
+ from _ffi import CDLL, types
+ libfoo = CDLL(self.libfoo_name)
+ sum_xy = libfoo.getfunc('sum_xy_us', [types.ushort, types.ushort],
+ types.ushort)
+ assert sum_xy(32000, 8000) == 40000
+ assert sum_xy(60000, 30000) == 90000 % 65536
+
def test_TypeError_numargs(self):
from _ffi import CDLL, types
libfoo = CDLL(self.libfoo_name)
Modified: pypy/trunk/pypy/rlib/libffi.py
==============================================================================
--- pypy/trunk/pypy/rlib/libffi.py (original)
+++ pypy/trunk/pypy/rlib/libffi.py Wed Oct 20 13:35:27 2010
@@ -35,27 +35,33 @@
@staticmethod
@jit.purefunction
def getkind(ffi_type):
+ """Returns 'v' for void, 'f' for float, 'i' for signed integer,
+ and 'u' for unsigned integer.
+ """
if ffi_type is types.void: return 'v'
elif ffi_type is types.double: return 'f'
elif ffi_type is types.pointer: return 'i'
#
elif ffi_type is types.schar: return 'i'
- elif ffi_type is types.uchar: return 'i'
+ elif ffi_type is types.uchar: return 'u'
elif ffi_type is types.sshort: return 'i'
- elif ffi_type is types.ushort: return 'i'
+ elif ffi_type is types.ushort: return 'u'
elif ffi_type is types.sint: return 'i'
- elif ffi_type is types.uint: return 'i'
+ elif ffi_type is types.uint: return 'u'
elif ffi_type is types.slong: return 'i'
- elif ffi_type is types.ulong: return 'i'
+ elif ffi_type is types.ulong: return 'u'
#
elif ffi_type is types.sint8: return 'i'
- elif ffi_type is types.uint8: return 'i'
+ elif ffi_type is types.uint8: return 'u'
elif ffi_type is types.sint16: return 'i'
- elif ffi_type is types.uint16: return 'i'
+ elif ffi_type is types.uint16: return 'u'
elif ffi_type is types.sint32: return 'i'
- elif ffi_type is types.uint32: return 'i'
+ elif ffi_type is types.uint32: return 'u'
+ ## we only support integers that fit in a lltype.Signed (==rffi.LONG)
+ ## (on 64-bit platforms, types.sint64 is types.slong and the case is
+ ## caught above)
## elif ffi_type is types.sint64: return 'i'
- ## elif ffi_type is types.uint64: return 'i'
+ ## elif ffi_type is types.uint64: return 'u'
raise KeyError
types._import()
From arigo at codespeak.net Wed Oct 20 13:43:06 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 13:43:06 +0200 (CEST)
Subject: [pypy-svn] r78126 - pypy/trunk/pypy/rlib
Message-ID: <20101020114306.A88C8282BDA@codespeak.net>
Author: arigo
Date: Wed Oct 20 13:43:05 2010
New Revision: 78126
Modified:
pypy/trunk/pypy/rlib/clibffi.py
Log:
'track_allocation=False' must also be given on lltype.free().
Modified: pypy/trunk/pypy/rlib/clibffi.py
==============================================================================
--- pypy/trunk/pypy/rlib/clibffi.py (original)
+++ pypy/trunk/pypy/rlib/clibffi.py Wed Oct 20 13:43:05 2010
@@ -414,10 +414,10 @@
def __del__(self):
if self.ll_cif:
- lltype.free(self.ll_cif, flavor='raw')
+ lltype.free(self.ll_cif, flavor='raw', track_allocation=False)
self.ll_cif = lltype.nullptr(FFI_CIFP.TO)
if self.ll_argtypes:
- lltype.free(self.ll_argtypes, flavor='raw')
+ lltype.free(self.ll_argtypes, flavor='raw', track_allocation=False)
self.ll_argtypes = lltype.nullptr(FFI_TYPE_PP.TO)
# as long as CallbackFuncPtr is kept alive, the underlaying userdata
From arigo at codespeak.net Wed Oct 20 13:43:32 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 13:43:32 +0200 (CEST)
Subject: [pypy-svn] r78127 - pypy/branch/jit-sign-descr
Message-ID: <20101020114332.9E9EF282BDA@codespeak.net>
Author: arigo
Date: Wed Oct 20 13:43:31 2010
New Revision: 78127
Removed:
pypy/branch/jit-sign-descr/
Log:
Remove merged branch.
From afa at codespeak.net Wed Oct 20 14:25:40 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 14:25:40 +0200 (CEST)
Subject: [pypy-svn] r78129 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101020122540.CFBC2282BDA@codespeak.net>
Author: afa
Date: Wed Oct 20 14:25:39 2010
New Revision: 78129
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
Log:
FileIO.truncate()
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Wed Oct 20 14:25:39 2010
@@ -7,7 +7,7 @@
from pypy.rlib.rarithmetic import r_longlong
from pypy.rlib.rstring import StringBuilder
from os import O_RDONLY, O_WRONLY, O_RDWR, O_CREAT, O_TRUNC
-import os, stat, errno
+import sys, os, stat, errno
O_BINARY = getattr(os, "O_BINARY", 0)
O_APPEND = getattr(os, "O_APPEND", 0)
@@ -347,6 +347,25 @@
total += len(chunk)
return space.wrap(builder.build())
+ if sys.platform == "win32":
+ def _truncate(self, size):
+ from pypy.rlib.streamio import ftruncate_win32
+ ftruncate_win32(self.fd, size)
+ else:
+ def _truncate(self, size):
+ os.ftruncate(self.fd, size)
+
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def truncate_w(self, space, w_size=None):
+ if space.is_w(w_size, space.w_None):
+ w_size = self.tell_w(space)
+
+ try:
+ self._truncate(space.r_longlong_w(w_size))
+ except OSError, e:
+ raise wrap_oserror(space, e)
+
+ return w_size
W_FileIO.typedef = TypeDef(
'FileIO', W_RawIOBase.typedef,
@@ -358,6 +377,7 @@
write = interp2app(W_FileIO.write_w),
read = interp2app(W_FileIO.read_w),
readall = interp2app(W_FileIO.readall_w),
+ truncate = interp2app(W_FileIO.truncate_w),
close = interp2app(W_FileIO.close_w),
readable = interp2app(W_FileIO.readable_w),
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py Wed Oct 20 14:25:39 2010
@@ -74,3 +74,20 @@
f.seek(3)
assert f.tell() == 3
f.close()
+
+ def test_truncate(self):
+ import _io
+ f = _io.FileIO(self.tmpfile, 'wb')
+ assert f.truncate(100) == 100 # grow the file
+ f.close()
+ f = _io.FileIO(self.tmpfile)
+ assert len(f.read()) == 100
+ f.close()
+ #
+ f = _io.FileIO(self.tmpfile, 'wb')
+ f.seek(50)
+ assert f.truncate() == 50
+ f.close()
+ f = _io.FileIO(self.tmpfile)
+ assert len(f.read()) == 50
+ f.close()
From afa at codespeak.net Wed Oct 20 14:45:07 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 14:45:07 +0200 (CEST)
Subject: [pypy-svn] r78130 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101020124507.83A90282BDA@codespeak.net>
Author: afa
Date: Wed Oct 20 14:45:05 2010
New Revision: 78130
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
Log:
FileIO.readlines()
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Wed Oct 20 14:45:05 2010
@@ -163,6 +163,29 @@
return space.wrap(builder.build())
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def readlines_w(self, space, w_hint=None):
+ hint = convert_size(space, w_hint)
+
+ if hint <= 0:
+ return space.newlist(space.unpackiterable(self))
+
+ lines_w = []
+ length = 0
+ while True:
+ w_line = space.call_method(self, "readline")
+ line_length = space.int_w(space.len(w_line))
+ if line_length == 0: # done
+ break
+
+ lines_w.append(w_line)
+
+ length += line_length
+ if length > hint:
+ break
+
+ return space.newlist(lines_w)
+
W_IOBase.typedef = TypeDef(
'_IOBase',
__new__ = generic_new_descr(W_IOBase),
@@ -180,6 +203,7 @@
closed = GetSetProperty(W_IOBase.closed_get_w),
readline = interp2app(W_IOBase.readline_w),
+ readlines = interp2app(W_IOBase.readlines_w),
)
class W_RawIOBase(W_IOBase):
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py Wed Oct 20 14:45:05 2010
@@ -44,6 +44,14 @@
assert f.readline() == ''
f.close()
+ def test_readlines(self):
+ import _io
+ f = _io.FileIO(self.tmpfile, 'rb')
+ assert f.readlines() == ["a\n", "b\n", "c"]
+ f.seek(0)
+ assert f.readlines(3) == ["a\n", "b\n"]
+ f.close()
+
def test_readall(self):
import _io
f = _io.FileIO(self.tmpfile, 'rb')
From afa at codespeak.net Wed Oct 20 15:14:17 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 15:14:17 +0200 (CEST)
Subject: [pypy-svn] r78131 - in pypy/branch/fast-forward/pypy/objspace/std:
. test
Message-ID: <20101020131417.CDBC8282BDA@codespeak.net>
Author: afa
Date: Wed Oct 20 15:14:16 2010
New Revision: 78131
Modified:
pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py
Log:
buffer interface for bytearray objects
Modified: pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py Wed Oct 20 15:14:16 2010
@@ -11,6 +11,7 @@
from pypy.objspace.std.sliceobject import W_SliceObject, normalize_simple_slice
from pypy.objspace.std import slicetype
from pypy.interpreter import gateway
+from pypy.interpreter.buffer import RWBuffer
class W_BytearrayObject(W_Object):
from pypy.objspace.std.bytearraytype import bytearray_typedef as typedef
@@ -400,5 +401,25 @@
assert stop >= 0
w_bytearray.data[start:stop] = [c for c in space.str_w(w_other)]
+# __________________________________________________________
+# Buffer interface
+
+class BytearrayBuffer(RWBuffer):
+ def __init__(self, data):
+ self.data = data
+
+ def getlength(self):
+ return len(self.data)
+
+ def getitem(self, index):
+ return self.data[index]
+
+ def setitem(self, index, char):
+ self.data[index] = char
+
+def buffer__Bytearray(space, self):
+ b = BytearrayBuffer(self.data)
+ return space.wrap(b)
+
from pypy.objspace.std import bytearraytype
register_all(vars(), bytearraytype)
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py Wed Oct 20 15:14:16 2010
@@ -175,3 +175,12 @@
assert b == 'ABC...defghi'
b[3:6] = '()'
assert b == 'ABC()defghi'
+
+ def test_buffer(self):
+ b = bytearray('abcdefghi')
+ buf = buffer(b)
+ assert buf[2] == 'c'
+ buf[3] = 'D'
+ assert b == 'abcDefghi'
+ buf[4:6] = 'EF'
+ assert b == 'abcDEFghi'
From afa at codespeak.net Wed Oct 20 15:18:59 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 15:18:59 +0200 (CEST)
Subject: [pypy-svn] r78132 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101020131859.635C136E3FB@codespeak.net>
Author: afa
Date: Wed Oct 20 15:18:58 2010
New Revision: 78132
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
Log:
FileIO.readinto()
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Wed Oct 20 15:18:58 2010
@@ -321,6 +321,21 @@
return space.wrap(s)
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def readinto_w(self, space, w_buffer):
+ self._check_closed(space)
+
+ # XXX check readable
+ rwbuffer = space.rwbuffer_w(w_buffer)
+ length = rwbuffer.getlength()
+ try:
+ buf = os.read(self.fd, length)
+ except OSError, e:
+ raise wrap_oserror(space, e,
+ exception_name='w_IOError')
+ rwbuffer.setslice(0, buf)
+ return space.wrap(len(buf))
+
@unwrap_spec('self', ObjSpace)
def readall_w(self, space):
self._check_closed(space)
@@ -376,6 +391,7 @@
tell = interp2app(W_FileIO.tell_w),
write = interp2app(W_FileIO.write_w),
read = interp2app(W_FileIO.read_w),
+ readinto = interp2app(W_FileIO.readinto_w),
readall = interp2app(W_FileIO.readall_w),
truncate = interp2app(W_FileIO.truncate_w),
close = interp2app(W_FileIO.close_w),
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py Wed Oct 20 15:18:58 2010
@@ -85,17 +85,32 @@
def test_truncate(self):
import _io
- f = _io.FileIO(self.tmpfile, 'wb')
+ f = _io.FileIO(self.tmpfile, 'r+b')
assert f.truncate(100) == 100 # grow the file
f.close()
f = _io.FileIO(self.tmpfile)
assert len(f.read()) == 100
f.close()
#
- f = _io.FileIO(self.tmpfile, 'wb')
+ f = _io.FileIO(self.tmpfile, 'r+b')
f.seek(50)
assert f.truncate() == 50
f.close()
f = _io.FileIO(self.tmpfile)
assert len(f.read()) == 50
f.close()
+
+ def test_readinto(self):
+ import _io
+ a = bytearray('x' * 10)
+ f = _io.FileIO(self.tmpfile, 'r+')
+ assert f.readinto(a) == 10
+ f.close()
+ assert a == 'a\nb\nc\0\0\0\0\0'
+ #
+ a = bytearray('x' * 10)
+ f = _io.FileIO(self.tmpfile, 'r+')
+ f.truncate(3)
+ assert f.readinto(a) == 3
+ f.close()
+ assert a == 'a\nbxxxxxxx'
From arigo at codespeak.net Wed Oct 20 15:54:18 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 15:54:18 +0200 (CEST)
Subject: [pypy-svn] r78133 - in pypy/trunk/pypy/translator: . c c/src c/test
Message-ID: <20101020135418.6C230282BDA@codespeak.net>
Author: arigo
Date: Wed Oct 20 15:54:16 2010
New Revision: 78133
Modified:
pypy/trunk/pypy/translator/c/funcgen.py
pypy/trunk/pypy/translator/c/src/stack.h
pypy/trunk/pypy/translator/c/test/test_genc.py
pypy/trunk/pypy/translator/c/test/test_standalone.py
pypy/trunk/pypy/translator/transform.py
Log:
(antocuni, arigo)
Hopefully fix lib-python/.../test_descr on 64-bit platforms. The issue
is that we can get a cycle of tail recursions (more easily on x86-64
than on x86-32), and in such cycles, the stack overflow detection does
not work as a way to interrupt the cycle. Thus we can get caught in
what seems like an infinite recursion (but is not in C).
Modified: pypy/trunk/pypy/translator/c/funcgen.py
==============================================================================
--- pypy/trunk/pypy/translator/c/funcgen.py (original)
+++ pypy/trunk/pypy/translator/c/funcgen.py Wed Oct 20 15:54:16 2010
@@ -427,7 +427,7 @@
r = self.expr(op.result)
return 'OP_CALL_ARGS((%s), %s);' % (', '.join(args), r)
- def generic_call(self, FUNC, fnexpr, args_v, v_result):
+ def generic_call(self, FUNC, fnexpr, args_v, v_result, targets=None):
args = []
assert len(args_v) == len(FUNC.TO.ARGS)
for v, ARGTYPE in zip(args_v, FUNC.TO.ARGS):
@@ -444,17 +444,23 @@
# skip assignment of 'void' return value
r = self.expr(v_result)
line = '%s = %s' % (r, line)
+ if targets:
+ for func in targets:
+ graph = getattr(func._obj, 'graph', None)
+ if getattr(graph, 'inhibit_tail_call', False):
+ line += '\nPYPY_INHIBIT_TAIL_CALL();'
+ break
return line
def OP_DIRECT_CALL(self, op):
fn = op.args[0]
return self.generic_call(fn.concretetype, self.expr(fn),
- op.args[1:], op.result)
+ op.args[1:], op.result, [fn.value])
def OP_INDIRECT_CALL(self, op):
fn = op.args[0]
return self.generic_call(fn.concretetype, self.expr(fn),
- op.args[1:-1], op.result)
+ op.args[1:-1], op.result, op.args[-1].value)
def OP_ADR_CALL(self, op):
ARGTYPES = [v.concretetype for v in op.args[1:]]
Modified: pypy/trunk/pypy/translator/c/src/stack.h
==============================================================================
--- pypy/trunk/pypy/translator/c/src/stack.h (original)
+++ pypy/trunk/pypy/translator/c/src/stack.h Wed Oct 20 15:54:16 2010
@@ -33,6 +33,12 @@
&& LL_stack_too_big_slowpath());
}
+#ifdef __GNUC__
+# define PYPY_INHIBIT_TAIL_CALL() asm("/* inhibit_tail_call */")
+#else
+# define PYPY_INHIBIT_TAIL_CALL() /* add hints for other compilers here */
+#endif
+
#ifndef PYPY_NOT_MAIN_FILE
#include
Modified: pypy/trunk/pypy/translator/c/test/test_genc.py
==============================================================================
--- pypy/trunk/pypy/translator/c/test/test_genc.py (original)
+++ pypy/trunk/pypy/translator/c/test/test_genc.py Wed Oct 20 15:54:16 2010
@@ -478,3 +478,23 @@
t.bar = llhelper(FTPTR, a_f.make_func())
fn = compile(chooser, [bool])
assert fn(True)
+
+def test_inhibit_tail_call():
+ from pypy.rpython.lltypesystem import lltype
+ def foobar_fn(n):
+ return 42
+ foobar_fn._dont_inline_ = True
+ def main(n):
+ return foobar_fn(n)
+ #
+ t = Translation(main, [int], backend="c")
+ t.rtype()
+ t.context._graphof(foobar_fn).inhibit_tail_call = True
+ t.source_c()
+ lines = t.driver.cbuilder.c_source_filename.readlines()
+ for i, line in enumerate(lines):
+ if '= pypy_g_foobar_fn' in line:
+ break
+ else:
+ assert 0, "the call was not found in the C source"
+ assert 'PYPY_INHIBIT_TAIL_CALL();' in lines[i+1]
Modified: pypy/trunk/pypy/translator/c/test/test_standalone.py
==============================================================================
--- pypy/trunk/pypy/translator/c/test/test_standalone.py (original)
+++ pypy/trunk/pypy/translator/c/test/test_standalone.py Wed Oct 20 15:54:16 2010
@@ -16,11 +16,16 @@
class StandaloneTests(object):
config = None
- def compile(self, entry_point, debug=True, shared=False):
+ def compile(self, entry_point, debug=True, shared=False,
+ stackcheck=False):
t = TranslationContext(self.config)
t.buildannotator().build_types(entry_point, [s_list_of_strings])
t.buildrtyper().specialize()
+ if stackcheck:
+ from pypy.translator.transform import insert_ll_stackcheck
+ insert_ll_stackcheck(t)
+
t.config.translation.shared = shared
cbuilder = CStandaloneBuilder(t, entry_point, t.config)
@@ -630,6 +635,22 @@
else:
os.environ['CC'] = old_cc
+ def test_inhibit_tail_call(self):
+ # the point is to check that the f()->f() recursion stops
+ from pypy.rlib.rstackovf import StackOverflow
+ def f(n):
+ if n <= 0:
+ return 42
+ return f(n+1)
+ def entry_point(argv):
+ try:
+ return f(1)
+ except StackOverflow:
+ print 'hi!'
+ return 0
+ t, cbuilder = self.compile(entry_point, stackcheck=True)
+ out = cbuilder.cmdexec("")
+ assert out.strip() == "hi!"
class TestMaemo(TestStandalone):
def setup_class(cls):
Modified: pypy/trunk/pypy/translator/transform.py
==============================================================================
--- pypy/trunk/pypy/translator/transform.py (original)
+++ pypy/trunk/pypy/translator/transform.py Wed Oct 20 15:54:16 2010
@@ -221,15 +221,19 @@
stack_check_ptr_const = Constant(stack_check_ptr, lltype.typeOf(stack_check_ptr))
edges = set()
insert_in = set()
+ block2graph = {}
for caller in translator.graphs:
for block, callee in find_calls_from(translator, caller):
if getattr(getattr(callee, 'func', None),
'insert_stack_check_here', False):
insert_in.add(callee.startblock)
+ block2graph[callee.startblock] = callee
continue
if block is not caller.startblock:
edges.add((caller.startblock, block))
+ block2graph[caller.startblock] = caller
edges.add((block, callee.startblock))
+ block2graph[block] = caller
edgelist = [Edge(block1, block2) for (block1, block2) in edges]
edgedict = make_edge_dict(edgelist)
@@ -241,6 +245,10 @@
v.concretetype = lltype.Void
unwind_op = SpaceOperation('direct_call', [stack_check_ptr_const], v)
block.operations.insert(0, unwind_op)
+ # prevents cycles of tail calls from occurring -- such cycles would
+ # not consume any stack, so would turn into potentially infinite loops
+ graph = block2graph[block]
+ graph.inhibit_tail_call = True
return len(insert_in)
From afa at codespeak.net Wed Oct 20 16:06:24 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 16:06:24 +0200 (CEST)
Subject: [pypy-svn] r78134 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101020140624.DA12036E3FB@codespeak.net>
Author: afa
Date: Wed Oct 20 16:06:23 2010
New Revision: 78134
Added:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (contents, props changed)
pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py (contents, props changed)
Modified:
pypy/branch/fast-forward/pypy/module/_io/__init__.py
pypy/branch/fast-forward/pypy/module/_io/interp_io.py
pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py (contents, props changed)
Log:
Move classes to their own files
Modified: pypy/branch/fast-forward/pypy/module/_io/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/__init__.py Wed Oct 20 16:06:23 2010
@@ -11,17 +11,17 @@
'BlockingIOError': 'interp_io.W_BlockingIOError',
'_IOBase': 'interp_iobase.W_IOBase',
'_RawIOBase': 'interp_iobase.W_RawIOBase',
- '_BufferedIOBase': 'interp_io.W_BufferedIOBase',
- '_TextIOBase': 'interp_io.W_TextIOBase',
+ '_BufferedIOBase': 'interp_bufferedio.W_BufferedIOBase',
+ '_TextIOBase': 'interp_textio.W_TextIOBase',
'FileIO': 'interp_fileio.W_FileIO',
- 'BytesIO': 'interp_io.W_BytesIO',
+ 'BytesIO': 'interp_bytesio.W_BytesIO',
'StringIO': 'interp_stringio.W_StringIO',
- 'BufferedReader': 'interp_io.W_BufferedReader',
- 'BufferedWriter': 'interp_io.W_BufferedWriter',
- 'BufferedRWPair': 'interp_io.W_BufferedRWPair',
- 'BufferedRandom': 'interp_io.W_BufferedRandom',
- 'TextIOWrapper': 'interp_io.W_TextIOWrapper',
+ 'BufferedReader': 'interp_bufferedio.W_BufferedReader',
+ 'BufferedWriter': 'interp_bufferedio.W_BufferedWriter',
+ 'BufferedRWPair': 'interp_bufferedio.W_BufferedRWPair',
+ 'BufferedRandom': 'interp_bufferedio.W_BufferedRandom',
+ 'TextIOWrapper': 'interp_textio.W_TextIOWrapper',
'open': 'interp_io.open',
'IncrementalNewlineDecoder': 'space.w_None',
Added: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Wed Oct 20 16:06:23 2010
@@ -0,0 +1,39 @@
+from pypy.module._io.interp_iobase import W_IOBase
+from pypy.interpreter.typedef import (
+ TypeDef, generic_new_descr)
+
+class W_BufferedIOBase(W_IOBase):
+ pass
+W_BufferedIOBase.typedef = TypeDef(
+ '_BufferedIOBase', W_IOBase.typedef,
+ __new__ = generic_new_descr(W_BufferedIOBase),
+ )
+
+class W_BufferedReader(W_BufferedIOBase):
+ pass
+W_BufferedReader.typedef = TypeDef(
+ 'BufferedReader', W_BufferedIOBase.typedef,
+ __new__ = generic_new_descr(W_BufferedReader),
+ )
+
+class W_BufferedWriter(W_BufferedIOBase):
+ pass
+W_BufferedWriter.typedef = TypeDef(
+ 'BufferedWriter', W_BufferedIOBase.typedef,
+ __new__ = generic_new_descr(W_BufferedWriter),
+ )
+
+class W_BufferedRWPair(W_BufferedIOBase):
+ pass
+W_BufferedRWPair.typedef = TypeDef(
+ 'BufferedRWPair', W_BufferedIOBase.typedef,
+ __new__ = generic_new_descr(W_BufferedRWPair),
+ )
+
+class W_BufferedRandom(W_BufferedIOBase):
+ pass
+W_BufferedRandom.typedef = TypeDef(
+ 'BufferedRandom', W_BufferedIOBase.typedef,
+ __new__ = generic_new_descr(W_BufferedRandom),
+ )
+
Added: pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py Wed Oct 20 16:06:23 2010
@@ -0,0 +1,11 @@
+from pypy.module._io.interp_bufferedio import W_BufferedIOBase
+from pypy.interpreter.typedef import (
+ TypeDef, generic_new_descr)
+
+class W_BytesIO(W_BufferedIOBase):
+ pass
+W_BytesIO.typedef = TypeDef(
+ 'BytesIO', W_BufferedIOBase.typedef,
+ __new__ = generic_new_descr(W_BytesIO),
+ )
+
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_io.py Wed Oct 20 16:06:23 2010
@@ -26,14 +26,6 @@
characters_written = interp_attrproperty('written', W_BlockingIOError),
)
-class W_BufferedIOBase(W_IOBase):
- pass
-
-W_BufferedIOBase.typedef = TypeDef(
- '_BufferedIOBase', W_IOBase.typedef,
- __new__ = generic_new_descr(W_BufferedIOBase),
- )
-
class W_TextIOBase(W_IOBase):
pass
W_TextIOBase.typedef = TypeDef(
@@ -41,41 +33,6 @@
__new__ = generic_new_descr(W_TextIOBase),
)
-class W_BytesIO(W_BufferedIOBase):
- pass
-W_BytesIO.typedef = TypeDef(
- 'BytesIO', W_BufferedIOBase.typedef,
- __new__ = generic_new_descr(W_BytesIO),
- )
-
-class W_BufferedReader(W_BufferedIOBase):
- pass
-W_BufferedReader.typedef = TypeDef(
- 'BufferedReader', W_BufferedIOBase.typedef,
- __new__ = generic_new_descr(W_BufferedReader),
- )
-
-class W_BufferedWriter(W_BufferedIOBase):
- pass
-W_BufferedWriter.typedef = TypeDef(
- 'BufferedWriter', W_BufferedIOBase.typedef,
- __new__ = generic_new_descr(W_BufferedWriter),
- )
-
-class W_BufferedRWPair(W_BufferedIOBase):
- pass
-W_BufferedRWPair.typedef = TypeDef(
- 'BufferedRWPair', W_BufferedIOBase.typedef,
- __new__ = generic_new_descr(W_BufferedRWPair),
- )
-
-class W_BufferedRandom(W_BufferedIOBase):
- pass
-W_BufferedRandom.typedef = TypeDef(
- 'BufferedRandom', W_BufferedIOBase.typedef,
- __new__ = generic_new_descr(W_BufferedRandom),
- )
-
class W_TextIOWrapper(W_TextIOBase):
pass
W_TextIOWrapper.typedef = TypeDef(
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py Wed Oct 20 16:06:23 2010
@@ -1,116 +1,116 @@
-from pypy.conftest import gettestobjspace
-from pypy.tool.udir import udir
-import os
-
-class AppTestFileIO:
- def setup_class(cls):
- cls.space = gettestobjspace(usemodules=['_io'])
- tmpfile = udir.join('tmpfile')
- tmpfile.write("a\nb\nc", mode='wb')
- cls.w_tmpfile = cls.space.wrap(str(tmpfile))
- cls.w_tmpdir = cls.space.wrap(str(udir))
- cls.w_posix = cls.space.appexec([], """():
- import %s as m;
- return m""" % os.name)
-
- def test_constructor(self):
- import _io
- f = _io.FileIO(self.tmpfile, 'a')
- assert f.name.endswith('tmpfile')
- assert f.mode == 'wb'
- assert f.closefd is True
- f.close()
-
- def test_open_fd(self):
- import _io
- os = self.posix
- fd = os.open(self.tmpfile, os.O_RDONLY, 0666)
- f = _io.FileIO(fd, "rb", closefd=False)
- assert f.fileno() == fd
- assert f.closefd is False
- f.close()
- os.close(fd)
-
- def test_open_directory(self):
- import _io
- raises(IOError, _io.FileIO, self.tmpdir, "rb")
-
- def test_readline(self):
- import _io
- f = _io.FileIO(self.tmpfile, 'rb')
- assert f.readline() == 'a\n'
- assert f.readline() == 'b\n'
- assert f.readline() == 'c'
- assert f.readline() == ''
- f.close()
-
- def test_readlines(self):
- import _io
- f = _io.FileIO(self.tmpfile, 'rb')
- assert f.readlines() == ["a\n", "b\n", "c"]
- f.seek(0)
- assert f.readlines(3) == ["a\n", "b\n"]
- f.close()
-
- def test_readall(self):
- import _io
- f = _io.FileIO(self.tmpfile, 'rb')
- assert f.readall() == "a\nb\nc"
- f.close()
-
- def test_write(self):
- import _io
- filename = self.tmpfile + '_w'
- f = _io.FileIO(filename, 'wb')
- f.write("test")
- # try without flushing
- f2 = _io.FileIO(filename, 'rb')
- assert f2.read() == "test"
- f.close()
- f2.close()
-
- def test_seek(self):
- import _io
- f = _io.FileIO(self.tmpfile, 'rb')
- f.seek(0)
- self.posix.close(f.fileno())
- raises(IOError, f.seek, 0)
-
- def test_tell(self):
- import _io
- f = _io.FileIO(self.tmpfile, 'rb')
- f.seek(3)
- assert f.tell() == 3
- f.close()
-
- def test_truncate(self):
- import _io
- f = _io.FileIO(self.tmpfile, 'r+b')
- assert f.truncate(100) == 100 # grow the file
- f.close()
- f = _io.FileIO(self.tmpfile)
- assert len(f.read()) == 100
- f.close()
- #
- f = _io.FileIO(self.tmpfile, 'r+b')
- f.seek(50)
- assert f.truncate() == 50
- f.close()
- f = _io.FileIO(self.tmpfile)
- assert len(f.read()) == 50
- f.close()
-
- def test_readinto(self):
- import _io
- a = bytearray('x' * 10)
- f = _io.FileIO(self.tmpfile, 'r+')
- assert f.readinto(a) == 10
- f.close()
- assert a == 'a\nb\nc\0\0\0\0\0'
- #
- a = bytearray('x' * 10)
- f = _io.FileIO(self.tmpfile, 'r+')
- f.truncate(3)
- assert f.readinto(a) == 3
- f.close()
- assert a == 'a\nbxxxxxxx'
+from pypy.conftest import gettestobjspace
+from pypy.tool.udir import udir
+import os
+
+class AppTestFileIO:
+ def setup_class(cls):
+ cls.space = gettestobjspace(usemodules=['_io'])
+ tmpfile = udir.join('tmpfile')
+ tmpfile.write("a\nb\nc", mode='wb')
+ cls.w_tmpfile = cls.space.wrap(str(tmpfile))
+ cls.w_tmpdir = cls.space.wrap(str(udir))
+ cls.w_posix = cls.space.appexec([], """():
+ import %s as m;
+ return m""" % os.name)
+
+ def test_constructor(self):
+ import _io
+ f = _io.FileIO(self.tmpfile, 'a')
+ assert f.name.endswith('tmpfile')
+ assert f.mode == 'wb'
+ assert f.closefd is True
+ f.close()
+
+ def test_open_fd(self):
+ import _io
+ os = self.posix
+ fd = os.open(self.tmpfile, os.O_RDONLY, 0666)
+ f = _io.FileIO(fd, "rb", closefd=False)
+ assert f.fileno() == fd
+ assert f.closefd is False
+ f.close()
+ os.close(fd)
+
+ def test_open_directory(self):
+ import _io
+ raises(IOError, _io.FileIO, self.tmpdir, "rb")
+
+ def test_readline(self):
+ import _io
+ f = _io.FileIO(self.tmpfile, 'rb')
+ assert f.readline() == 'a\n'
+ assert f.readline() == 'b\n'
+ assert f.readline() == 'c'
+ assert f.readline() == ''
+ f.close()
+
+ def test_readlines(self):
+ import _io
+ f = _io.FileIO(self.tmpfile, 'rb')
+ assert f.readlines() == ["a\n", "b\n", "c"]
+ f.seek(0)
+ assert f.readlines(3) == ["a\n", "b\n"]
+ f.close()
+
+ def test_readall(self):
+ import _io
+ f = _io.FileIO(self.tmpfile, 'rb')
+ assert f.readall() == "a\nb\nc"
+ f.close()
+
+ def test_write(self):
+ import _io
+ filename = self.tmpfile + '_w'
+ f = _io.FileIO(filename, 'wb')
+ f.write("test")
+ # try without flushing
+ f2 = _io.FileIO(filename, 'rb')
+ assert f2.read() == "test"
+ f.close()
+ f2.close()
+
+ def test_seek(self):
+ import _io
+ f = _io.FileIO(self.tmpfile, 'rb')
+ f.seek(0)
+ self.posix.close(f.fileno())
+ raises(IOError, f.seek, 0)
+
+ def test_tell(self):
+ import _io
+ f = _io.FileIO(self.tmpfile, 'rb')
+ f.seek(3)
+ assert f.tell() == 3
+ f.close()
+
+ def test_truncate(self):
+ import _io
+ f = _io.FileIO(self.tmpfile, 'r+b')
+ assert f.truncate(100) == 100 # grow the file
+ f.close()
+ f = _io.FileIO(self.tmpfile)
+ assert len(f.read()) == 100
+ f.close()
+ #
+ f = _io.FileIO(self.tmpfile, 'r+b')
+ f.seek(50)
+ assert f.truncate() == 50
+ f.close()
+ f = _io.FileIO(self.tmpfile)
+ assert len(f.read()) == 50
+ f.close()
+
+ def test_readinto(self):
+ import _io
+ a = bytearray('x' * 10)
+ f = _io.FileIO(self.tmpfile, 'r+')
+ assert f.readinto(a) == 10
+ f.close()
+ assert a == 'a\nb\nc\0\0\0\0\0'
+ #
+ a = bytearray('x' * 10)
+ f = _io.FileIO(self.tmpfile, 'r+')
+ f.truncate(3)
+ assert f.readinto(a) == 3
+ f.close()
+ assert a == 'a\nbxxxxxxx'
From hakanardo at codespeak.net Wed Oct 20 16:29:47 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Wed, 20 Oct 2010 16:29:47 +0200 (CEST)
Subject: [pypy-svn] r78135 -
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark
Message-ID: <20101020142947.B5E01282BDA@codespeak.net>
Author: hakanardo
Date: Wed Oct 20 16:29:46 2010
New Revision: 78135
Added:
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/result.txt
Modified:
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py
Log:
some benchmark results
Modified: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile Wed Oct 20 16:29:46 2010
@@ -13,10 +13,12 @@
run-%: %
@echo $^
- @echo -n ' C: '
- @/usr/bin/time -f '%e s' ./$^
- @echo -n ' pypy: '
+ @echo -n ' C: '
+ @/usr/bin/time -f '%e s' ./$^ 2>&1 | tail -1
+ @echo -n ' pypy: '
@/usr/bin/time -f '%e s' ../../../translator/goal/pypy-c $^tst.py 2>&1 | tail -1
- @echo -n ' cpython: '
- @/usr/bin/time -f '%e s' python $^tst.py
+ @echo -n 'pypy-trunk: '
+ @/usr/bin/time -f '%e s' ../../../../../trunk/pypy/translator/goal/pypy-c $^tst.py 2>&1 | tail -1
+ @echo -n ' cpython: '
+ @/usr/bin/time -f '%e s' python $^tst.py 2>&1 | tail -1
@echo
Modified: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py Wed Oct 20 16:29:46 2010
@@ -4,14 +4,14 @@
self = array.__new__(cls, 'd', range(65536))
return self
def __getitem__(self, i):
- assert self.__len__() == 65536
+ assert len(self) == 65536
return array.__getitem__(self, i & 65535)
def main():
buf = Circular()
i = 10
sa = 0
- while i < 20000000:
+ while i < 200000000:
sa += buf[i-2] + buf[i-1] + buf[i] + buf[i+1] + buf[i+2]
i += 1
return sa
Added: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/result.txt
==============================================================================
--- (empty file)
+++ pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/result.txt Wed Oct 20 16:29:46 2010
@@ -0,0 +1,18 @@
+intimg
+ C: 0.29 s
+ pypy: 0.99 s
+pypy-trunk: 1.33 s
+ cpython: 68.20 s
+
+sum
+ C: 0.61 s
+ pypy: 0.59 s
+pypy-trunk: 0.74 s
+ cpython: 24.77 s
+
+circular
+ C: 0.47 s
+ pypy: 1.48 s
+pypy-trunk: 21.13 s
+ cpython: 869.56 s
+
From arigo at codespeak.net Wed Oct 20 16:46:49 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 16:46:49 +0200 (CEST)
Subject: [pypy-svn] r78136 - pypy/trunk/pypy/translator/c
Message-ID: <20101020144649.80E1F36E3FB@codespeak.net>
Author: arigo
Date: Wed Oct 20 16:46:48 2010
New Revision: 78136
Modified:
pypy/trunk/pypy/translator/c/funcgen.py
Log:
Fix.
Modified: pypy/trunk/pypy/translator/c/funcgen.py
==============================================================================
--- pypy/trunk/pypy/translator/c/funcgen.py (original)
+++ pypy/trunk/pypy/translator/c/funcgen.py Wed Oct 20 16:46:48 2010
@@ -445,8 +445,7 @@
r = self.expr(v_result)
line = '%s = %s' % (r, line)
if targets:
- for func in targets:
- graph = getattr(func._obj, 'graph', None)
+ for graph in targets:
if getattr(graph, 'inhibit_tail_call', False):
line += '\nPYPY_INHIBIT_TAIL_CALL();'
break
@@ -454,8 +453,12 @@
def OP_DIRECT_CALL(self, op):
fn = op.args[0]
+ try:
+ targets = [fn.value._obj.graph]
+ except AttributeError:
+ targets = None
return self.generic_call(fn.concretetype, self.expr(fn),
- op.args[1:], op.result, [fn.value])
+ op.args[1:], op.result, targets)
def OP_INDIRECT_CALL(self, op):
fn = op.args[0]
From afa at codespeak.net Wed Oct 20 16:53:02 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 16:53:02 +0200 (CEST)
Subject: [pypy-svn] r78137 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101020145302.3BAA1282BDA@codespeak.net>
Author: afa
Date: Wed Oct 20 16:53:00 2010
New Revision: 78137
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
Log:
FileIO.__repr__, and make FileIO.name a readwrite attribute
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Wed Oct 20 16:53:00 2010
@@ -1,6 +1,7 @@
from pypy.module._io.interp_iobase import W_RawIOBase
from pypy.interpreter.typedef import (
- TypeDef, interp_attrproperty, interp_attrproperty_w, GetSetProperty)
+ TypeDef, interp_attrproperty, interp_attrproperty_w, GetSetProperty,
+ make_weakref_descr)
from pypy.interpreter.gateway import interp2app, unwrap_spec, Arguments
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
from pypy.interpreter.error import OperationError, wrap_oserror, wrap_oserror2
@@ -9,6 +10,27 @@
from os import O_RDONLY, O_WRONLY, O_RDWR, O_CREAT, O_TRUNC
import sys, os, stat, errno
+def interp_member_w(name, cls, doc=None):
+ "NOT_RPYTHON: initialization-time only"
+ def fget(space, obj):
+ w_value = getattr(obj, name)
+ if w_value is None:
+ raise OperationError(space.w_AttributeError,
+ space.wrap(name))
+ else:
+ return w_value
+ def fset(space, obj, w_value):
+ setattr(obj, name, w_value)
+ def fdel(space, obj):
+ w_value = getattr(obj, name)
+ if w_value is None:
+ raise OperationError(space.w_AttributeError,
+ space.wrap(name))
+ setattr(obj, name, None)
+
+ return GetSetProperty(fget, fset, fdel, cls=cls, doc=doc)
+
+
O_BINARY = getattr(os, "O_BINARY", 0)
O_APPEND = getattr(os, "O_APPEND", 0)
@@ -288,6 +310,21 @@
raise wrap_oserror(space, e)
return space.wrap(res)
+ @unwrap_spec('self', ObjSpace)
+ def repr_w(self, space):
+ if self.fd < 0:
+ return space.wrap("<_io.FileIO [closed]>")
+
+ if self.w_name is None:
+ return space.wrap(
+ "<_io.FileIO fd=%d mode='%s'>" % (
+ self.fd, self._mode()))
+ else:
+ w_repr = space.repr(self.w_name)
+ return space.wrap(
+ "<_io.FileIO name=%s mode='%s'>" % (
+ space.str_w(w_repr), self._mode()))
+
# ______________________________________________
@unwrap_spec('self', ObjSpace, W_Root)
@@ -386,6 +423,8 @@
'FileIO', W_RawIOBase.typedef,
__new__ = interp2app(W_FileIO.descr_new.im_func),
__init__ = interp2app(W_FileIO.descr_init),
+ __repr__ = interp2app(W_FileIO.repr_w),
+ __weakref__ = make_weakref_descr(W_FileIO),
seek = interp2app(W_FileIO.seek_w),
tell = interp2app(W_FileIO.tell_w),
@@ -401,7 +440,7 @@
seekable = interp2app(W_FileIO.seekable_w),
fileno = interp2app(W_FileIO.fileno_w),
isatty = interp2app(W_FileIO.isatty_w),
- name = interp_attrproperty_w('w_name', cls=W_FileIO),
+ name = interp_member_w('w_name', cls=W_FileIO),
closefd = interp_attrproperty('closefd', cls=W_FileIO),
mode = GetSetProperty(W_FileIO.descr_get_mode),
)
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py Wed Oct 20 16:53:00 2010
@@ -21,6 +21,14 @@
assert f.closefd is True
f.close()
+ def test_weakrefable(self):
+ import _io
+ from weakref import proxy
+ f = _io.FileIO(self.tmpfile)
+ p = proxy(f)
+ assert p.mode == 'rb'
+ f.close()
+
def test_open_fd(self):
import _io
os = self.posix
@@ -114,3 +122,15 @@
assert f.readinto(a) == 3
f.close()
assert a == 'a\nbxxxxxxx'
+
+ def test_repr(self):
+ import _io
+ f = _io.FileIO(self.tmpfile, 'r')
+ assert repr(f) == ("<_io.FileIO name=%r mode='%s'>"
+ % (f.name, f.mode))
+ del f.name
+ assert repr(f) == ("<_io.FileIO fd=%r mode='%s'>"
+ % (f.fileno(), f.mode))
+ f.close()
+ assert repr(f) == "<_io.FileIO [closed]>"
+
From arigo at codespeak.net Wed Oct 20 16:53:19 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 16:53:19 +0200 (CEST)
Subject: [pypy-svn] r78138 - pypy/branch/cleanup-dict-impl
Message-ID: <20101020145319.062F4282BDA@codespeak.net>
Author: arigo
Date: Wed Oct 20 16:53:18 2010
New Revision: 78138
Added:
pypy/branch/cleanup-dict-impl/
- copied from r78136, pypy/trunk/
Log:
(cfbolz, arigo)
A branch to clean up the various superfluous dict implementations.
From afa at codespeak.net Wed Oct 20 17:09:27 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 17:09:27 +0200 (CEST)
Subject: [pypy-svn] r78139 - pypy/branch/fast-forward/pypy/module/_io
Message-ID: <20101020150927.5B0DC282BDA@codespeak.net>
Author: afa
Date: Wed Oct 20 17:09:25 2010
New Revision: 78139
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_io.py
pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py
Log:
Kill duplicates
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_io.py Wed Oct 20 17:09:25 2010
@@ -26,20 +26,6 @@
characters_written = interp_attrproperty('written', W_BlockingIOError),
)
-class W_TextIOBase(W_IOBase):
- pass
-W_TextIOBase.typedef = TypeDef(
- '_TextIOBase', W_IOBase.typedef,
- __new__ = generic_new_descr(W_TextIOBase),
- )
-
-class W_TextIOWrapper(W_TextIOBase):
- pass
-W_TextIOWrapper.typedef = TypeDef(
- 'TextIOWrapper', W_TextIOBase.typedef,
- __new__ = generic_new_descr(W_TextIOWrapper),
- )
-
@unwrap_spec(ObjSpace, Arguments)
def open(space, __args__):
# XXX cheat!
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py Wed Oct 20 17:09:25 2010
@@ -1,4 +1,4 @@
-from pypy.module._io.interp_io import W_TextIOBase
+from pypy.module._io.interp_textio import W_TextIOBase
from pypy.interpreter.typedef import TypeDef
from pypy.interpreter.gateway import interp2app, unwrap_spec
from pypy.interpreter.error import operationerrfmt
From arigo at codespeak.net Wed Oct 20 17:19:50 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 17:19:50 +0200 (CEST)
Subject: [pypy-svn] r78140 - in pypy/branch/cleanup-dict-impl/pypy: config
config/test doc/config interpreter module/__builtin__
objspace objspace/std objspace/std/test
Message-ID: <20101020151950.683D4282BE7@codespeak.net>
Author: arigo
Date: Wed Oct 20 17:19:47 2010
New Revision: 78140
Removed:
pypy/branch/cleanup-dict-impl/pypy/doc/config/objspace.std.withshadowtracking.txt
pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_shadowtracking.py
Modified:
pypy/branch/cleanup-dict-impl/pypy/config/pypyoption.py
pypy/branch/cleanup-dict-impl/pypy/config/test/test_pypyoption.py
pypy/branch/cleanup-dict-impl/pypy/interpreter/baseobjspace.py
pypy/branch/cleanup-dict-impl/pypy/interpreter/typedef.py
pypy/branch/cleanup-dict-impl/pypy/module/__builtin__/interp_classobj.py
pypy/branch/cleanup-dict-impl/pypy/objspace/descroperation.py
pypy/branch/cleanup-dict-impl/pypy/objspace/std/callmethod.py
pypy/branch/cleanup-dict-impl/pypy/objspace/std/celldict.py
pypy/branch/cleanup-dict-impl/pypy/objspace/std/dictmultiobject.py
pypy/branch/cleanup-dict-impl/pypy/objspace/std/inlinedict.py
pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py
pypy/branch/cleanup-dict-impl/pypy/objspace/std/objspace.py
pypy/branch/cleanup-dict-impl/pypy/objspace/std/proxyobject.py
pypy/branch/cleanup-dict-impl/pypy/objspace/std/sharingdict.py
pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_dictmultiobject.py
Log:
(cfbolz, arigo) Kill shadow tracking.
Modified: pypy/branch/cleanup-dict-impl/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/config/pypyoption.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/config/pypyoption.py Wed Oct 20 17:19:47 2010
@@ -239,14 +239,12 @@
BoolOption("withinlineddict",
"make instances more compact by revoming a level of indirection",
- default=False,
- requires=[("objspace.std.withshadowtracking", False)]),
+ default=False),
BoolOption("withmapdict",
"make instances really small but slow without the JIT",
default=False,
- requires=[("objspace.std.withshadowtracking", False),
- ("objspace.std.withinlineddict", False),
+ requires=[("objspace.std.withinlineddict", False),
("objspace.std.withsharingdict", False),
("objspace.std.getattributeshortcut", True),
("objspace.std.withtypeversion", True),
@@ -265,12 +263,6 @@
# weakrefs needed, because of get_subclasses()
requires=[("translation.rweakref", True)]),
- BoolOption("withshadowtracking",
- "track whether an instance attribute shadows a type"
- " attribute",
- default=False,
- requires=[("objspace.std.withtypeversion", True),
- ("translation.rweakref", True)]),
BoolOption("withmethodcache",
"try to cache method lookups",
default=False,
Modified: pypy/branch/cleanup-dict-impl/pypy/config/test/test_pypyoption.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/config/test/test_pypyoption.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/config/test/test_pypyoption.py Wed Oct 20 17:19:47 2010
@@ -59,7 +59,6 @@
assert not conf.objspace.std.withtypeversion
assert not conf.objspace.std.withmethodcache
- assert not conf.objspace.std.withshadowtracking
def test_check_documentation():
def check_file_exists(fn):
Modified: pypy/branch/cleanup-dict-impl/pypy/interpreter/baseobjspace.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/interpreter/baseobjspace.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/interpreter/baseobjspace.py Wed Oct 20 17:19:47 2010
@@ -36,13 +36,10 @@
return space.finditem_str(w_dict, attr)
return None
- def getdictvalue_attr_is_in_class(self, space, attr):
- return self.getdictvalue(space, attr)
-
- def setdictvalue(self, space, attr, w_value, shadows_type=True):
+ def setdictvalue(self, space, attr, w_value):
w_dict = self.getdict()
if w_dict is not None:
- space.setitem_str(w_dict, attr, w_value, shadows_type)
+ space.setitem_str(w_dict, attr, w_value)
return True
return False
@@ -657,7 +654,7 @@
"""shortcut for space.int_w(space.hash(w_obj))"""
return self.int_w(self.hash(w_obj))
- def setitem_str(self, w_obj, key, w_value, shadows_type=True):
+ def setitem_str(self, w_obj, key, w_value):
return self.setitem(w_obj, self.wrap(key), w_value)
def finditem_str(self, w_obj, key):
Modified: pypy/branch/cleanup-dict-impl/pypy/interpreter/typedef.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/interpreter/typedef.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/interpreter/typedef.py Wed Oct 20 17:19:47 2010
@@ -290,15 +290,6 @@
def setclass(self, space, w_subtype):
# only used by descr_set___class__
self.w__class__ = w_subtype
- if space.config.objspace.std.withshadowtracking:
- self.w__dict__.set_shadows_anything()
-
- def getdictvalue_attr_is_in_class(self, space, name):
- w_dict = self.w__dict__
- if space.config.objspace.std.withshadowtracking:
- if not w_dict.shadows_anything():
- return None
- return space.finditem_str(w_dict, name)
add(Proto)
Modified: pypy/branch/cleanup-dict-impl/pypy/module/__builtin__/interp_classobj.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/module/__builtin__/interp_classobj.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/module/__builtin__/interp_classobj.py Wed Oct 20 17:19:47 2010
@@ -410,8 +410,7 @@
if w_meth is not None:
space.call_function(w_meth, w_name, w_value)
else:
- # bit obscure: appease normalization
- self.setdictvalue(space, name, w_value, True)
+ self.setdictvalue(space, name, w_value)
def descr_delattr(self, space, w_name):
name = unwrap_attr(space, w_name)
Modified: pypy/branch/cleanup-dict-impl/pypy/objspace/descroperation.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/descroperation.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/descroperation.py Wed Oct 20 17:19:47 2010
@@ -64,9 +64,7 @@
w_type = space.type(w_obj)
return space.get_and_call_function(w_get, w_descr, w_obj,
w_type)
- w_value = w_obj.getdictvalue_attr_is_in_class(space, name)
- else:
- w_value = w_obj.getdictvalue(space, name)
+ w_value = w_obj.getdictvalue(space, name)
if w_value is not None:
return w_value
if w_descr is not None:
@@ -76,13 +74,11 @@
def descr__setattr__(space, w_obj, w_name, w_value):
name = space.str_w(w_name)
w_descr = space.lookup(w_obj, name)
- shadows_type = False
if w_descr is not None:
if space.is_data_descr(w_descr):
space.set(w_descr, w_obj, w_value)
return
- shadows_type = True
- if w_obj.setdictvalue(space, name, w_value, shadows_type):
+ if w_obj.setdictvalue(space, name, w_value):
return
raiseattrerror(space, w_obj, name, w_descr)
Modified: pypy/branch/cleanup-dict-impl/pypy/objspace/std/callmethod.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/callmethod.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/std/callmethod.py Wed Oct 20 17:19:47 2010
@@ -44,7 +44,7 @@
else:
typ = type(w_descr)
if typ is function.Function or typ is function.FunctionWithFixedCode:
- w_value = w_obj.getdictvalue_attr_is_in_class(space, name)
+ w_value = w_obj.getdictvalue(space, name)
if w_value is None:
# fast method path: a function object in the class,
# nothing in the instance
@@ -103,7 +103,7 @@
w_descr = space.lookup(w_obj, methname)
typ = type(w_descr)
if typ is function.Function or typ is function.FunctionWithFixedCode:
- w_value = w_obj.getdictvalue_attr_is_in_class(space, methname)
+ w_value = w_obj.getdictvalue(space, methname)
if w_value is None:
# fast method path: a function object in the class,
# nothing in the instance
Modified: pypy/branch/cleanup-dict-impl/pypy/objspace/std/celldict.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/celldict.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/std/celldict.py Wed Oct 20 17:19:47 2010
@@ -47,7 +47,7 @@
else:
self._as_rdict().impl_fallback_setitem(w_key, w_value)
- def impl_setitem_str(self, name, w_value, shadows_type=True):
+ def impl_setitem_str(self, name, w_value):
self.getcell(name, True).w_value = w_value
def impl_delitem(self, w_key):
Modified: pypy/branch/cleanup-dict-impl/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/std/dictmultiobject.py Wed Oct 20 17:19:47 2010
@@ -53,10 +53,6 @@
from pypy.objspace.std.sharingdict import SharedDictImplementation
assert w_type is None
return SharedDictImplementation(space)
- elif (space.config.objspace.std.withshadowtracking and instance and
- classofinstance is not None):
- assert w_type is None
- return ShadowDetectingDictImplementation(space, classofinstance)
elif instance or strdict or module:
assert w_type is None
return StrDictImplementation(space)
@@ -112,7 +108,7 @@
#return w_value or None
raise NotImplementedError("abstract base class")
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
raise NotImplementedError("abstract base class")
def impl_setitem(self, w_key, w_value):
@@ -165,20 +161,13 @@
key = OPTIMIZED_BUILTINS[i]
return self.impl_getitem_str(key)
- # this method will only be seen whan a certain config option is used
- def impl_shadows_anything(self):
- return True
-
- def impl_set_shadows_anything(self):
- pass
-
# _________________________________________________________________
# fallback implementation methods
def impl_fallback_setitem(self, w_key, w_value):
self.r_dict_content[w_key] = w_value
- def impl_fallback_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_fallback_setitem_str(self, key, w_value):
return self.impl_fallback_setitem(self.space.wrap(key), w_value)
def impl_fallback_delitem(self, w_key):
@@ -211,18 +200,12 @@
key = OPTIMIZED_BUILTINS[i]
return self.impl_fallback_getitem_str(key)
- def impl_fallback_shadows_anything(self):
- return True
-
- def impl_fallback_set_shadows_anything(self):
- pass
-
implementation_methods = [
("getitem", 1),
("getitem_str", 1),
("length", 0),
- ("setitem_str", 3),
+ ("setitem_str", 2),
("setitem", 2),
("delitem", 1),
("iter", 0),
@@ -231,8 +214,6 @@
("keys", 0),
("clear", 0),
("get_builtin_indexed", 1),
- ("shadows_anything", 0),
- ("set_shadows_anything", 0),
]
@@ -312,7 +293,7 @@
else:
self._as_rdict().impl_fallback_setitem(w_key, w_value)
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
self.content[key] = w_value
def impl_delitem(self, w_key):
@@ -388,47 +369,12 @@
return None, None
-class ShadowDetectingDictImplementation(StrDictImplementation):
- def __init__(self, space, w_type):
- StrDictImplementation.__init__(self, space)
- self.w_type = w_type
- self.original_version_tag = w_type.version_tag()
- if self.original_version_tag is None:
- self._shadows_anything = True
- else:
- self._shadows_anything = False
-
- def impl_setitem_str(self, key, w_value, shadows_type=True):
- if shadows_type:
- self._shadows_anything = True
- StrDictImplementation.impl_setitem_str(
- self, key, w_value, shadows_type)
-
- def impl_setitem(self, w_key, w_value):
- space = self.space
- if space.is_w(space.type(w_key), space.w_str):
- if not self._shadows_anything:
- w_obj = self.w_type.lookup(space.str_w(w_key))
- if w_obj is not None:
- self._shadows_anything = True
- StrDictImplementation.impl_setitem_str(
- self, self.space.str_w(w_key), w_value, False)
- else:
- self._as_rdict().impl_fallback_setitem(w_key, w_value)
-
- def impl_shadows_anything(self):
- return (self._shadows_anything or
- self.w_type.version_tag() is not self.original_version_tag)
-
- def impl_set_shadows_anything(self):
- self._shadows_anything = True
-
class WaryDictImplementation(StrDictImplementation):
def __init__(self, space):
StrDictImplementation.__init__(self, space)
self.shadowed = [None] * len(BUILTIN_TO_INDEX)
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
i = BUILTIN_TO_INDEX.get(key, -1)
if i != -1:
self.shadowed[i] = w_value
@@ -558,7 +504,7 @@
self.info.writes += 1
self.content[w_key] = w_value
self.info.maxcontents = max(self.info.maxcontents, len(self.content))
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
self.info.setitem_strs += 1
self.impl_setitem(self.space.wrap(key), w_value)
def impl_delitem(self, w_key):
Modified: pypy/branch/cleanup-dict-impl/pypy/objspace/std/inlinedict.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/inlinedict.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/std/inlinedict.py Wed Oct 20 17:19:47 2010
@@ -91,14 +91,8 @@
w_dict = self.getdict()
return w_dict.getitem_str(attr)
- def getdictvalue_attr_is_in_class(self, space, attr):
- return self.getdictvalue(space, attr)
-
- def setdictvalue(self, space, attr, w_value, shadows_type=True):
+ def setdictvalue(self, space, attr, w_value):
if self._inlined_dict_valid():
- # XXX so far we ignore shadows_type, which is a small
- # performance-degradation if the JIT is not used (i.e. shadow
- # tracking does not work). Maybe we don't care.
self.impl_setitem_str(attr, w_value)
return True
w_dict = self.getdict()
Modified: pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py Wed Oct 20 17:19:47 2010
@@ -291,7 +291,7 @@
def getdictvalue(self, space, attrname):
return self._get_mapdict_map().read(self, (attrname, DICT))
- def setdictvalue(self, space, attrname, w_value, shadows_type=True):
+ def setdictvalue(self, space, attrname, w_value):
return self._get_mapdict_map().write(self, (attrname, DICT), w_value)
def deldictvalue(self, space, w_name):
@@ -506,8 +506,8 @@
def impl_getitem_str(self, key):
return self.w_obj.getdictvalue(self.space, key)
- def impl_setitem_str(self, key, w_value, shadows_type=True):
- flag = self.w_obj.setdictvalue(self.space, key, w_value, shadows_type)
+ def impl_setitem_str(self, key, w_value):
+ flag = self.w_obj.setdictvalue(self.space, key, w_value)
assert flag
def impl_setitem(self, w_key, w_value):
@@ -588,8 +588,6 @@
# ____________________________________________________________
# Magic caching
-# XXX we also would like getdictvalue_attr_is_in_class() above
-
class CacheEntry(object):
map = None
version_tag = None
Modified: pypy/branch/cleanup-dict-impl/pypy/objspace/std/objspace.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/objspace.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/std/objspace.py Wed Oct 20 17:19:47 2010
@@ -437,7 +437,7 @@
if is_data:
w_get = self.lookup(w_descr, "__get__")
if w_get is None:
- w_value = w_obj.getdictvalue_attr_is_in_class(self, name)
+ w_value = w_obj.getdictvalue(self, name)
if w_value is not None:
return w_value
if not is_data:
@@ -489,14 +489,12 @@
return w_obj.getitem(w_key)
return ObjSpace.finditem(self, w_obj, w_key)
- def setitem_str(self, w_obj, key, w_value, shadows_type=True):
+ def setitem_str(self, w_obj, key, w_value):
""" Same as setitem, but takes string instead of any wrapped object
-
- XXX what shadows_type means???
"""
if (isinstance(w_obj, W_DictMultiObject) and
not w_obj.user_overridden_class):
- w_obj.setitem_str(key, w_value, shadows_type)
+ w_obj.setitem_str(key, w_value)
else:
self.setitem(w_obj, self.wrap(key), w_value)
Modified: pypy/branch/cleanup-dict-impl/pypy/objspace/std/proxyobject.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/proxyobject.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/std/proxyobject.py Wed Oct 20 17:19:47 2010
@@ -43,7 +43,7 @@
raise
return None
- def setdictvalue(self, space, attr, w_value, shadows_type=True):
+ def setdictvalue(self, space, attr, w_value):
try:
space.call_function(self.w_controller, space.wrap('__setattr__'),
space.wrap(attr), w_value)
Modified: pypy/branch/cleanup-dict-impl/pypy/objspace/std/sharingdict.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/sharingdict.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/std/sharingdict.py Wed Oct 20 17:19:47 2010
@@ -87,7 +87,7 @@
self._as_rdict().impl_fallback_setitem(w_key, w_value)
@unroll_safe
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
i = self.structure.lookup_position(key)
if i != -1:
self.entries[i] = w_value
Modified: pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_dictmultiobject.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_dictmultiobject.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_dictmultiobject.py Wed Oct 20 17:19:47 2010
@@ -632,7 +632,6 @@
withsharingdict = False
withsmalldicts = False
withcelldict = False
- withshadowtracking = False
class opcodes:
CALL_LIKELY_BUILTIN = False
From arigo at codespeak.net Wed Oct 20 17:29:14 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 17:29:14 +0200 (CEST)
Subject: [pypy-svn] r78141 - in pypy/branch/cleanup-dict-impl/pypy: config
interpreter jit/tl objspace/std objspace/std/test
Message-ID: <20101020152914.17402282BE7@codespeak.net>
Author: arigo
Date: Wed Oct 20 17:29:13 2010
New Revision: 78141
Removed:
pypy/branch/cleanup-dict-impl/pypy/objspace/std/inlinedict.py
pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_inlinedict.py
Modified:
pypy/branch/cleanup-dict-impl/pypy/config/pypyoption.py
pypy/branch/cleanup-dict-impl/pypy/interpreter/typedef.py
pypy/branch/cleanup-dict-impl/pypy/jit/tl/pypyjit.py
Log:
(cfbolz, arigo) Kill inlinedict.
Modified: pypy/branch/cleanup-dict-impl/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/config/pypyoption.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/config/pypyoption.py Wed Oct 20 17:29:13 2010
@@ -237,15 +237,10 @@
"about dictionaries",
default=False),
- BoolOption("withinlineddict",
- "make instances more compact by revoming a level of indirection",
- default=False),
-
BoolOption("withmapdict",
"make instances really small but slow without the JIT",
default=False,
- requires=[("objspace.std.withinlineddict", False),
- ("objspace.std.withsharingdict", False),
+ requires=[("objspace.std.withsharingdict", False),
("objspace.std.getattributeshortcut", True),
("objspace.std.withtypeversion", True),
]),
@@ -336,7 +331,6 @@
config.objspace.std.suggest(newshortcut=True)
if type_system != 'ootype':
config.objspace.std.suggest(withsharingdict=True)
- config.objspace.std.suggest(withinlineddict=True)
# extra costly optimizations only go in level 3
if level == '3':
Modified: pypy/branch/cleanup-dict-impl/pypy/interpreter/typedef.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/interpreter/typedef.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/interpreter/typedef.py Wed Oct 20 17:29:13 2010
@@ -262,16 +262,7 @@
return self.slots_w[index]
add(Proto)
- wantdict = "dict" in features
- if wantdict and config.objspace.std.withinlineddict:
- from pypy.objspace.std.objectobject import W_ObjectObject
- from pypy.objspace.std.inlinedict import make_mixin
- if supercls is W_ObjectObject:
- Mixin = make_mixin(config)
- add(Mixin)
- wantdict = False
-
- if wantdict:
+ if "dict" in features:
base_user_setup = supercls.user_setup.im_func
if "user_setup" in body:
base_user_setup = body["user_setup"]
Modified: pypy/branch/cleanup-dict-impl/pypy/jit/tl/pypyjit.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/jit/tl/pypyjit.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/jit/tl/pypyjit.py Wed Oct 20 17:29:13 2010
@@ -44,7 +44,6 @@
config.objspace.usemodules._ffi = True
#
set_pypy_opt_level(config, level='jit')
-config.objspace.std.withinlineddict = True
if BACKEND == 'c':
config.objspace.std.multimethods = 'mrd'
From arigo at codespeak.net Wed Oct 20 17:42:08 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 17:42:08 +0200 (CEST)
Subject: [pypy-svn] r78142 - in pypy/branch/cleanup-dict-impl/pypy: config
config/test module/__builtin__/test objspace/std objspace/std/test
Message-ID: <20101020154208.42918282BE7@codespeak.net>
Author: arigo
Date: Wed Oct 20 17:42:06 2010
New Revision: 78142
Removed:
pypy/branch/cleanup-dict-impl/pypy/objspace/std/sharingdict.py
pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_sharingdict.py
Modified:
pypy/branch/cleanup-dict-impl/pypy/config/pypyoption.py
pypy/branch/cleanup-dict-impl/pypy/config/test/test_pypyoption.py
pypy/branch/cleanup-dict-impl/pypy/module/__builtin__/test/test_classobj.py
pypy/branch/cleanup-dict-impl/pypy/objspace/std/dictmultiobject.py
pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_dictmultiobject.py
Log:
(cfbolz, arigo) Kill sharingdict.
Modified: pypy/branch/cleanup-dict-impl/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/config/pypyoption.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/config/pypyoption.py Wed Oct 20 17:42:06 2010
@@ -228,10 +228,6 @@
requires=[("objspace.opcodes.CALL_LIKELY_BUILTIN", False),
("objspace.honor__builtins__", False)]),
- BoolOption("withsharingdict",
- "use dictionaries that share the keys part",
- default=False),
-
BoolOption("withdictmeasurement",
"create huge files with masses of information "
"about dictionaries",
@@ -240,8 +236,7 @@
BoolOption("withmapdict",
"make instances really small but slow without the JIT",
default=False,
- requires=[("objspace.std.withsharingdict", False),
- ("objspace.std.getattributeshortcut", True),
+ requires=[("objspace.std.getattributeshortcut", True),
("objspace.std.withtypeversion", True),
]),
@@ -329,8 +324,6 @@
config.objspace.std.suggest(optimized_list_getitem=True)
config.objspace.std.suggest(getattributeshortcut=True)
config.objspace.std.suggest(newshortcut=True)
- if type_system != 'ootype':
- config.objspace.std.suggest(withsharingdict=True)
# extra costly optimizations only go in level 3
if level == '3':
@@ -359,7 +352,7 @@
# extra optimizations with the JIT
if level == 'jit':
config.objspace.std.suggest(withcelldict=True)
- #config.objspace.std.suggest(withmapdict=True)
+ config.objspace.std.suggest(withmapdict=True)
def enable_allworkingmodules(config):
Modified: pypy/branch/cleanup-dict-impl/pypy/config/test/test_pypyoption.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/config/test/test_pypyoption.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/config/test/test_pypyoption.py Wed Oct 20 17:42:06 2010
@@ -47,7 +47,7 @@
def test_set_pypy_opt_level():
conf = get_pypy_config()
set_pypy_opt_level(conf, '2')
- assert conf.objspace.std.withsharingdict
+ assert conf.objspace.std.newshortcut
conf = get_pypy_config()
set_pypy_opt_level(conf, '0')
assert not conf.objspace.std.newshortcut
Modified: pypy/branch/cleanup-dict-impl/pypy/module/__builtin__/test/test_classobj.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/module/__builtin__/test/test_classobj.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/module/__builtin__/test/test_classobj.py Wed Oct 20 17:42:06 2010
@@ -954,28 +954,7 @@
raises(TypeError, descr.__delete__, a)
-class AppTestOldStyleSharing(AppTestOldstyle):
- def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True})
- if option.runappdirect:
- py.test.skip("can only be run on py.py")
- def is_sharing(space, w_inst):
- from pypy.objspace.std.sharingdict import SharedDictImplementation
- w_d = w_inst.getdict()
- return space.wrap(isinstance(w_d, SharedDictImplementation) and w_d.r_dict_content is None)
- cls.w_is_sharing = cls.space.wrap(gateway.interp2app(is_sharing))
-
-
- def test_real_sharing(self):
- class A:
- def __init__(self):
- self.x = 42
- A1, A2, A3 = A(), A(), A()
- assert self.is_sharing(A3)
- assert self.is_sharing(A2)
- assert self.is_sharing(A1)
-
-class AppTestOldStyleModDict(object):
+class AppTestOldStyleClassStrDict(object):
def setup_class(cls):
if option.runappdirect:
py.test.skip("can only be run on py.py")
Modified: pypy/branch/cleanup-dict-impl/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/std/dictmultiobject.py Wed Oct 20 17:42:06 2010
@@ -49,10 +49,6 @@
elif space.config.objspace.std.withdictmeasurement:
assert w_type is None
return MeasuringDictImplementation(space)
- elif space.config.objspace.std.withsharingdict and instance:
- from pypy.objspace.std.sharingdict import SharedDictImplementation
- assert w_type is None
- return SharedDictImplementation(space)
elif instance or strdict or module:
assert w_type is None
return StrDictImplementation(space)
Modified: pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_dictmultiobject.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_dictmultiobject.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_dictmultiobject.py Wed Oct 20 17:42:06 2010
@@ -4,7 +4,6 @@
StrDictImplementation
from pypy.objspace.std.celldict import ModuleDictImplementation
-from pypy.objspace.std.sharingdict import SharedDictImplementation
from pypy.conftest import gettestobjspace
@@ -510,32 +509,6 @@
assert getattr(a, s) == 42
-class TestW_DictSharing(TestW_DictObject):
- def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True})
-
-class AppTest_DictSharing(AppTest_DictObject):
- def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True})
-
- def test_values_does_not_share(self):
- class A(object):
- pass
- a = A()
- a.abc = 12
- l = a.__dict__.values()
- assert l == [12]
- l[0] = 24
- assert a.abc == 12
-
- def test_items(self):
- class A(object):
- pass
- a = A()
- a.abc = 12
- a.__dict__.items() == [("abc", 12)]
-
-
class AppTestModuleDict(object):
def setup_class(cls):
cls.space = gettestobjspace(**{"objspace.std.withcelldict": True})
@@ -629,7 +602,6 @@
class objspace:
class std:
withdictmeasurement = False
- withsharingdict = False
withsmalldicts = False
withcelldict = False
class opcodes:
@@ -766,9 +738,6 @@
string = "int"
string2 = "isinstance"
-class TestSharedDictImplementation(BaseTestRDictImplementation):
- ImplementionClass = SharedDictImplementation
-
class BaseTestDevolvedDictImplementation(BaseTestRDictImplementation):
def fill_impl(self):
@@ -790,8 +759,6 @@
string = "int"
string2 = "isinstance"
-class TestDevolvedSharedDictImplementation(BaseTestDevolvedDictImplementation):
- ImplementionClass = SharedDictImplementation
def test_module_uses_strdict():
fakespace = FakeSpace()
From arigo at codespeak.net Wed Oct 20 17:45:14 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 17:45:14 +0200 (CEST)
Subject: [pypy-svn] r78143 -
pypy/branch/cleanup-dict-impl/pypy/module/pypyjit/test
Message-ID: <20101020154514.69CC9282BDA@codespeak.net>
Author: arigo
Date: Wed Oct 20 17:45:13 2010
New Revision: 78143
Modified:
pypy/branch/cleanup-dict-impl/pypy/module/pypyjit/test/test_pypy_c.py
Log:
(cfbolz, arigo) Now withmapdict is the default with the JIT.
Modified: pypy/branch/cleanup-dict-impl/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/module/pypyjit/test/test_pypy_c.py Wed Oct 20 17:45:13 2010
@@ -278,7 +278,7 @@
assert len(ops) == 2
assert not ops[0].get_opnames("call")
assert not ops[0].get_opnames("new")
- assert len(ops[0].get_opnames("guard")) <= 3 # we get 2 withmapdict
+ assert len(ops[0].get_opnames("guard")) <= 2
assert not ops[1] # second LOOKUP_METHOD folded away
ops = self.get_by_bytecode("CALL_METHOD")
@@ -323,8 +323,8 @@
assert len(ops) == 2
assert not ops[0].get_opnames("call")
assert not ops[0].get_opnames("new")
- assert len(ops[0].get_opnames("guard")) <= 3 # we get 2 withmapdict
- assert len(ops[0].get_opnames("getfield")) <= 5 # we get <5 withmapdict
+ assert len(ops[0].get_opnames("guard")) <= 2
+ assert len(ops[0].get_opnames("getfield")) <= 4
assert not ops[1] # second LOOKUP_METHOD folded away
def test_default_and_kw(self):
From arigo at codespeak.net Wed Oct 20 17:59:13 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 20 Oct 2010 17:59:13 +0200 (CEST)
Subject: [pypy-svn] r78144 -
pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests
Message-ID: <20101020155913.BCB9E282BDA@codespeak.net>
Author: arigo
Date: Wed Oct 20 17:59:12 2010
New Revision: 78144
Modified:
pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py
Log:
Fix for 64-bits.
Modified: pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py
==============================================================================
--- pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py (original)
+++ pypy/trunk/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py Wed Oct 20 17:59:12 2010
@@ -10,7 +10,7 @@
"""
def test_array_of_pointers(self):
# tests array item assignements & pointer.contents = ...
- A = POINTER(c_int) * 24
+ A = POINTER(c_long) * 24
a = A()
l = c_long(2)
p = pointer(l)
From afa at codespeak.net Wed Oct 20 18:24:24 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 18:24:24 +0200 (CEST)
Subject: [pypy-svn] r78145 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101020162424.76AEB282BDA@codespeak.net>
Author: afa
Date: Wed Oct 20 18:24:22 2010
New Revision: 78145
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
Log:
Add finalizer for IOBase (and all subclasses)
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Wed Oct 20 18:24:22 2010
@@ -16,6 +16,7 @@
# XXX: IOBase thinks it has to maintain its own internal state in
# `__IOBase_closed` and call flush() by itself, but it is redundant
# with whatever behaviour a non-trivial derived class will implement.
+ self.space = space
self.__IOBase_closed = False
def _closed(self, space):
@@ -26,6 +27,20 @@
return True
return False
+ def __del__(self):
+ space = self.space
+ w_closed = space.findattr(self, space.wrap('closed'))
+ try:
+ # If `closed` doesn't exist or can't be evaluated as bool, then
+ # the object is probably in an unusable state, so ignore.
+ if w_closed is not None and not space.is_true(w_closed):
+ space.call_method(self, "close")
+ except OperationError:
+ # Silencing I/O errors is bad, but printing spurious tracebacks is
+ # equally as bad, and potentially more frequent (because of
+ # shutdown issues).
+ pass
+
def _CLOSED(self):
# Use this macro whenever you want to check the internal `closed`
# status of the IOBase object rather than the virtual `closed`
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Wed Oct 20 18:24:22 2010
@@ -42,6 +42,25 @@
import _io
e = _io.UnsupportedOperation("seek")
+ def test_destructor(self):
+ import io
+ io.IOBase()
+
+ record = []
+ class MyIO(io.IOBase):
+ def __del__(self):
+ record.append(1)
+ super(MyIO, self).__del__()
+ def close(self):
+ record.append(2)
+ super(MyIO, self).close()
+ def flush(self):
+ record.append(3)
+ super(MyIO, self).flush()
+ MyIO()
+ import gc; gc.collect()
+ assert record == [1, 2, 3]
+
class AppTestOpen:
def setup_class(cls):
tmpfile = udir.join('tmpfile').ensure()
From afa at codespeak.net Wed Oct 20 20:03:58 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 20:03:58 +0200 (CEST)
Subject: [pypy-svn] r78147 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101020180358.72176282BE8@codespeak.net>
Author: afa
Date: Wed Oct 20 20:03:56 2010
New Revision: 78147
Added:
pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
Log:
Start a BufferedIO
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Wed Oct 20 20:03:56 2010
@@ -1,19 +1,80 @@
-from pypy.module._io.interp_iobase import W_IOBase
from pypy.interpreter.typedef import (
TypeDef, generic_new_descr)
+from pypy.interpreter.gateway import interp2app, unwrap_spec
+from pypy.interpreter.baseobjspace import ObjSpace, W_Root
+from pypy.interpreter.error import OperationError
+from pypy.rpython.lltypesystem import lltype, rffi
+from pypy.module._io.interp_iobase import W_IOBase
+from pypy.module._io.interp_io import DEFAULT_BUFFER_SIZE
+from pypy.module.thread.os_lock import Lock
class W_BufferedIOBase(W_IOBase):
- pass
+ def __init__(self, space):
+ W_IOBase.__init__(self, space)
+ self.buffer = None
+ self.lock = None
+
+ def _init(self, space):
+ if self.buffer_size <= 0:
+ raise OperationError(space.w_ValueError, space.wrap(
+ "buffer size must be strictly positive"))
+
+ if self.buffer:
+ lltype.free(self.buffer, flavor='raw')
+ self.buffer = lltype.malloc(rffi.CCHARP.TO, self.buffer_size,
+ flavor='raw')
+
+ ## XXX cannot free a Lock?
+ ## if self.lock:
+ ## self.lock.free()
+ self.lock = Lock(space)
+
+ try:
+ self._raw_tell(space)
+ except OperationError:
+ pass
+
+ def _raw_tell(self, space):
+ w_pos = space.call_method(self.raw, "tell")
+ pos = space.r_longlong_w(w_pos)
+ if pos < 0:
+ raise OperationError(space.w_IOError, space.wrap(
+ "raw stream returned invalid position"))
+
+ self.abs_pos = pos
+ return pos
+
W_BufferedIOBase.typedef = TypeDef(
'_BufferedIOBase', W_IOBase.typedef,
__new__ = generic_new_descr(W_BufferedIOBase),
)
class W_BufferedReader(W_BufferedIOBase):
- pass
+ def __init__(self, space):
+ W_BufferedIOBase.__init__(self, space)
+ self.ok = False
+ self.detached = False
+
+ @unwrap_spec('self', ObjSpace, W_Root, int)
+ def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE):
+ raw = space.interp_w(W_IOBase, w_raw)
+ raw.check_readable_w(space)
+
+ self.raw = raw
+ self.buffer_size = buffer_size
+ self.readable = True
+ self.writable = False
+
+ self._init(space)
+ self._reset_buf()
+
+ def _reset_buf(self):
+ self.read_end = -1
+
W_BufferedReader.typedef = TypeDef(
'BufferedReader', W_BufferedIOBase.typedef,
__new__ = generic_new_descr(W_BufferedReader),
+ __init__ = interp2app(W_BufferedReader.descr_init),
)
class W_BufferedWriter(W_BufferedIOBase):
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Wed Oct 20 20:03:56 2010
@@ -110,6 +110,20 @@
return space.w_False
@unwrap_spec('self', ObjSpace)
+ def check_readable_w(self, space):
+ if not space.is_true(space.call_method(self, 'readable')):
+ raise OperationError(
+ space.w_IOError,
+ space.wrap("file or stream is not readable"))
+
+ @unwrap_spec('self', ObjSpace)
+ def check_writable_w(self, space):
+ if not space.is_true(space.call_method(self, 'writable')):
+ raise OperationError(
+ space.w_IOError,
+ space.wrap("file or stream is not writable"))
+
+ @unwrap_spec('self', ObjSpace)
def check_seekable_w(self, space):
if not space.is_true(space.call_method(self, 'seekable')):
raise OperationError(
@@ -214,6 +228,8 @@
readable = interp2app(W_IOBase.readable_w),
writable = interp2app(W_IOBase.writable_w),
seekable = interp2app(W_IOBase.seekable_w),
+ _checkReadable = interp2app(W_IOBase.check_readable_w),
+ _checkWritable = interp2app(W_IOBase.check_writable_w),
_checkSeekable = interp2app(W_IOBase.check_seekable_w),
closed = GetSetProperty(W_IOBase.closed_get_w),
Added: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Wed Oct 20 20:03:56 2010
@@ -0,0 +1,14 @@
+from pypy.conftest import gettestobjspace
+from pypy.tool.udir import udir
+
+class AppTestBufferedIO:
+ def setup_class(cls):
+ cls.space = gettestobjspace(usemodules=['_io'])
+ tmpfile = udir.join('tmpfile')
+ tmpfile.write("a\nb\nc", mode='wb')
+ cls.w_tmpfile = cls.space.wrap(str(tmpfile))
+
+ def test_simple(self):
+ import _io
+ raw = _io.FileIO(self.tmpfile)
+ _io.BufferedReader(raw)
From afa at codespeak.net Wed Oct 20 23:44:18 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 20 Oct 2010 23:44:18 +0200 (CEST)
Subject: [pypy-svn] r78148 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101020214418.87F84282BDC@codespeak.net>
Author: afa
Date: Wed Oct 20 23:44:16 2010
New Revision: 78148
Added:
pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (contents, props changed)
Modified:
pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (contents, props changed)
Log:
Add missing file
Added: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Wed Oct 20 23:44:16 2010
@@ -0,0 +1,17 @@
+from pypy.module._io.interp_iobase import W_IOBase
+from pypy.interpreter.typedef import (
+ TypeDef, generic_new_descr)
+
+class W_TextIOBase(W_IOBase):
+ pass
+W_TextIOBase.typedef = TypeDef(
+ '_TextIOBase', W_IOBase.typedef,
+ __new__ = generic_new_descr(W_TextIOBase),
+ )
+
+class W_TextIOWrapper(W_TextIOBase):
+ pass
+W_TextIOWrapper.typedef = TypeDef(
+ 'TextIOWrapper', W_TextIOBase.typedef,
+ __new__ = generic_new_descr(W_TextIOWrapper),
+ )
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Wed Oct 20 23:44:16 2010
@@ -1,14 +1,14 @@
-from pypy.conftest import gettestobjspace
-from pypy.tool.udir import udir
-
-class AppTestBufferedIO:
- def setup_class(cls):
- cls.space = gettestobjspace(usemodules=['_io'])
- tmpfile = udir.join('tmpfile')
- tmpfile.write("a\nb\nc", mode='wb')
- cls.w_tmpfile = cls.space.wrap(str(tmpfile))
-
- def test_simple(self):
- import _io
- raw = _io.FileIO(self.tmpfile)
- _io.BufferedReader(raw)
+from pypy.conftest import gettestobjspace
+from pypy.tool.udir import udir
+
+class AppTestBufferedIO:
+ def setup_class(cls):
+ cls.space = gettestobjspace(usemodules=['_io'])
+ tmpfile = udir.join('tmpfile')
+ tmpfile.write("a\nb\nc", mode='wb')
+ cls.w_tmpfile = cls.space.wrap(str(tmpfile))
+
+ def test_simple(self):
+ import _io
+ raw = _io.FileIO(self.tmpfile)
+ _io.BufferedReader(raw)
From afa at codespeak.net Thu Oct 21 00:38:44 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 21 Oct 2010 00:38:44 +0200 (CEST)
Subject: [pypy-svn] r78149 - pypy/branch/fast-forward/pypy/module/_io
Message-ID: <20101020223844.2035536E3D6@codespeak.net>
Author: afa
Date: Thu Oct 21 00:38:42 2010
New Revision: 78149
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
Log:
Fix translation
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Thu Oct 21 00:38:42 2010
@@ -11,7 +11,7 @@
class W_BufferedIOBase(W_IOBase):
def __init__(self, space):
W_IOBase.__init__(self, space)
- self.buffer = None
+ self.buffer = lltype.nullptr(rffi.CCHARP.TO)
self.lock = None
def _init(self, space):
From trundle at codespeak.net Thu Oct 21 01:44:13 2010
From: trundle at codespeak.net (trundle at codespeak.net)
Date: Thu, 21 Oct 2010 01:44:13 +0200 (CEST)
Subject: [pypy-svn] r78150 - in pypy/branch/fast-forward: lib-python
pypy/interpreter pypy/objspace/flow
Message-ID: <20101020234413.E3CD8282BDC@codespeak.net>
Author: trundle
Date: Thu Oct 21 01:44:12 2010
New Revision: 78150
Modified:
pypy/branch/fast-forward/lib-python/TODO
pypy/branch/fast-forward/pypy/interpreter/pyopcode.py
pypy/branch/fast-forward/pypy/objspace/flow/objspace.py
Log:
Catching a string exception now raises a DeprecationWarning.
Modified: pypy/branch/fast-forward/lib-python/TODO
==============================================================================
--- pypy/branch/fast-forward/lib-python/TODO (original)
+++ pypy/branch/fast-forward/lib-python/TODO Thu Oct 21 01:44:12 2010
@@ -32,9 +32,6 @@
- Ast objects should be picklable, see in pypy/module/_ast/test/test_ast.py:
test_pickle()
-- catching a string should emit a DeprecationWarning ("catching of
- string exceptions is deprecated")
-
- missing builtin: memoryview
Longer tasks
Modified: pypy/branch/fast-forward/pypy/interpreter/pyopcode.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyopcode.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyopcode.py Thu Oct 21 01:44:12 2010
@@ -747,6 +747,16 @@
return self.space.not_(self.space.is_(w_1, w_2))
def cmp_exc_match(self, w_1, w_2):
+ if self.space.is_true(self.space.isinstance(w_2, self.space.w_tuple)):
+ for w_t in self.space.fixedview(w_2):
+ if self.space.is_true(self.space.isinstance(w_t,
+ self.space.w_str)):
+ self.space.warn("catching of string exceptions is "
+ "deprecated",
+ self.space.w_DeprecationWarning)
+ elif self.space.is_true(self.space.isinstance(w_2, self.space.w_str)):
+ self.space.warn("catching of string exceptions is deprecated",
+ self.space.w_DeprecationWarning)
return self.space.newbool(self.space.exception_match(w_1, w_2))
def COMPARE_OP(self, testnum, next_instr):
Modified: pypy/branch/fast-forward/pypy/objspace/flow/objspace.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/flow/objspace.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/flow/objspace.py Thu Oct 21 01:44:12 2010
@@ -77,6 +77,9 @@
self.specialcases = {}
#self.make_builtins()
#self.make_sys()
+ # w_str is needed because cmp_exc_match of frames checks against it,
+ # as string exceptions are deprecated
+ self.w_str = Constant(str)
# objects which should keep their SomeObjectness
self.not_really_const = NOT_REALLY_CONST
From hakanardo at codespeak.net Thu Oct 21 08:20:15 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Thu, 21 Oct 2010 08:20:15 +0200 (CEST)
Subject: [pypy-svn] r78151 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test
Message-ID: <20101021062015.4FFD3282BDC@codespeak.net>
Author: hakanardo
Date: Thu Oct 21 08:20:12 2010
New Revision: 78151
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
pure_operations inherited from first iteration
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Thu Oct 21 08:20:12 2010
@@ -43,6 +43,11 @@
if not v.is_constant() and v.box:
v.fromstart = True
+ for op in self.optimizer.pure_operations.values():
+ v = self.getvalue(op.result)
+ v.fromstart = True
+
+
self.snapshot_map ={None: None}
inputargs = []
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Thu Oct 21 08:20:12 2010
@@ -121,6 +121,7 @@
fieldvalue = self._fields[ofs]
fieldvalue.get_args_for_fail(modifier)
+ # FIXME: circular references
def get_forced_boxes(self):
if self.box is None:
lst = self._get_field_descr_list()
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Thu Oct 21 08:20:12 2010
@@ -1131,8 +1131,8 @@
jump(i1)
"""
expected = """
- [i]
- jump(5)
+ []
+ jump()
"""
self.node.value = 5
self.optimize_loop(ops, 'Not', expected)
@@ -3590,6 +3590,23 @@
"""
self.optimize_loop(ops, 'Not, Not, Not', expected)
+ def test_pure(self):
+ ops = """
+ [p42]
+ p53 = getfield_gc(ConstPtr(myptr), descr=nextdescr)
+ p59 = getfield_gc_pure(p53, descr=valuedescr)
+ i61 = call(1, p59, descr=nonwritedescr)
+ jump(p42)
+ """
+ expected = """
+ [p42, p59]
+ i61 = call(1, p59, descr=nonwritedescr)
+ jump(p42, p59)
+
+ """
+ self.node.value = 5
+ self.optimize_loop(ops, 'Not', expected)
+
def test_addsub_ovf(self):
ops = """
[i0]
From afa at codespeak.net Thu Oct 21 08:45:30 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 21 Oct 2010 08:45:30 +0200 (CEST)
Subject: [pypy-svn] r78152 - in pypy/branch/fast-forward/pypy/module/array:
. test
Message-ID: <20101021064530.0AA06282BDC@codespeak.net>
Author: afa
Date: Thu Oct 21 08:45:29 2010
New Revision: 78152
Modified:
pypy/branch/fast-forward/pypy/module/array/interp_array.py
pypy/branch/fast-forward/pypy/module/array/test/test_array.py
Log:
Add overflow checks in array module
Modified: pypy/branch/fast-forward/pypy/module/array/interp_array.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/array/interp_array.py (original)
+++ pypy/branch/fast-forward/pypy/module/array/interp_array.py Thu Oct 21 08:45:29 2010
@@ -6,6 +6,7 @@
from pypy.rlib.jit import dont_look_inside
from pypy.rlib import rgc
from pypy.rlib.unroll import unrolling_iterable
+from pypy.rlib.rarithmetic import ovfcheck
from pypy.rlib.rstruct.runpack import runpack
from pypy.interpreter.argument import Arguments, Signature
from pypy.interpreter.baseobjspace import ObjSpace, W_Root, Wrappable
@@ -488,7 +489,11 @@
raise
a = mytype.w_class(space)
repeat = max(repeat, 0)
- a.setlen(self.len * repeat)
+ try:
+ newlen = ovfcheck(self.len * repeat)
+ except OverflowError:
+ raise MemoryError
+ a.setlen(newlen)
for r in range(repeat):
for i in range(self.len):
a.buffer[r * self.len + i] = self.buffer[i]
@@ -506,7 +511,11 @@
raise
oldlen = self.len
repeat = max(repeat, 0)
- self.setlen(self.len * repeat)
+ try:
+ newlen = ovfcheck(self.len * repeat)
+ except OverflowError:
+ raise MemoryError
+ self.setlen(newlen)
for r in range(1, repeat):
for i in range(oldlen):
self.buffer[r * oldlen + i] = self.buffer[i]
@@ -544,7 +553,10 @@
raise OperationError(space.w_TypeError, space.wrap(msg))
n = space.int_w(w_n)
- size = self.itemsize * n
+ try:
+ size = ovfcheck(self.itemsize * n)
+ except OverflowError:
+ raise MemoryError
w_item = space.call_method(w_f, 'read', space.wrap(size))
item = space.str_w(w_item)
if len(item) < size:
Modified: pypy/branch/fast-forward/pypy/module/array/test/test_array.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/array/test/test_array.py (original)
+++ pypy/branch/fast-forward/pypy/module/array/test/test_array.py Thu Oct 21 08:45:29 2010
@@ -1,4 +1,5 @@
from pypy.conftest import gettestobjspace
+import sys
import py
import py.test
@@ -590,6 +591,9 @@
a += self.array('i', (7,))
assert repr(a) == "array('i', [1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 7])"
+ raises(MemoryError, "a * self.maxint")
+ raises(MemoryError, "a *= self.maxint")
+
raises(TypeError, "a = self.array('i') + 2")
raises(TypeError, "self.array('i') + self.array('b')")
a = self.array('i')
@@ -812,7 +816,8 @@
cls.array = array.array
import struct
cls.struct = struct
- cls.tempfile = str(py.test.ensuretemp('array').join('tmpfile'))
+ cls.tempfile = str(py.test.ensuretemp('array').join('tmpfile'))
+ cls.maxint = sys.maxint
class AppTestArray(BaseArrayTests):
def setup_class(cls):
@@ -831,6 +836,7 @@
""")
cls.w_tempfile = cls.space.wrap(
str(py.test.ensuretemp('array').join('tmpfile')))
+ cls.w_maxint = cls.space.wrap(sys.maxint)
From afa at codespeak.net Thu Oct 21 08:59:15 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 21 Oct 2010 08:59:15 +0200 (CEST)
Subject: [pypy-svn] r78153 - pypy/branch/fast-forward/pypy/module/select
Message-ID: <20101021065915.268D5282BDC@codespeak.net>
Author: afa
Date: Thu Oct 21 08:59:13 2010
New Revision: 78153
Modified:
pypy/branch/fast-forward/pypy/module/select/interp_select.py
Log:
Add Poll.modify()
Modified: pypy/branch/fast-forward/pypy/module/select/interp_select.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/select/interp_select.py (original)
+++ pypy/branch/fast-forward/pypy/module/select/interp_select.py Thu Oct 21 08:59:13 2010
@@ -2,8 +2,10 @@
from pypy.interpreter.typedef import TypeDef
from pypy.interpreter.baseobjspace import Wrappable
from pypy.interpreter.gateway import W_Root, ObjSpace, interp2app
-from pypy.interpreter.error import OperationError, operationerrfmt
+from pypy.interpreter.error import (
+ OperationError, operationerrfmt, wrap_oserror)
from pypy.rlib import rpoll
+import errno
defaultevents = rpoll.POLLIN | rpoll.POLLOUT | rpoll.POLLPRI
@@ -41,6 +43,14 @@
self.fddict[fd] = events
register.unwrap_spec = ['self', ObjSpace, W_Root, int]
+ def modify(self, space, w_fd, events):
+ fd = as_fd_w(space, w_fd)
+ if fd not in self.fddict:
+ raise wrap_oserror(space, OSError(errno.ENOENT, "poll.modify"),
+ exception_name='w_IOError')
+ self.fddict[fd] = events
+ modify.unwrap_spec = ['self', ObjSpace, W_Root, int]
+
def unregister(self, space, w_fd):
fd = as_fd_w(space, w_fd)
try:
@@ -81,7 +91,7 @@
poll.unwrap_spec = ['self', ObjSpace, W_Root]
pollmethods = {}
-for methodname in 'register unregister poll'.split():
+for methodname in 'register modify unregister poll'.split():
method = getattr(Poll, methodname)
assert hasattr(method,'unwrap_spec'), methodname
assert method.im_func.func_code.co_argcount == len(method.unwrap_spec), methodname
From afa at codespeak.net Thu Oct 21 09:00:47 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 21 Oct 2010 09:00:47 +0200 (CEST)
Subject: [pypy-svn] r78154 - pypy/branch/fast-forward/lib-python
Message-ID: <20101021070047.59434282BDC@codespeak.net>
Author: afa
Date: Thu Oct 21 09:00:45 2010
New Revision: 78154
Modified:
pypy/branch/fast-forward/lib-python/TODO
Log:
Add a TODO item
Modified: pypy/branch/fast-forward/lib-python/TODO
==============================================================================
--- pypy/branch/fast-forward/lib-python/TODO (original)
+++ pypy/branch/fast-forward/lib-python/TODO Thu Oct 21 09:00:45 2010
@@ -26,6 +26,8 @@
- Fix fcntl.fcntl(fd, fcntl.F_NOTIFY, fcntl.DN_MULTISHOT) on 32bit platform.
+- missing functions in itertools: combinations, product, compress...
+
Medium tasks
------------
From arigo at codespeak.net Thu Oct 21 11:22:13 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 21 Oct 2010 11:22:13 +0200 (CEST)
Subject: [pypy-svn] r78156 - pypy/trunk/pypy/rlib
Message-ID: <20101021092213.1F30F282BDC@codespeak.net>
Author: arigo
Date: Thu Oct 21 11:22:12 2010
New Revision: 78156
Modified:
pypy/trunk/pypy/rlib/jit.py
Log:
Fix order-dependency annotation issues.
Modified: pypy/trunk/pypy/rlib/jit.py
==============================================================================
--- pypy/trunk/pypy/rlib/jit.py (original)
+++ pypy/trunk/pypy/rlib/jit.py Thu Oct 21 11:22:12 2010
@@ -380,6 +380,11 @@
def compute_result_annotation(self, **kwds_s):
from pypy.annotation import model as annmodel
+
+ if self.instance.__name__ == 'jit_merge_point':
+ if not self.annotate_hooks(**kwds_s):
+ return None # wrong order, try again later
+
driver = self.instance.im_self
keys = kwds_s.keys()
keys.sort()
@@ -408,22 +413,19 @@
key[2:])
cache[key] = s_value
- if self.instance.__name__ == 'jit_merge_point':
- self.annotate_hooks(**kwds_s)
-
return annmodel.s_None
def annotate_hooks(self, **kwds_s):
driver = self.instance.im_self
s_jitcell = self.bookkeeper.valueoftype(BaseJitCell)
- self.annotate_hook(driver.get_jitcell_at, driver.greens, **kwds_s)
- self.annotate_hook(driver.set_jitcell_at, driver.greens, [s_jitcell],
- **kwds_s)
- self.annotate_hook(driver.get_printable_location, driver.greens, **kwds_s)
+ h = self.annotate_hook
+ return (h(driver.get_jitcell_at, driver.greens, **kwds_s)
+ and h(driver.set_jitcell_at, driver.greens, [s_jitcell], **kwds_s)
+ and h(driver.get_printable_location, driver.greens, **kwds_s))
def annotate_hook(self, func, variables, args_s=[], **kwds_s):
if func is None:
- return
+ return True
bk = self.bookkeeper
s_func = bk.immutablevalue(func)
uniquekey = 'jitdriver.%s' % func.func_name
@@ -435,9 +437,11 @@
objname, fieldname = name.split('.')
s_instance = kwds_s['s_' + objname]
s_arg = s_instance.classdef.about_attribute(fieldname)
- assert s_arg is not None
+ if s_arg is None:
+ return False # wrong order, try again later
args_s.append(s_arg)
bk.emulate_pbc_call(uniquekey, s_func, args_s)
+ return True
def specialize_call(self, hop, **kwds_i):
# XXX to be complete, this could also check that the concretetype
From arigo at codespeak.net Thu Oct 21 11:23:06 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 21 Oct 2010 11:23:06 +0200 (CEST)
Subject: [pypy-svn] r78157 - pypy/trunk/pypy/rpython/lltypesystem
Message-ID: <20101021092306.D4EC7282BDC@codespeak.net>
Author: arigo
Date: Thu Oct 21 11:23:05 2010
New Revision: 78157
Modified:
pypy/trunk/pypy/rpython/lltypesystem/llarena.py
pypy/trunk/pypy/rpython/lltypesystem/rffi.py
Log:
A non-rpython version of str2charp that allocates an immortal string.
Modified: pypy/trunk/pypy/rpython/lltypesystem/llarena.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/llarena.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/llarena.py Thu Oct 21 11:23:05 2010
@@ -440,7 +440,7 @@
[rffi.INT],
rffi.INT,
sandboxsafe=True, _nowrapper=True)
- _dev_zero = rffi.str2charp('/dev/zero') # prebuilt
+ _dev_zero = rffi.str2charp_immortal('/dev/zero') # prebuilt
def clear_large_memory_chunk(baseaddr, size):
# on some Unixy platforms, reading from /dev/zero is the fastest way
Modified: pypy/trunk/pypy/rpython/lltypesystem/rffi.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/rffi.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/rffi.py Thu Oct 21 11:23:05 2010
@@ -607,6 +607,15 @@
return array
str2charp._annenforceargs_ = [strtype]
+ def str2charp_immortal(s):
+ "NOT_RPYTHON"
+ array = lltype.malloc(TYPEP.TO, len(s) + 1, flavor='raw',
+ immortal=True)
+ for i in range(len(s)):
+ array[i] = s[i]
+ array[len(s)] = lastchar
+ return array
+
def free_charp(cp):
lltype.free(cp, flavor='raw')
@@ -725,19 +734,19 @@
l = [cp[i] for i in range(size)]
return emptystr.join(l)
- return (str2charp, free_charp, charp2str,
+ return (str2charp, str2charp_immortal, free_charp, charp2str,
get_nonmovingbuffer, free_nonmovingbuffer,
alloc_buffer, str_from_buffer, keep_buffer_alive_until_here,
charp2strn, charpsize2str,
)
-(str2charp, free_charp, charp2str,
+(str2charp, str2charp_immortal, free_charp, charp2str,
get_nonmovingbuffer, free_nonmovingbuffer,
alloc_buffer, str_from_buffer, keep_buffer_alive_until_here,
charp2strn, charpsize2str,
) = make_string_mappings(str)
-(unicode2wcharp, free_wcharp, wcharp2unicode,
+(unicode2wcharp, unicode2wcharp_immortal, free_wcharp, wcharp2unicode,
get_nonmoving_unicodebuffer, free_nonmoving_unicodebuffer,
alloc_unicodebuffer, unicode_from_buffer, keep_unicodebuffer_alive_until_here,
wcharp2unicoden, wcharpsize2unicode,
From arigo at codespeak.net Thu Oct 21 11:45:03 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 21 Oct 2010 11:45:03 +0200 (CEST)
Subject: [pypy-svn] r78158 - in pypy/trunk/pypy/rpython/lltypesystem: . test
Message-ID: <20101021094503.CA5EC282BDC@codespeak.net>
Author: arigo
Date: Thu Oct 21 11:45:02 2010
New Revision: 78158
Modified:
pypy/trunk/pypy/rpython/lltypesystem/ll2ctypes.py
pypy/trunk/pypy/rpython/lltypesystem/test/test_rffi.py
Log:
Test and fix.
Modified: pypy/trunk/pypy/rpython/lltypesystem/ll2ctypes.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/ll2ctypes.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/ll2ctypes.py Thu Oct 21 11:45:02 2010
@@ -823,6 +823,8 @@
except (ValueError, OverflowError):
for tc in 'HIL':
if array(tc).itemsize == array('u').itemsize:
+ import struct
+ cobj &= 256 ** struct.calcsize(tc) - 1
llobj = array('u', array(tc, (cobj,)).tostring())[0]
break
else:
Modified: pypy/trunk/pypy/rpython/lltypesystem/test/test_rffi.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/test/test_rffi.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/test/test_rffi.py Thu Oct 21 11:45:02 2010
@@ -787,3 +787,10 @@
mixann.getgraph(f2, [], s_None)
mixann.finish()
+def test_force_cast_unichar():
+ x = cast(lltype.UniChar, -1)
+ assert isinstance(x, unicode)
+ if sys.maxunicode == 65535:
+ assert cast(LONG, x) == 65535
+ else:
+ assert cast(LONG, cast(INT, x)) == -1
From arigo at codespeak.net Thu Oct 21 12:44:57 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 21 Oct 2010 12:44:57 +0200 (CEST)
Subject: [pypy-svn] r78159 - pypy/trunk/pypy/module/_rawffi/test
Message-ID: <20101021104457.8CC7A282BDC@codespeak.net>
Author: arigo
Date: Thu Oct 21 12:44:56 2010
New Revision: 78159
Modified:
pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py
Log:
- add a missing "const"
- add the symbol "ret_un_func" to the list of exported symbols (needed
on Mac OS/X, for example)
Modified: pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py
==============================================================================
--- pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py (original)
+++ pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py Thu Oct 21 12:44:56 2010
@@ -70,7 +70,7 @@
return s[num];
}
- char *char_check(char x, char y)
+ const char *char_check(char x, char y)
{
if (y == static_str[0])
return static_str;
@@ -185,6 +185,7 @@
sum_x_y
give perturb get_s2a check_s2a
AAA_first_ordinal_function
+ ret_un_func
""".split()
eci = ExternalCompilationInfo(export_symbols=symbols)
return str(platform.compile([c_file], eci, 'x', standalone=False))
From cfbolz at codespeak.net Thu Oct 21 12:57:01 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 21 Oct 2010 12:57:01 +0200 (CEST)
Subject: [pypy-svn] r78160 - pypy/extradoc/talk/pepm2011
Message-ID: <20101021105701.96049282BDC@codespeak.net>
Author: cfbolz
Date: Thu Oct 21 12:57:00 2010
New Revision: 78160
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
an attempt to rewrite section 4
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 21 12:57:00 2010
@@ -29,6 +29,8 @@
fancyvrb=true,
showstringspaces=false,
%keywords={def,while,if,elif,return,class,get,set,new,guard_class}
+ numberstyle = \tiny,
+ numbersep = 0pt,
}
@@ -388,12 +390,7 @@
\begin{figure}
-\lstset{
- numbers = right,
- numberstyle = \tiny,
- numbersep = 0pt
-}
-\begin{lstlisting}[mathescape]
+\begin{lstlisting}[mathescape,numbers = right]
# arguments to the trace: $p_{0}$, $p_{1}$
# inside f: res.add(y)
guard_class($p_{1}$, BoxedInteger)
@@ -472,8 +469,8 @@
\item \lstinline{new} creates a new object.
\item \lstinline{get} reads an attribute of an object.
\item \lstinline{set} writes to an attribute of an object.
- \item \lstinline{guard_class} precedes an (inlined) method call and is
- followed by the trace of the called method.
+ \item \lstinline{guard_class} is a precise type check and precedes an
+ (inlined) method call and is followed by the trace of the called method.
\item \lstinline{int_add} and \lstinline{int_gt} are integer addition and
comparison (``greater than''), respectively.
\end{itemize}
@@ -564,41 +561,63 @@
\label{sec:statics}
The main insight to improve the code shown in the last section is that objects
-in category 1 do not survive very long -- they are used only inside the loop
-and there is no other outside reference to them. Therefore the optimizer
-identifies objects in category 1 and removes the allocation of these objects,
-and all operations manipulating them.
+in category 1 do not survive very long -- they are used only inside the loop and
+there is no other outside reference to them. The optimizer identifies objects in
+category 1 and removes the allocation of these objects, and all operations
+manipulating them.
This is a process that is usually called \emph{escape analysis}
-\cite{goldberg_higher_1990}. In this paper we will
-perform escape analysis by using partial evaluation. The use of partial evaluation is a
-bit peculiar in that it receives no static input arguments for the trace,
-but it is only used to optimize operations within the trace.
-
-To optimize the trace, it is traversed from beginning to end. Every
-operation in the input trace is either removed, or new operations are
-produced. Whenever a \lstinline{new} operation is seen, the operation it is
-removed optimistically and a \emph{static object}\footnote{Here ``static'' is
-meant in the sense of partial evaluation, \ie known at partial evaluation time,
-not in the sense of static allocation or static method.} is constructed and
-associated with the result variable. The static object describes the shape of
-the original object, \ie where the values that would be stored in the fields of
-the allocated object come from, as well as the type of the object.
-
-When a \lstinline{set} that writes into a static object is optimized, the
-corresponding shape description is updated and the operation is removed. This
-means that the operation was done at partial evaluation time. When the
-optimizer encounters a \lstinline{get} from a static object, the result is read
-from the shape description, and the operation is also removed. Equivalently, a
-\lstinline{guard_class} on a variable that has a shape description can be
-removed as well, because the shape description stores the type and thus the
-outcome of the type check the guard does is statically known. Operations that
-have dynamic (\ie non-static) objects as arguments are just left untouched by
-the optimizer.
-
-In the example from Section~\ref{sub:example}, the following operations
-of Figure~\ref{fig:unopt-trace} (lines 10-17) produce two
-static objects, and can be completely removed from the optimized trace:
+\cite{goldberg_higher_1990}. In this paper we will perform escape analysis by
+using partial evaluation. The use of partial evaluation is a bit peculiar in
+that it receives no static input arguments for the trace, but it is only used to
+optimize operations within the trace. This section will give an informal account
+of this process by examining the example trace in Figure~\ref{fig:unopt-trace}.
+The final trace after optimization can be seen in Figure~\ref{fig:step1} (the
+line numbers are the lines of the unoptimized trace where the operation comes
+from).
+
+\begin{figure}
+\begin{lstlisting}[mathescape,numbers=right,escapechar=|,numberstyle = \tiny,numbersep=0pt, numberblanklines=false]
+# arguments to the trace: $p_{0}$, $p_{1}$ |\setcounter{lstnumber}{2}|
+guard_class($p_1$, BoxedInteger) |\setcounter{lstnumber}{4}|
+$i_2$ = get($p_1$, intval)
+guard_class($p_0$, BoxedInteger) |\setcounter{lstnumber}{7}|
+$i_3$ = get($p_0$, intval)
+$i_4$ = int_add($i_2$, $i_3$) |\setcounter{lstnumber}{25}|
+$i_9$ = int_add($i_4$, -100) |\setcounter{lstnumber}{35}|
+
+guard_class($p_0$, BoxedInteger) |\setcounter{lstnumber}{38}|
+$i_{12}$ = get($p_0$, intval) |\setcounter{lstnumber}{42}|
+$i_{14}$ = int_add($i_{12}$, -1) |\setcounter{lstnumber}{50}|
+
+$i_{17}$ = int_gt($i_{14}$, 0) |\setcounter{lstnumber}{53}|
+guard_true($i_{17}$) |\setcounter{lstnumber}{42}|
+
+$p_{15}$ = new(BoxedInteger) |\setcounter{lstnumber}{45}|
+set($p_{15}$, intval, $i_{14}$) |\setcounter{lstnumber}{26}|
+$p_{10}$ = new(BoxedInteger) |\setcounter{lstnumber}{28}|
+set($p_{10}$, intval, $i_9$) |\setcounter{lstnumber}{53}|
+
+jump($p_{15}$, $p_{10}$)
+\end{lstlisting}
+
+\caption{Resulting Trace After Allocation Removal}
+\label{fig:step1}
+\end{figure}
+
+To optimize the trace, it is traversed from beginning to end and an output trace
+is produced at the same time. Every operation in the input trace is either
+removed, or put into the output trace. Sometimes new operations need to be
+produced as well. The optimizer can only remove operations that manipulate
+objects that have been allocated within the trace, all others are copied to the
+output trace untouched.
+
+Looking at the example trace of Figure~\ref{fig:unopt-trace}, this is what
+happens with the operations in lines 1-9. They are manipulating objects that
+existed before the trace because they are passed in as arguments. Therefore the
+optimizer just puts them into the output trace.
+
+The following operations (lines 10-17) are more interesting:
\begin{lstlisting}[mathescape,xleftmargin=20pt]
$p_{5}$ = new(BoxedInteger)
@@ -607,6 +626,22 @@
set($p_{6}$, intval, -100)
\end{lstlisting}
+When the optimizer encounters a \lstinline{new}, it removes it optimistically,
+and assumes that the object is in category 1. When the object escapes later, it
+will be allocated at that point. The optimizer needs to keep track
+of how the object that the operation creates looks like at various points in
+the trace. This is done with the help of a \emph{static object}\footnote{Here
+``static'' is meant in the sense of partial evaluation, \ie known at partial
+evaluation time, not in the sense of ``static allocation'' or ``static
+method''.}. The static object describes the shape of the object that would have
+been allocated, \ie the type of the object and where the values that would be
+stored in the fields of the allocated object come from.
+
+In the snippet above, the two \lstinline{new} operations are removed and two
+static objects are constructed. The \lstinline{set} operations manipulate a
+static object, therefore they can be removed as well and their effect is
+remembered in the static objects.
+
The static object associated with $p_{5}$ would store the knowledge that it is a
\lstinline{BoxedInteger} whose \lstinline{intval} field contains $i_{4}$; the
one associated with $p_{6}$ would store that it is a \lstinline{BoxedInteger}
@@ -625,36 +660,40 @@
$i_{9}$ = int_add($i_{7}$, $i_{8}$)
\end{lstlisting}
-First, the \lstinline{guard_class} operations can be removed, because the classes of $p_{5}$ and
-$p_{6}$ are known to be \lstinline{BoxedInteger}. Second, the \lstinline{get} operations can be removed
-and $i_{7}$ and $i_{8}$ are just replaced by $i_{4}$ and -100. The only
-remaining operation in the optimized trace would be:
+The \lstinline{guard_class} operations can be removed, since their argument is a
+static object with the matching type \lstinline{BoxedInteger}. The
+\lstinline{get} operations can be removed as well, because each of them reads a
+field out of a static object. The results of the get operation are replaced with
+what the static object stores in these fields: $i_{7}$ and $i_{8}$ are just
+replaced by $i_{4}$ and -100. The only operation put into the optimized trace
+is the addition:
\begin{lstlisting}[mathescape,xleftmargin=20pt]
$i_{9}$ = int_add($i_{4}$, -100)
\end{lstlisting}
-The rest of the trace from Figure~\ref{fig:unopt-trace} is optimized similarly.
-
-
-So far we have only described what happens when static objects are used in guards and in
-operations that read and write fields. When the static
-object is used in any other operation, it cannot remain static. For example, when
-a static object is stored in a globally accessible place, the object has to
-be allocated, as it might live longer than one iteration of the loop and as
-arbitrary \lstinline{set} operations could change it due to aliasing. This
-means that the static
-object needs to be turned into a dynamic one, \ie lifted. This makes it
-necessary to put operations into the residual code that allocate the
-static object at runtime.
-
-This is what happens at the end of the trace in Figure~\ref{fig:unopt-trace}, when the \lstinline{jump} operation
-is optimized. The arguments of the jump are at this point static objects. Before the
-jump is emitted, they are \emph{lifted}. This means that the optimizer produces code
-that allocates a new object of the right type and sets its fields to the field
-values that the static object has (if the static object points to other static
-objects, those need to be lifted as well, recursively). This means that instead of a simple jump,
-the following operations are emitted:
+The rest of the trace from Figure~\ref{fig:unopt-trace} is optimized in a
+similar vein. The operations in lines 27-35 produce two more static objects and
+are removed. Those in line 36-39 are just put into the output trace because they
+manipulate objects that are allocated before the trace. Lines 40-42 are removed
+because they operate on a static object. Line 43 is put into the output trace.
+Lines 44-46 produce a new static object and are removed, lines 48-51 manipulate
+that static object and are removed as well. Lines 52-54 are put into the output
+trace.
+
+The last operation (line 55) is an interesting case. It is the \lstinline{jump}
+operation that passes control back to the beginning of the trace. The two
+arguments to this operation are at this point static objects. However, because
+they passed into the next iteration of the loop they live longer than the trace
+and therefore cannot remain static. They need to be turned into a dynamic
+(runtime) object before the actual \lstinline{jump} operation. This process of
+turning a static object into a dynamic one is called \emph{lifting}.
+
+Lifting a static object puts \lstinline{new} and \lstinline{set} operations into
+the output trace. Those operations produce an object at runtime that has the
+same shape that the static object describes. This process is a bit delicate,
+because the static objects could form an arbitrary graph structure. In our
+example is is simple, though:
\begin{lstlisting}[mathescape,xleftmargin=20pt]
$p_{15}$ = new(BoxedInteger)
@@ -664,47 +703,21 @@
jump($p_{15}$, $p_{10}$)
\end{lstlisting}
-Observe how the operations for creating these two instances have been moved to later point in the
-trace.
-At first sight, it may look like for these operations we didn't gain much, as
-the objects are still allocated in the end. However, our optimizations were still
-worthwhile, because some operations that have been performed
-on the lifted static objects have been removed (some \lstinline{get} operations
-and \lstinline{guard_class} operations).
-
-\begin{figure}
-\begin{lstlisting}[mathescape,numbers=right,escapechar=|,numberstyle = \tiny,numbersep=0pt, numberblanklines=false]
-# arguments to the trace: $p_{0}$, $p_{1}$ |\setcounter{lstnumber}{2}|
-guard_class($p_1$, BoxedInteger) |\setcounter{lstnumber}{4}|
-$i_2$ = get($p_1$, intval)
-guard_class($p_0$, BoxedInteger) |\setcounter{lstnumber}{7}|
-$i_3$ = get($p_0$, intval)
-$i_4$ = int_add($i_2$, $i_3$) |\setcounter{lstnumber}{25}|
-$i_9$ = int_add($i_4$, -100) |\setcounter{lstnumber}{35}|
-
-guard_class($p_0$, BoxedInteger) |\setcounter{lstnumber}{38}|
-$i_{12}$ = get($p_0$, intval) |\setcounter{lstnumber}{42}|
-$i_{14}$ = int_add($i_{12}$, -1) |\setcounter{lstnumber}{50}|
-
-$i_{17}$ = int_gt($i_{14}$, 0) |\setcounter{lstnumber}{53}|
-guard_true($i_{17}$) |\setcounter{lstnumber}{42}|
-
-$p_{15}$ = new(BoxedInteger) |\setcounter{lstnumber}{45}|
-set($p_{15}$, intval, $i_{14}$) |\setcounter{lstnumber}{26}|
-$p_{10}$ = new(BoxedInteger) |\setcounter{lstnumber}{28}|
-set($p_{10}$, intval, $i_9$) |\setcounter{lstnumber}{53}|
-jump($p_{15}$, $p_{10}$)
-\end{lstlisting}
-\caption{Resulting Trace After Allocation Removal}
-\label{fig:step1}
-\end{figure}
+Observe how the operations for creating these two instances have been moved to a
+later point in the trace. This is worthwhile even though the objects have to be
+allocated in the end because some \lstinline{get} operations and
+\lstinline{guard_class} operations on the lifted static objects could be
+removed.
+
+A bit more generally, lifting needs to occur if a static object is used in any
+operation apart from \lstinline{get}, \lstinline{set}, and \lstinline{guard}.
+It also needs to occur if \lstinline{set} is used to store a static object into
+a non-static one.
The final optimized trace of the example can be seen in Figure~\ref{fig:step1}.
The optimized trace contains only two allocations, instead of the original five,
-and only three \lstinline{guard_class} operations, from the original seven. The
-line numbers are the lines where the operations occurred in the original trace
-in Figure~\ref{fig:unopt-trace}.
+and only three \lstinline{guard_class} operations, from the original seven.
\section{Formal Description of the Algorithm}
\label{sec:formal}
@@ -849,9 +862,12 @@
To optimize the simple traces of the last section, we use online partial
evaluation. The partial evaluator optimizes one operation of a trace at a
time. Every operation in the unoptimized trace is replaced by a list of
-operations in the optimized trace. This list is empty if the operation
-can be optimized away. The optimization rules can be seen in
-Figure~\ref{fig:optimization}.
+operations in the optimized trace. This list is empty if the operation can be
+optimized away. The optimization rules can be seen in
+Figure~\ref{fig:optimization}. Lists are written using angular brackets $<...>$,
+list concatenation is expressed using two colons: $l_1::l_2$.
+
+XXX input/output of optimizer
The state of the optimizer is stored in an environment $E$ and a \emph{static
heap} $S$. The environment is a partial function from variables in the
@@ -955,7 +971,8 @@
not escape) are completely removed; moreover, objects in category 2
(\ie escaping) are still partially optimized: all the operations in between the
creation of the object and the point where it escapes that involve the object
-are removed.
+are removed. Objects in category 3 and 4 are also partially optimized, their
+allocation is delayed till the end of the trace.
The optimization is particularly effective for chains of operations.
For example, it is typical for an interpreter to generate sequences of
@@ -1097,7 +1114,7 @@
result. The errors were computed using a confidence interval with a 95\%
confidence level \cite{georges_statistically_2007}. The results are reported in
Figure~\ref{fig:times}. In addition to the run times the table also reports the
-speedup that PyPy achieves when the optimization is turned on.
+speedup that PyPy achieves when the optimization is turned on. XXX sounds sucky
With the optimization turned on, PyPy's Python interpreter outperforms CPython
in all benchmarks except spambayes (which heavily relies on regular expression
From arigo at codespeak.net Thu Oct 21 13:23:06 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 21 Oct 2010 13:23:06 +0200 (CEST)
Subject: [pypy-svn] r78161 - in pypy/trunk/pypy/rlib/rsre: . test
Message-ID: <20101021112306.5557D282BDC@codespeak.net>
Author: arigo
Date: Thu Oct 21 13:23:04 2010
New Revision: 78161
Modified:
pypy/trunk/pypy/rlib/rsre/rsre_core.py
pypy/trunk/pypy/rlib/rsre/test/test_match.py
Log:
Fix the tests, at least on Mac.
Modified: pypy/trunk/pypy/rlib/rsre/rsre_core.py
==============================================================================
--- pypy/trunk/pypy/rlib/rsre/rsre_core.py (original)
+++ pypy/trunk/pypy/rlib/rsre/rsre_core.py Thu Oct 21 13:23:04 2010
@@ -170,6 +170,8 @@
def __init__(self, pattern, string, match_start, end, flags):
AbstractMatchContext.__init__(self, pattern, match_start, end, flags)
self._string = string
+ if not we_are_translated() and isinstance(string, unicode):
+ self.flags |= rsre_char.SRE_FLAG_UNICODE # for rsre_re.py
def str(self, index):
check_nonneg(index)
Modified: pypy/trunk/pypy/rlib/rsre/test/test_match.py
==============================================================================
--- pypy/trunk/pypy/rlib/rsre/test/test_match.py (original)
+++ pypy/trunk/pypy/rlib/rsre/test/test_match.py Thu Oct 21 13:23:04 2010
@@ -1,24 +1,49 @@
-import _sre, re, sre_compile
-from pypy.rlib.rsre import rsre_core
+import re
+from pypy.rlib.rsre import rsre_core, rsre_char
-def get_code(regexp, flags=0, allargs=False):
- class GotIt(Exception):
- pass
- def my_compile(pattern, flags, code, *args):
- print code
- raise GotIt(code, flags, args)
- saved = _sre.compile
- try:
- _sre.compile = my_compile
- try:
- sre_compile.compile(regexp, flags)
- except GotIt, e:
- pass
+def get_hacked_sre_compile(my_compile):
+ """Return a copy of the sre_compile module for which the _sre
+ module is a custom module that has _sre.compile == my_compile
+ and CODESIZE == rsre_char.CODESIZE.
+ """
+ import sre_compile, __builtin__, new
+ sre_hacked = new.module("_sre_hacked")
+ sre_hacked.compile = my_compile
+ sre_hacked.MAGIC = sre_compile.MAGIC
+ sre_hacked.CODESIZE = rsre_char.CODESIZE
+ sre_hacked.getlower = rsre_char.getlower
+ def my_import(name, *args):
+ if name == '_sre':
+ return sre_hacked
else:
- raise ValueError("did not reach _sre.compile()!")
+ return default_import(name, *args)
+ src = sre_compile.__file__
+ if src.lower().endswith('.pyc') or src.lower().endswith('.pyo'):
+ src = src[:-1]
+ mod = new.module("sre_compile_hacked")
+ default_import = __import__
+ try:
+ __builtin__.__import__ = my_import
+ execfile(src, mod.__dict__)
finally:
- _sre.compile = saved
+ __builtin__.__import__ = default_import
+ return mod
+
+class GotIt(Exception):
+ pass
+def my_compile(pattern, flags, code, *args):
+ print code
+ raise GotIt(code, flags, args)
+sre_compile_hacked = get_hacked_sre_compile(my_compile)
+
+def get_code(regexp, flags=0, allargs=False):
+ try:
+ sre_compile_hacked.compile(regexp, flags)
+ except GotIt, e:
+ pass
+ else:
+ raise ValueError("did not reach _sre.compile()!")
if allargs:
return e.args
else:
From antocuni at codespeak.net Thu Oct 21 13:23:20 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Thu, 21 Oct 2010 13:23:20 +0200 (CEST)
Subject: [pypy-svn] r78162 - pypy/extradoc/talk/pepm2011
Message-ID: <20101021112320.7E45A282BDC@codespeak.net>
Author: antocuni
Date: Thu Oct 21 13:23:18 2010
New Revision: 78162
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
review the new section 4
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 21 13:23:18 2010
@@ -560,7 +560,7 @@
\section{Allocation Removal in Traces}
\label{sec:statics}
-The main insight to improve the code shown in the last section is that objects
+The main insight to improve the code shown in section \ref{sub:example} is that objects
in category 1 do not survive very long -- they are used only inside the loop and
there is no other outside reference to them. The optimizer identifies objects in
category 1 and removes the allocation of these objects, and all operations
@@ -609,15 +609,16 @@
is produced at the same time. Every operation in the input trace is either
removed, or put into the output trace. Sometimes new operations need to be
produced as well. The optimizer can only remove operations that manipulate
-objects that have been allocated within the trace, all others are copied to the
+objects that have been allocated within the trace, while all other operations are copied to the
output trace untouched.
-Looking at the example trace of Figure~\ref{fig:unopt-trace}, this is what
-happens with the operations in lines 1-9. They are manipulating objects that
-existed before the trace because they are passed in as arguments. Therefore the
-optimizer just puts them into the output trace.
+Looking at the example trace of Figure~\ref{fig:unopt-trace}, the operations
+in lines 1-9 are manipulating objects which existed before the trace and that
+are passed in as arguments: therefore the optimizer just puts them into the
+output trace.
The following operations (lines 10-17) are more interesting:
+\anto{what about putting the line numbers also here? Moreover, figure 4 is put just after this listing, it could be confused}
\begin{lstlisting}[mathescape,xleftmargin=20pt]
$p_{5}$ = new(BoxedInteger)
@@ -626,8 +627,9 @@
set($p_{6}$, intval, -100)
\end{lstlisting}
+\anto{I don't like ``how the object looks like''. I don't know what else to say, though :-(}
When the optimizer encounters a \lstinline{new}, it removes it optimistically,
-and assumes that the object is in category 1. When the object escapes later, it
+and assumes that the object is in category 1. If later the optimizer finds that the object escapes, it
will be allocated at that point. The optimizer needs to keep track
of how the object that the operation creates looks like at various points in
the trace. This is done with the help of a \emph{static object}\footnote{Here
@@ -664,7 +666,7 @@
static object with the matching type \lstinline{BoxedInteger}. The
\lstinline{get} operations can be removed as well, because each of them reads a
field out of a static object. The results of the get operation are replaced with
-what the static object stores in these fields: $i_{7}$ and $i_{8}$ are just
+what the static object stores in these fields: all the occurences of $i_{7}$ and $i_{8}$ in the trace are just
replaced by $i_{4}$ and -100. The only operation put into the optimized trace
is the addition:
From arigo at codespeak.net Thu Oct 21 13:24:11 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 21 Oct 2010 13:24:11 +0200 (CEST)
Subject: [pypy-svn] r78163 - pypy/trunk/pypy/jit/backend/test
Message-ID: <20101021112411.4D166282BDC@codespeak.net>
Author: arigo
Date: Thu Oct 21 13:24:09 2010
New Revision: 78163
Modified:
pypy/trunk/pypy/jit/backend/test/runner_test.py
Log:
Fix test.
Modified: pypy/trunk/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/trunk/pypy/jit/backend/test/runner_test.py Thu Oct 21 13:24:09 2010
@@ -519,7 +519,8 @@
def test_call_to_c_function(self):
from pypy.rlib.libffi import CDLL, types, ArgChain
- libc = CDLL('libc.so.6')
+ from pypy.rpython.lltypesystem.ll2ctypes import libc_name
+ libc = CDLL(libc_name)
c_tolower = libc.getpointer('tolower', [types.uchar], types.sint)
argchain = ArgChain().arg(ord('A'))
assert c_tolower.call(argchain, rffi.INT) == ord('a')
From arigo at codespeak.net Thu Oct 21 13:29:49 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 21 Oct 2010 13:29:49 +0200 (CEST)
Subject: [pypy-svn] r78164 - pypy/trunk/pypy/translator/platform
Message-ID: <20101021112949.59E27282BDC@codespeak.net>
Author: arigo
Date: Thu Oct 21 13:29:47 2010
New Revision: 78164
Modified:
pypy/trunk/pypy/translator/platform/darwin.py
Log:
Use $CC on mac too. Fixes test_gcc_options.
Modified: pypy/trunk/pypy/translator/platform/darwin.py
==============================================================================
--- pypy/trunk/pypy/translator/platform/darwin.py (original)
+++ pypy/trunk/pypy/translator/platform/darwin.py Thu Oct 21 13:29:47 2010
@@ -14,7 +14,10 @@
def __init__(self, cc=None):
if cc is None:
- cc = 'gcc'
+ try:
+ cc = os.environ['CC']
+ except KeyError:
+ cc = 'gcc'
self.cc = cc
def _args_for_shared(self, args):
From antocuni at codespeak.net Thu Oct 21 13:46:17 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Thu, 21 Oct 2010 13:46:17 +0200 (CEST)
Subject: [pypy-svn] r78165 - pypy/trunk/pypy/rlib/test
Message-ID: <20101021114617.671A7282BDC@codespeak.net>
Author: antocuni
Date: Thu Oct 21 13:46:15 2010
New Revision: 78165
Modified:
pypy/trunk/pypy/rlib/test/test_libffi.py
Log:
checkin this test which passes on 32bit, but I expect it to fail on 64
Modified: pypy/trunk/pypy/rlib/test/test_libffi.py
==============================================================================
--- pypy/trunk/pypy/rlib/test/test_libffi.py (original)
+++ pypy/trunk/pypy/rlib/test/test_libffi.py Thu Oct 21 13:46:15 2010
@@ -173,6 +173,19 @@
res = self.call(func, [chr(20), 22], rffi.LONG)
assert res == 42
+ def test_unsigned_short_args(self):
+ """
+ unsigned short sum_xy_us(unsigned short x, unsigned short y)
+ {
+ return x+y;
+ }
+ """
+ libfoo = self.get_libfoo()
+ func = (libfoo, 'sum_xy_us', [types.ushort, types.ushort], types.ushort)
+ res = self.call(func, [32000, 8000], rffi.USHORT)
+ assert res == 40000
+
+
def test_pointer_as_argument(self):
"""#include
long inc(long* x)
From cfbolz at codespeak.net Thu Oct 21 13:46:48 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 21 Oct 2010 13:46:48 +0200 (CEST)
Subject: [pypy-svn] r78166 - pypy/extradoc/talk/pepm2011
Message-ID: <20101021114648.EE1C0282BDC@codespeak.net>
Author: cfbolz
Date: Thu Oct 21 13:46:47 2010
New Revision: 78166
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
fix some of anto's issues, will try to add line numbers now
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 21 13:46:47 2010
@@ -560,7 +560,7 @@
\section{Allocation Removal in Traces}
\label{sec:statics}
-The main insight to improve the code shown in section \ref{sub:example} is that objects
+The main insight to improve the code shown in Section~\ref{sub:example} is that objects
in category 1 do not survive very long -- they are used only inside the loop and
there is no other outside reference to them. The optimizer identifies objects in
category 1 and removes the allocation of these objects, and all operations
@@ -576,35 +576,6 @@
line numbers are the lines of the unoptimized trace where the operation comes
from).
-\begin{figure}
-\begin{lstlisting}[mathescape,numbers=right,escapechar=|,numberstyle = \tiny,numbersep=0pt, numberblanklines=false]
-# arguments to the trace: $p_{0}$, $p_{1}$ |\setcounter{lstnumber}{2}|
-guard_class($p_1$, BoxedInteger) |\setcounter{lstnumber}{4}|
-$i_2$ = get($p_1$, intval)
-guard_class($p_0$, BoxedInteger) |\setcounter{lstnumber}{7}|
-$i_3$ = get($p_0$, intval)
-$i_4$ = int_add($i_2$, $i_3$) |\setcounter{lstnumber}{25}|
-$i_9$ = int_add($i_4$, -100) |\setcounter{lstnumber}{35}|
-
-guard_class($p_0$, BoxedInteger) |\setcounter{lstnumber}{38}|
-$i_{12}$ = get($p_0$, intval) |\setcounter{lstnumber}{42}|
-$i_{14}$ = int_add($i_{12}$, -1) |\setcounter{lstnumber}{50}|
-
-$i_{17}$ = int_gt($i_{14}$, 0) |\setcounter{lstnumber}{53}|
-guard_true($i_{17}$) |\setcounter{lstnumber}{42}|
-
-$p_{15}$ = new(BoxedInteger) |\setcounter{lstnumber}{45}|
-set($p_{15}$, intval, $i_{14}$) |\setcounter{lstnumber}{26}|
-$p_{10}$ = new(BoxedInteger) |\setcounter{lstnumber}{28}|
-set($p_{10}$, intval, $i_9$) |\setcounter{lstnumber}{53}|
-
-jump($p_{15}$, $p_{10}$)
-\end{lstlisting}
-
-\caption{Resulting Trace After Allocation Removal}
-\label{fig:step1}
-\end{figure}
-
To optimize the trace, it is traversed from beginning to end and an output trace
is produced at the same time. Every operation in the input trace is either
removed, or put into the output trace. Sometimes new operations need to be
@@ -618,7 +589,7 @@
output trace.
The following operations (lines 10-17) are more interesting:
-\anto{what about putting the line numbers also here? Moreover, figure 4 is put just after this listing, it could be confused}
+% XXX line numbers
\begin{lstlisting}[mathescape,xleftmargin=20pt]
$p_{5}$ = new(BoxedInteger)
@@ -627,12 +598,11 @@
set($p_{6}$, intval, -100)
\end{lstlisting}
-\anto{I don't like ``how the object looks like''. I don't know what else to say, though :-(}
When the optimizer encounters a \lstinline{new}, it removes it optimistically,
and assumes that the object is in category 1. If later the optimizer finds that the object escapes, it
will be allocated at that point. The optimizer needs to keep track
-of how the object that the operation creates looks like at various points in
-the trace. This is done with the help of a \emph{static object}\footnote{Here
+of the state of the object that the operation would have created.
+This is done with the help of a \emph{static object}\footnote{Here
``static'' is meant in the sense of partial evaluation, \ie known at partial
evaluation time, not in the sense of ``static allocation'' or ``static
method''.}. The static object describes the shape of the object that would have
@@ -721,6 +691,36 @@
The optimized trace contains only two allocations, instead of the original five,
and only three \lstinline{guard_class} operations, from the original seven.
+\begin{figure}
+\begin{lstlisting}[mathescape,numbers=right,escapechar=|,numberstyle = \tiny,numbersep=0pt, numberblanklines=false]
+# arguments to the trace: $p_{0}$, $p_{1}$ |\setcounter{lstnumber}{2}|
+guard_class($p_1$, BoxedInteger) |\setcounter{lstnumber}{4}|
+$i_2$ = get($p_1$, intval)
+guard_class($p_0$, BoxedInteger) |\setcounter{lstnumber}{7}|
+$i_3$ = get($p_0$, intval)
+$i_4$ = int_add($i_2$, $i_3$) |\setcounter{lstnumber}{25}|
+$i_9$ = int_add($i_4$, -100) |\setcounter{lstnumber}{35}|
+
+guard_class($p_0$, BoxedInteger) |\setcounter{lstnumber}{38}|
+$i_{12}$ = get($p_0$, intval) |\setcounter{lstnumber}{42}|
+$i_{14}$ = int_add($i_{12}$, -1) |\setcounter{lstnumber}{50}|
+
+$i_{17}$ = int_gt($i_{14}$, 0) |\setcounter{lstnumber}{53}|
+guard_true($i_{17}$) |\setcounter{lstnumber}{42}|
+
+$p_{15}$ = new(BoxedInteger) |\setcounter{lstnumber}{45}|
+set($p_{15}$, intval, $i_{14}$) |\setcounter{lstnumber}{26}|
+$p_{10}$ = new(BoxedInteger) |\setcounter{lstnumber}{28}|
+set($p_{10}$, intval, $i_9$) |\setcounter{lstnumber}{53}|
+
+jump($p_{15}$, $p_{10}$)
+\end{lstlisting}
+
+\caption{Resulting Trace After Allocation Removal}
+\label{fig:step1}
+\end{figure}
+
+
\section{Formal Description of the Algorithm}
\label{sec:formal}
From arigo at codespeak.net Thu Oct 21 13:49:18 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 21 Oct 2010 13:49:18 +0200 (CEST)
Subject: [pypy-svn] r78167 - pypy/trunk/pypy/module/array
Message-ID: <20101021114918.3B8CF282BDC@codespeak.net>
Author: arigo
Date: Thu Oct 21 13:49:16 2010
New Revision: 78167
Modified:
pypy/trunk/pypy/module/array/interp_array.py
Log:
Reorder a bit the code in item_w(). It should help the JIT
generate less operations, because it doesn't need to compare
the item with the lower and upper bounds explicitly. It only
needs to do the casting (which requires masking) and then
comparing the result with the original value.
Modified: pypy/trunk/pypy/module/array/interp_array.py
==============================================================================
--- pypy/trunk/pypy/module/array/interp_array.py (original)
+++ pypy/trunk/pypy/module/array/interp_array.py Thu Oct 21 13:49:16 2010
@@ -192,32 +192,30 @@
mytype.bytes
raise OperationError(space.w_OverflowError,
space.wrap(msg))
- elif mytype.unwrap == 'str_w' or mytype.unwrap == 'unicode_w':
+ return rffi.cast(mytype.itemtype, item)
+ if mytype.unwrap == 'str_w' or mytype.unwrap == 'unicode_w':
if len(item) != 1:
msg = 'array item must be char'
raise OperationError(space.w_TypeError, space.wrap(msg))
item = item[0]
-
+ return rffi.cast(mytype.itemtype, item)
+ #
+ # "regular" case: it fits in an rpython integer (lltype.Signed)
+ result = rffi.cast(mytype.itemtype, item)
if mytype.canoverflow:
- msg = None
- if mytype.signed:
- if item < -1 << (mytype.bytes * 8 - 1):
+ if rffi.cast(lltype.Signed, result) != item:
+ # overflow. build the correct message
+ if item < 0:
msg = ('signed %d-byte integer is less than minimum' %
mytype.bytes)
- elif item > (1 << (mytype.bytes * 8 - 1)) - 1:
+ else:
msg = ('signed %d-byte integer is greater than maximum'
% mytype.bytes)
- else:
- if item < 0:
- msg = ('unsigned %d-byte integer is less than minimum'
- % mytype.bytes)
- elif item > (1 << (mytype.bytes * 8)) - 1:
- msg = ('unsigned %d-byte integer is greater'
- ' than maximum' % mytype.bytes)
- if msg is not None:
+ if not mytype.signed:
+ msg = 'un' + msg # 'signed' => 'unsigned'
raise OperationError(space.w_OverflowError,
space.wrap(msg))
- return rffi.cast(mytype.itemtype, item)
+ return result
def __del__(self):
self.setlen(0)
From arigo at codespeak.net Thu Oct 21 13:54:09 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 21 Oct 2010 13:54:09 +0200 (CEST)
Subject: [pypy-svn] r78168 - pypy/trunk/pypy/module/pypyjit/test
Message-ID: <20101021115409.3E372282BDC@codespeak.net>
Author: arigo
Date: Thu Oct 21 13:54:07 2010
New Revision: 78168
Modified:
pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
Log:
Fix the test on 64 bits.
Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Thu Oct 21 13:54:07 2010
@@ -608,16 +608,17 @@
# call that can raise is not exchanged into getarrayitem_gc
def test_overflow_checking(self):
+ startvalue = sys.maxint - 2147483647
self.run_source('''
def main():
def f(a,b):
if a < 0: return -1
return a-b
- total = 0
+ total = %d
for i in range(100000):
total += f(i, 5)
return total
- ''', 170, ([], 4999450000L))
+ ''' % startvalue, 170, ([], startvalue + 4999450000L))
def test_boolrewrite_invers(self):
for a, b, res, ops in (('2000', '2000', 20001000, 51),
From cfbolz at codespeak.net Thu Oct 21 14:02:27 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 21 Oct 2010 14:02:27 +0200 (CEST)
Subject: [pypy-svn] r78169 - pypy/extradoc/talk/pepm2011
Message-ID: <20101021120227.14F7B282BDC@codespeak.net>
Author: cfbolz
Date: Thu Oct 21 14:02:26 2010
New Revision: 78169
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
add line numbers
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 21 14:02:26 2010
@@ -30,7 +30,7 @@
showstringspaces=false,
%keywords={def,while,if,elif,return,class,get,set,new,guard_class}
numberstyle = \tiny,
- numbersep = 0pt,
+ numbersep = -20pt,
}
@@ -591,10 +591,10 @@
The following operations (lines 10-17) are more interesting:
% XXX line numbers
-\begin{lstlisting}[mathescape,xleftmargin=20pt]
-$p_{5}$ = new(BoxedInteger)
-set($p_{5}$, intval, $i_{4}$)
-$p_{6}$ = new(BoxedInteger)
+\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=10]
+$p_{5}$ = new(BoxedInteger) |\setcounter{lstnumber}{11}|
+set($p_{5}$, intval, $i_{4}$) |\setcounter{lstnumber}{14}|
+$p_{6}$ = new(BoxedInteger) |\setcounter{lstnumber}{16}|
set($p_{6}$, intval, -100)
\end{lstlisting}
@@ -622,12 +622,10 @@
The subsequent operations (line 19-29) in Figure~\ref{fig:unopt-trace}, which
use $p_{5}$ and $p_{6}$, can then be optimized using that knowledge:
-\begin{lstlisting}[mathescape,xleftmargin=20pt]
-guard_class($p_{5}$, BoxedInteger)
-# inside BoxedInteger.add
+\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=20]
+guard_class($p_{5}$, BoxedInteger) |\setcounter{lstnumber}{21}|
$i_{7}$ = get($p_{5}$, intval)
-guard_class($p_{6}$, BoxedInteger)
-# inside BoxedInteger.add__int
+guard_class($p_{6}$, BoxedInteger) |\setcounter{lstnumber}{24}|
$i_{8}$ = get($p_{6}$, intval)
$i_{9}$ = int_add($i_{7}$, $i_{8}$)
\end{lstlisting}
@@ -640,7 +638,7 @@
replaced by $i_{4}$ and -100. The only operation put into the optimized trace
is the addition:
-\begin{lstlisting}[mathescape,xleftmargin=20pt]
+\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=26]
$i_{9}$ = int_add($i_{4}$, -100)
\end{lstlisting}
@@ -667,11 +665,11 @@
because the static objects could form an arbitrary graph structure. In our
example is is simple, though:
-\begin{lstlisting}[mathescape,xleftmargin=20pt]
-$p_{15}$ = new(BoxedInteger)
-set($p_{15}$, intval, $i_{14}$)
-$p_{10}$ = new(BoxedInteger)
-set($p_{10}$, intval, $i_{9}$)
+\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=44]
+$p_{15}$ = new(BoxedInteger) |\setcounter{lstnumber}{45}|
+set($p_{15}$, intval, $i_{14}$) |\setcounter{lstnumber}{26}|
+$p_{10}$ = new(BoxedInteger) |\setcounter{lstnumber}{28}|
+set($p_{10}$, intval, $i_9$) |\setcounter{lstnumber}{54}|
jump($p_{15}$, $p_{10}$)
\end{lstlisting}
@@ -692,7 +690,7 @@
and only three \lstinline{guard_class} operations, from the original seven.
\begin{figure}
-\begin{lstlisting}[mathescape,numbers=right,escapechar=|,numberstyle = \tiny,numbersep=0pt, numberblanklines=false]
+\begin{lstlisting}[mathescape,numbers=right,escapechar=|, numberblanklines=false]
# arguments to the trace: $p_{0}$, $p_{1}$ |\setcounter{lstnumber}{2}|
guard_class($p_1$, BoxedInteger) |\setcounter{lstnumber}{4}|
$i_2$ = get($p_1$, intval)
From arigo at codespeak.net Thu Oct 21 14:14:42 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 21 Oct 2010 14:14:42 +0200 (CEST)
Subject: [pypy-svn] r78170 - pypy/extradoc/talk/pepm2011
Message-ID: <20101021121442.AAE3A282BDC@codespeak.net>
Author: arigo
Date: Thu Oct 21 14:14:41 2010
New Revision: 78170
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
A typo, and a semicolon (sorry, couldn't resist).
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 21 14:14:41 2010
@@ -611,7 +611,7 @@
In the snippet above, the two \lstinline{new} operations are removed and two
static objects are constructed. The \lstinline{set} operations manipulate a
-static object, therefore they can be removed as well and their effect is
+static object, therefore they can be removed as well; their effect is
remembered in the static objects.
The static object associated with $p_{5}$ would store the knowledge that it is a
@@ -663,7 +663,7 @@
the output trace. Those operations produce an object at runtime that has the
same shape that the static object describes. This process is a bit delicate,
because the static objects could form an arbitrary graph structure. In our
-example is is simple, though:
+example it is simple, though:
\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=44]
$p_{15}$ = new(BoxedInteger) |\setcounter{lstnumber}{45}|
From cfbolz at codespeak.net Thu Oct 21 14:23:50 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 21 Oct 2010 14:23:50 +0200 (CEST)
Subject: [pypy-svn] r78171 - pypy/extradoc/talk/pepm2011
Message-ID: <20101021122350.AD95B282BDC@codespeak.net>
Author: cfbolz
Date: Thu Oct 21 14:23:49 2010
New Revision: 78171
Modified:
pypy/extradoc/talk/pepm2011/paper.tex
Log:
did those
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 21 14:23:49 2010
@@ -589,7 +589,6 @@
output trace.
The following operations (lines 10-17) are more interesting:
-% XXX line numbers
\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=10]
$p_{5}$ = new(BoxedInteger) |\setcounter{lstnumber}{11}|
From antocuni at codespeak.net Thu Oct 21 14:26:50 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Thu, 21 Oct 2010 14:26:50 +0200 (CEST)
Subject: [pypy-svn] r78172 - in pypy/trunk/pypy/rlib: . test
Message-ID: <20101021122650.2BC2E282BDC@codespeak.net>
Author: antocuni
Date: Thu Oct 21 14:26:48 2010
New Revision: 78172
Modified:
pypy/trunk/pypy/rlib/libffi.py
pypy/trunk/pypy/rlib/test/test_libffi.py
Log:
make rlib.libffi more robust when calling a function that it's declared to return a smaller unsigned, but then called expecting a ULONG. This fixes test_unsigned_short_args in module/_ffi
Modified: pypy/trunk/pypy/rlib/libffi.py
==============================================================================
--- pypy/trunk/pypy/rlib/libffi.py (original)
+++ pypy/trunk/pypy/rlib/libffi.py Thu Oct 21 14:26:48 2010
@@ -188,6 +188,9 @@
#
if _fits_into_long(RESULT):
res = self._do_call_int(self.funcsym, ll_args)
+ if self.restype.c_size < types.slong.c_size:
+ # mask res to keep only the bits we are interested in
+ res &= ~(-1 << self.restype.c_size*8)
elif RESULT is rffi.DOUBLE:
return self._do_call_float(self.funcsym, ll_args)
elif RESULT is lltype.Void:
Modified: pypy/trunk/pypy/rlib/test/test_libffi.py
==============================================================================
--- pypy/trunk/pypy/rlib/test/test_libffi.py (original)
+++ pypy/trunk/pypy/rlib/test/test_libffi.py Thu Oct 21 14:26:48 2010
@@ -182,7 +182,9 @@
"""
libfoo = self.get_libfoo()
func = (libfoo, 'sum_xy_us', [types.ushort, types.ushort], types.ushort)
- res = self.call(func, [32000, 8000], rffi.USHORT)
+ # the ULONG in the next line is not a typo: we really want to test
+ # that we get the correct value even if we cast it to a larger type
+ res = self.call(func, [32000, 8000], rffi.ULONG)
assert res == 40000
From cfbolz at codespeak.net Thu Oct 21 14:50:45 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Thu, 21 Oct 2010 14:50:45 +0200 (CEST)
Subject: [pypy-svn] r78173 - pypy/extradoc/talk/pepm2011
Message-ID: <20101021125045.BC7A2282BDC@codespeak.net>
Author: cfbolz
Date: Thu Oct 21 14:50:44 2010
New Revision: 78173
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
some more tweaks by stephan
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Thu Oct 21 14:50:44 2010
@@ -573,22 +573,21 @@
optimize operations within the trace. This section will give an informal account
of this process by examining the example trace in Figure~\ref{fig:unopt-trace}.
The final trace after optimization can be seen in Figure~\ref{fig:step1} (the
-line numbers are the lines of the unoptimized trace where the operation comes
-from).
+line numbers are the lines of the unoptimized trace where the operation originates).
-To optimize the trace, it is traversed from beginning to end and an output trace
-is produced at the same time. Every operation in the input trace is either
-removed, or put into the output trace. Sometimes new operations need to be
+To optimize the trace, it is traversed from beginning to end while an output
+trace is produced. Every operation in the input trace is either
+removed or put into the output trace. Sometimes new operations need to be
produced as well. The optimizer can only remove operations that manipulate
objects that have been allocated within the trace, while all other operations are copied to the
-output trace untouched.
+output trace unchanged.
Looking at the example trace of Figure~\ref{fig:unopt-trace}, the operations
-in lines 1-9 are manipulating objects which existed before the trace and that
+in lines 1--9 are manipulating objects which existed before the trace and that
are passed in as arguments: therefore the optimizer just puts them into the
output trace.
-The following operations (lines 10-17) are more interesting:
+The following operations (lines 10--17) are more interesting:
\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=10]
$p_{5}$ = new(BoxedInteger) |\setcounter{lstnumber}{11}|
@@ -618,7 +617,7 @@
one associated with $p_{6}$ would store that it is a \lstinline{BoxedInteger}
whose \lstinline{intval} field contains the constant -100.
-The subsequent operations (line 19-29) in Figure~\ref{fig:unopt-trace}, which
+The subsequent operations (line 20--26) in Figure~\ref{fig:unopt-trace}, which
use $p_{5}$ and $p_{6}$, can then be optimized using that knowledge:
\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=20]
@@ -634,7 +633,7 @@
\lstinline{get} operations can be removed as well, because each of them reads a
field out of a static object. The results of the get operation are replaced with
what the static object stores in these fields: all the occurences of $i_{7}$ and $i_{8}$ in the trace are just
-replaced by $i_{4}$ and -100. The only operation put into the optimized trace
+replaced by $i_{4}$ and -100. The only operation copied into the optimized trace
is the addition:
\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=26]
@@ -642,25 +641,25 @@
\end{lstlisting}
The rest of the trace from Figure~\ref{fig:unopt-trace} is optimized in a
-similar vein. The operations in lines 27-35 produce two more static objects and
-are removed. Those in line 36-39 are just put into the output trace because they
-manipulate objects that are allocated before the trace. Lines 40-42 are removed
+similar vein. The operations in lines 27--35 produce two more static objects and
+are removed. Those in line 36--39 are just put into the output trace because they
+manipulate objects that are allocated before the trace. Lines 40--42 are removed
because they operate on a static object. Line 43 is put into the output trace.
-Lines 44-46 produce a new static object and are removed, lines 48-51 manipulate
-that static object and are removed as well. Lines 52-54 are put into the output
+Lines 44--46 produce a new static object and are removed, lines 48--51 manipulate
+that static object and are removed as well. Lines 52--54 are put into the output
trace.
The last operation (line 55) is an interesting case. It is the \lstinline{jump}
operation that passes control back to the beginning of the trace. The two
-arguments to this operation are at this point static objects. However, because
-they passed into the next iteration of the loop they live longer than the trace
-and therefore cannot remain static. They need to be turned into a dynamic
-(runtime) object before the actual \lstinline{jump} operation. This process of
+arguments to this operation at this point are static objects. However, because
+they are passed into the next iteration of the loop they live longer than the trace
+and therefore cannot remain static. They need to be turned into dynamic
+(runtime) objects before the actual \lstinline{jump} operation. This process of
turning a static object into a dynamic one is called \emph{lifting}.
Lifting a static object puts \lstinline{new} and \lstinline{set} operations into
the output trace. Those operations produce an object at runtime that has the
-same shape that the static object describes. This process is a bit delicate,
+shape described by the static object. This process is a bit delicate,
because the static objects could form an arbitrary graph structure. In our
example it is simple, though:
@@ -679,14 +678,15 @@
\lstinline{guard_class} operations on the lifted static objects could be
removed.
-A bit more generally, lifting needs to occur if a static object is used in any
+More generally, lifting needs to occur if a static object is used in any
operation apart from \lstinline{get}, \lstinline{set}, and \lstinline{guard}.
It also needs to occur if \lstinline{set} is used to store a static object into
a non-static one.
The final optimized trace of the example can be seen in Figure~\ref{fig:step1}.
-The optimized trace contains only two allocations, instead of the original five,
-and only three \lstinline{guard_class} operations, from the original seven.
+The optimized trace contains only two allocations, instead of the original
+five, and only three \lstinline{guard_class} operations, compared to the
+original seven.
\begin{figure}
\begin{lstlisting}[mathescape,numbers=right,escapechar=|, numberblanklines=false]
@@ -866,10 +866,12 @@
Figure~\ref{fig:optimization}. Lists are written using angular brackets $<...>$,
list concatenation is expressed using two colons: $l_1::l_2$.
-XXX input/output of optimizer
-
The state of the optimizer is stored in an environment $E$ and a \emph{static
-heap} $S$. The environment is a partial function from variables in the
+heap} $S$. Each step of the optimizer takes an operation, an environment and a
+static heap and produces a list of operations, a new environment and a new
+static heap.
+
+The environment is a partial function from variables in the
unoptimized trace $V$ to variables in the optimized trace $V^*$ (which are
themselves written with a
$\ ^*$ for clarity). The reason for introducing new variables in the optimized
@@ -1112,8 +1114,8 @@
code. The arithmetic mean of the times of the last 30 runs were used as the
result. The errors were computed using a confidence interval with a 95\%
confidence level \cite{georges_statistically_2007}. The results are reported in
-Figure~\ref{fig:times}. In addition to the run times the table also reports the
-speedup that PyPy achieves when the optimization is turned on. XXX sounds sucky
+Figure~\ref{fig:times}. For each implementation the table also reports the
+speedup that PyPy with optimization achieves over it.
With the optimization turned on, PyPy's Python interpreter outperforms CPython
in all benchmarks except spambayes (which heavily relies on regular expression
From afa at codespeak.net Thu Oct 21 16:14:14 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 21 Oct 2010 16:14:14 +0200 (CEST)
Subject: [pypy-svn] r78174 - in pypy/trunk/pypy: annotation/test interpreter
objspace/flow/test rpython/test translator/c/test
Message-ID: <20101021141414.6E553282BE0@codespeak.net>
Author: afa
Date: Thu Oct 21 16:14:12 2010
New Revision: 78174
Modified:
pypy/trunk/pypy/annotation/test/test_annrpython.py
pypy/trunk/pypy/interpreter/pyopcode.py
pypy/trunk/pypy/objspace/flow/test/test_objspace.py
pypy/trunk/pypy/rpython/test/test_llinterp.py
pypy/trunk/pypy/translator/c/test/test_typed.py
Log:
Add support for "with" statements in RPython.
The goal is to simplify resource management, like temporary allocations,
scoped locks, etc::
with lltype.alloc_buffer(30) as tmpbuf:
[...use tmpbuf...]
The __exit__ function is always called, even when an exception is raised.
Actually there was not much to do (a space.wrap() was missing)
... at least on top of CPython2.5
It's more complex for other versions of CPython, because this opcode
change implementations every release...
Modified: pypy/trunk/pypy/annotation/test/test_annrpython.py
==============================================================================
--- pypy/trunk/pypy/annotation/test/test_annrpython.py (original)
+++ pypy/trunk/pypy/annotation/test/test_annrpython.py Thu Oct 21 16:14:12 2010
@@ -3337,6 +3337,26 @@
s = a.build_types(f, [])
assert isinstance(s, annmodel.SomeChar)
+ def test_context_manager(self):
+ class C:
+ def __init__(self):
+ pass
+ def __enter__(self):
+ self.x = 1
+ def __exit__(self, *args):
+ self.x = 3
+ def f():
+ c = C()
+ with c:
+ pass
+ return c.x
+
+ a = self.RPythonAnnotator()
+ s = a.build_types(f, [])
+ assert isinstance(s, annmodel.SomeInteger)
+ # not a constant: both __enter__ and __exit__ have been annotated
+ assert not s.is_constant()
+
def g(n):
return [0,1,2,n]
Modified: pypy/trunk/pypy/interpreter/pyopcode.py
==============================================================================
--- pypy/trunk/pypy/interpreter/pyopcode.py (original)
+++ pypy/trunk/pypy/interpreter/pyopcode.py Thu Oct 21 16:14:12 2010
@@ -873,15 +873,34 @@
def WITH_CLEANUP(self, oparg, next_instr):
# see comment in END_FINALLY for stack state
- w_exitfunc = self.popvalue()
- w_unroller = self.peekvalue(2)
+ # This opcode changed a lot between CPython versions
+ if self.pycode.magic >= 0xa0df2ef:
+ # Implementation since 2.7a0: 62191 (introduce SETUP_WITH)
+ raise NotImplementedError("WITH_CLEANUP for CPython 2.7")
+ elif self.pycode.magic >= 0xa0df2d1:
+ # implementation since 2.6a1: 62161 (WITH_CLEANUP optimization)
+ self.popvalue()
+ self.popvalue()
+ w_unroller = self.popvalue()
+ w_exitfunc = self.popvalue()
+ self.pushvalue(w_unroller)
+ self.pushvalue(self.space.w_None)
+ self.pushvalue(self.space.w_None)
+ elif self.pycode.magic >= 0xa0df28c:
+ # Implemementation since 2.5a0: 62092 (changed WITH_CLEANUP opcode)
+ w_exitfunc = self.popvalue()
+ w_unroller = self.peekvalue(2)
+ else:
+ raise NotImplementedError("WITH_CLEANUP for CPython <= 2.4")
+
unroller = self.space.interpclass_w(w_unroller)
if isinstance(unroller, SApplicationException):
operr = unroller.operr
+ w_traceback = self.space.wrap(operr.application_traceback)
w_result = self.space.call_function(w_exitfunc,
operr.w_type,
operr.get_w_value(self.space),
- operr.application_traceback)
+ w_traceback)
if self.space.is_true(w_result):
# __exit__() returned True -> Swallow the exception.
self.settopvalue(self.space.w_None, 2)
Modified: pypy/trunk/pypy/objspace/flow/test/test_objspace.py
==============================================================================
--- pypy/trunk/pypy/objspace/flow/test/test_objspace.py (original)
+++ pypy/trunk/pypy/objspace/flow/test/test_objspace.py Thu Oct 21 16:14:12 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
import new
import py
from pypy.objspace.flow.model import Constant, Block, Link, Variable, traverse
@@ -828,6 +829,25 @@
simplify_graph(graph)
assert self.all_operations(graph) == {'getitem': 1}
+ def test_context_manager(self):
+ def f(c, x):
+ with x:
+ pass
+ graph = self.codetest(f)
+ # 2 method calls: x.__enter__() and x.__exit__(None, None, None)
+ assert self.all_operations(graph) == {'getattr': 2,
+ 'simple_call': 2}
+ #
+ def g(): pass
+ def f(c, x):
+ with x:
+ g()
+ graph = self.codetest(f)
+ assert self.all_operations(graph) == {
+ 'getattr': 2, # __enter__ and __exit__
+ 'simple_call': 4, # __enter__, g and 2 possible calls to __exit__
+ 'is_true': 1} # check the result of __exit__()
+
def monkey_patch_code(self, code, stacksize, flags, codestring, names, varnames):
c = code
return new.code(c.co_argcount, c.co_nlocals, stacksize, flags,
Modified: pypy/trunk/pypy/rpython/test/test_llinterp.py
==============================================================================
--- pypy/trunk/pypy/rpython/test/test_llinterp.py (original)
+++ pypy/trunk/pypy/rpython/test/test_llinterp.py Thu Oct 21 16:14:12 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
import py
import sys
from pypy.rpython.lltypesystem.lltype import typeOf, pyobjectptr, Ptr,\
@@ -5,6 +6,7 @@
from pypy.rpython.lltypesystem.lloperation import llop
from pypy.rpython.llinterp import LLInterpreter, LLException, log
from pypy.rpython.rmodel import inputconst
+from pypy.rpython.annlowlevel import hlstr
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.rint import signed_repr
from pypy.rpython.lltypesystem import rstr, lltype
@@ -622,3 +624,24 @@
interpret(f, [])
+def test_context_manager():
+ state = []
+ class C:
+ def __enter__(self):
+ state.append('acquire')
+ return self
+ def __exit__(self, *args):
+ if args[1] is not None:
+ state.append('raised')
+ state.append('release')
+ def f():
+ try:
+ with C() as c:
+ state.append('use')
+ raise ValueError
+ except ValueError:
+ pass
+ return ', '.join(state)
+ res = interpret(f, [])
+ assert hlstr(res) == 'acquire, use, raised, release'
+
Modified: pypy/trunk/pypy/translator/c/test/test_typed.py
==============================================================================
--- pypy/trunk/pypy/translator/c/test/test_typed.py (original)
+++ pypy/trunk/pypy/translator/c/test/test_typed.py Thu Oct 21 16:14:12 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
import autopath
import sys
import math
@@ -823,3 +824,29 @@
while int(x + frac) >= -sys.maxint-1:
x -= 1
assert f(x + frac) == -666
+
+ def test_context_manager(self):
+ state = []
+ class C:
+ def __init__(self, name):
+ self.name = name
+ def __enter__(self):
+ state.append('acquire')
+ return self
+ def __exit__(self, *args):
+ if args[1] is not None:
+ state.append('raised')
+ state.append('release')
+
+ def func():
+ try:
+ with C('hello') as c:
+ state.append(c.name)
+ raise ValueError
+ except ValueError:
+ pass
+ return ', '.join(state)
+ f = self.getcompiled(func, [])
+ res = f()
+ assert res == 'acquire, hello, raised, release'
+
From afa at codespeak.net Thu Oct 21 17:01:39 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 21 Oct 2010 17:01:39 +0200 (CEST)
Subject: [pypy-svn] r78175 - in pypy/trunk/pypy: interpreter objspace/flow
Message-ID: <20101021150139.4DD6B282BE0@codespeak.net>
Author: afa
Date: Thu Oct 21 17:01:37 2010
New Revision: 78175
Modified:
pypy/trunk/pypy/interpreter/pyopcode.py
pypy/trunk/pypy/objspace/flow/flowcontext.py
Log:
Implement the SETUP_WITH opcode for the flow object space running on code compiled by cpython 2.7.
Now all previous tests run on top of cpython 2.5 to 2.7.
The implementation was borrowed from the fast-forward branch, and slightly simplified
to adapt to the RPython type system, which has virtual methods but no __mro__ or special descriptors.
This is why this code is not in pyopcode.py
Modified: pypy/trunk/pypy/interpreter/pyopcode.py
==============================================================================
--- pypy/trunk/pypy/interpreter/pyopcode.py (original)
+++ pypy/trunk/pypy/interpreter/pyopcode.py Thu Oct 21 17:01:37 2010
@@ -874,10 +874,9 @@
def WITH_CLEANUP(self, oparg, next_instr):
# see comment in END_FINALLY for stack state
# This opcode changed a lot between CPython versions
- if self.pycode.magic >= 0xa0df2ef:
+ if (self.pycode.magic >= 0xa0df2ef
# Implementation since 2.7a0: 62191 (introduce SETUP_WITH)
- raise NotImplementedError("WITH_CLEANUP for CPython 2.7")
- elif self.pycode.magic >= 0xa0df2d1:
+ or self.pycode.magic >= 0xa0df2d1):
# implementation since 2.6a1: 62161 (WITH_CLEANUP optimization)
self.popvalue()
self.popvalue()
@@ -1249,10 +1248,18 @@
frame.pushvalue(frame.space.w_None)
return self.handlerposition # jump to the handler
+class WithBlock(FinallyBlock):
+
+ def really_handle(self, frame, unroller):
+ if (frame.space.full_exceptions and
+ isinstance(unroller, SApplicationException)):
+ unroller.operr.normalize_exception(frame.space)
+ return FinallyBlock.really_handle(self, frame, unroller)
block_classes = {'SETUP_LOOP': LoopBlock,
'SETUP_EXCEPT': ExceptBlock,
- 'SETUP_FINALLY': FinallyBlock}
+ 'SETUP_FINALLY': FinallyBlock,
+ 'SETUP_WITH': WithBlock}
### helpers written at the application-level ###
# Some of these functions are expected to be generally useful if other
Modified: pypy/trunk/pypy/objspace/flow/flowcontext.py
==============================================================================
--- pypy/trunk/pypy/objspace/flow/flowcontext.py (original)
+++ pypy/trunk/pypy/objspace/flow/flowcontext.py Thu Oct 21 17:01:37 2010
@@ -478,10 +478,19 @@
# `with` statement
- def SETUP_WITH(self, oparg, next_instr):
- raise NotImplementedError("SETUP_WITH")
+ def SETUP_WITH(self, offsettoend, next_instr):
+ # A simpler version than the 'real' 2.7 one:
+ # directly call manager.__enter__(), don't use special lookup functions
+ # which are too complex for the flow object space.
+ from pypy.interpreter.pyopcode import WithBlock
+ w_manager = self.peekvalue()
+ w_exit = self.space.getattr(w_manager, self.space.wrap("__exit__"))
+ self.settopvalue(w_exit)
+ w_result = self.space.call_method(w_manager, "__enter__")
+ block = WithBlock(self, next_instr + offsettoend)
+ self.append_block(block)
+ self.pushvalue(w_result)
-
def make_arguments(self, nargs):
return ArgumentsForTranslation(self.space, self.peekvalues(nargs))
def argument_factory(self, *args):
From afa at codespeak.net Thu Oct 21 17:08:51 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 21 Oct 2010 17:08:51 +0200 (CEST)
Subject: [pypy-svn] r78176 - pypy/trunk/pypy/objspace/flow
Message-ID: <20101021150851.74330282BE0@codespeak.net>
Author: afa
Date: Thu Oct 21 17:08:49 2010
New Revision: 78176
Modified:
pypy/trunk/pypy/objspace/flow/flowcontext.py
Log:
Move SETUP_WITH out of the Unimplemented section
Modified: pypy/trunk/pypy/objspace/flow/flowcontext.py
==============================================================================
--- pypy/trunk/pypy/objspace/flow/flowcontext.py (original)
+++ pypy/trunk/pypy/objspace/flow/flowcontext.py Thu Oct 21 17:08:49 2010
@@ -460,21 +460,6 @@
else:
v = self.peekvalue(oparg - 1)
self.space.call_method(v, 'append', w)
-
- # XXX Unimplemented 2.7 opcodes ----------------
-
- # Set literals, set comprehensions
-
- def BUILD_SET(self, oparg, next_instr):
- raise NotImplementedError("BUILD_SET")
-
- def SET_ADD(self, oparg, next_instr):
- raise NotImplementedError("SET_ADD")
-
- # Dict comprehensions
-
- def MAP_ADD(self, oparg, next_instr):
- raise NotImplementedError("MAP_ADD")
# `with` statement
@@ -491,6 +476,21 @@
self.append_block(block)
self.pushvalue(w_result)
+ # XXX Unimplemented 2.7 opcodes ----------------
+
+ # Set literals, set comprehensions
+
+ def BUILD_SET(self, oparg, next_instr):
+ raise NotImplementedError("BUILD_SET")
+
+ def SET_ADD(self, oparg, next_instr):
+ raise NotImplementedError("SET_ADD")
+
+ # Dict comprehensions
+
+ def MAP_ADD(self, oparg, next_instr):
+ raise NotImplementedError("MAP_ADD")
+
def make_arguments(self, nargs):
return ArgumentsForTranslation(self.space, self.peekvalues(nargs))
def argument_factory(self, *args):
From afa at codespeak.net Thu Oct 21 17:11:00 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 21 Oct 2010 17:11:00 +0200 (CEST)
Subject: [pypy-svn] r78177 - pypy/trunk/pypy/objspace/flow
Message-ID: <20101021151100.A38B6282BE0@codespeak.net>
Author: afa
Date: Thu Oct 21 17:10:59 2010
New Revision: 78177
Modified:
pypy/trunk/pypy/objspace/flow/flowcontext.py
Log:
Improve comment
Modified: pypy/trunk/pypy/objspace/flow/flowcontext.py
==============================================================================
--- pypy/trunk/pypy/objspace/flow/flowcontext.py (original)
+++ pypy/trunk/pypy/objspace/flow/flowcontext.py Thu Oct 21 17:10:59 2010
@@ -466,7 +466,7 @@
def SETUP_WITH(self, offsettoend, next_instr):
# A simpler version than the 'real' 2.7 one:
# directly call manager.__enter__(), don't use special lookup functions
- # which are too complex for the flow object space.
+ # which don't make sense on the RPython type system.
from pypy.interpreter.pyopcode import WithBlock
w_manager = self.peekvalue()
w_exit = self.space.getattr(w_manager, self.space.wrap("__exit__"))
From antocuni at codespeak.net Thu Oct 21 17:13:14 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Thu, 21 Oct 2010 17:13:14 +0200 (CEST)
Subject: [pypy-svn] r78178 - pypy/trunk/pypy/rlib/test
Message-ID: <20101021151314.95A88282BE0@codespeak.net>
Author: antocuni
Date: Thu Oct 21 17:13:12 2010
New Revision: 78178
Modified:
pypy/trunk/pypy/rlib/test/test_libffi.py
Log:
(antocuni, arigo): fix the test: it relied on the fact that dummy was 0 at the beginning of the test, but this is not necessarly true if the same shared library is used by someone else (like, metainterp/test/test_ffi_call)
Modified: pypy/trunk/pypy/rlib/test/test_libffi.py
==============================================================================
--- pypy/trunk/pypy/rlib/test/test_libffi.py (original)
+++ pypy/trunk/pypy/rlib/test/test_libffi.py Thu Oct 21 17:13:12 2010
@@ -249,7 +249,7 @@
def test_void_result(self):
"""
- int dummy = 0;
+ int dummy;
void set_dummy(int val) { dummy = val; }
int get_dummy() { return dummy; }
"""
@@ -257,11 +257,10 @@
set_dummy = (libfoo, 'set_dummy', [types.sint], types.void)
get_dummy = (libfoo, 'get_dummy', [], types.sint)
#
- res = self.call(get_dummy, [], rffi.LONG)
- assert res == 0
+ initval = self.call(get_dummy, [], rffi.LONG)
#
- res = self.call(set_dummy, [42], lltype.Void, init_result=None)
+ res = self.call(set_dummy, [initval+1], lltype.Void, init_result=None)
assert res is None
#
res = self.call(get_dummy, [], rffi.LONG)
- assert res == 42
+ assert res == initval+1
From antocuni at codespeak.net Thu Oct 21 17:30:46 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Thu, 21 Oct 2010 17:30:46 +0200 (CEST)
Subject: [pypy-svn] r78179 - in pypy/trunk/pypy: module/_ffi rlib rlib/test
Message-ID: <20101021153046.CAF65282BF0@codespeak.net>
Author: antocuni
Date: Thu Oct 21 17:30:45 2010
New Revision: 78179
Modified:
pypy/trunk/pypy/module/_ffi/interp_ffi.py
pypy/trunk/pypy/rlib/libffi.py
pypy/trunk/pypy/rlib/test/test_libffi.py
Log:
(antocuni, arigo around): finally fix test_unsigned_short_arguments on 64
bits, by handling all the possible cases separately.
Modified: pypy/trunk/pypy/module/_ffi/interp_ffi.py
==============================================================================
--- pypy/trunk/pypy/module/_ffi/interp_ffi.py (original)
+++ pypy/trunk/pypy/module/_ffi/interp_ffi.py Thu Oct 21 17:30:45 2010
@@ -89,11 +89,9 @@
argchain = self.build_argchain(space, self.func.argtypes, args_w)
reskind = libffi.types.getkind(self.func.restype)
if reskind == 'i':
- intres = self.func.call(argchain, rffi.LONG)
- return space.wrap(intres)
+ return self._call_int(space, argchain)
elif reskind == 'u':
- intres = self.func.call(argchain, rffi.ULONG)
- return space.wrap(intres)
+ return self._call_uint(space, argchain)
elif reskind == 'f':
floatres = self.func.call(argchain, rffi.DOUBLE)
return space.wrap(floatres)
@@ -102,6 +100,58 @@
assert voidres is None
return space.w_None
+ def _call_int(self, space, argchain):
+ # if the declared return type of the function is smaller than LONG,
+ # the result buffer may contains garbage in its higher bits. To get
+ # the correct value, and to be sure to handle the signed/unsigned case
+ # correctly, we need to cast the result to the correct type. After
+ # that, we cast it back to LONG, because this is what we want to pass
+ # to space.wrap in order to get a nice applevel .
+ #
+ restype = self.func.restype
+ call = self.func.call
+ if restype is libffi.types.slong:
+ intres = call(argchain, rffi.LONG)
+ elif restype is libffi.types.sint:
+ intres = rffi.cast(rffi.LONG, call(argchain, rffi.INT))
+ elif restype is libffi.types.sshort:
+ intres = rffi.cast(rffi.LONG, call(argchain, rffi.SHORT))
+ elif restype is libffi.types.schar:
+ intres = rffi.cast(rffi.LONG, call(argchain, rffi.SIGNEDCHAR))
+ else:
+ raise OperationError(space.w_ValueError,
+ space.wrap('Unsupported restype'))
+ return space.wrap(intres)
+
+ def _call_uint(self, space, argchain):
+ # the same comment as above apply. Moreover, we need to be careful
+ # when the return type is ULONG, because the value might not fit into
+ # a signed LONG: this is the only case in which we cast the result to
+ # something different than LONG; as a result, the applevel value will
+ # be a .
+ #
+ # Note that we check for ULONG before UINT: this is needed on 32bit
+ # machines, where they are they same: if we checked for UINT before
+ # ULONG, we would cast to the wrong type. Note that this also means
+ # that on 32bit the UINT case will never be entered (because it is
+ # handled by the ULONG case).
+ restype = self.func.restype
+ call = self.func.call
+ if restype is libffi.types.ulong:
+ # special case
+ uintres = call(argchain, rffi.ULONG)
+ return space.wrap(uintres)
+ elif restype is libffi.types.uint:
+ intres = rffi.cast(rffi.LONG, call(argchain, rffi.UINT))
+ elif restype is libffi.types.ushort:
+ intres = rffi.cast(rffi.LONG, call(argchain, rffi.USHORT))
+ elif restype is libffi.types.uchar:
+ intres = rffi.cast(rffi.LONG, call(argchain, rffi.UCHAR))
+ else:
+ raise OperationError(space.w_ValueError,
+ space.wrap('Unsupported restype'))
+ return space.wrap(intres)
+
@unwrap_spec('self', ObjSpace)
def getaddr(self, space):
"""
Modified: pypy/trunk/pypy/rlib/libffi.py
==============================================================================
--- pypy/trunk/pypy/rlib/libffi.py (original)
+++ pypy/trunk/pypy/rlib/libffi.py Thu Oct 21 17:30:45 2010
@@ -188,9 +188,6 @@
#
if _fits_into_long(RESULT):
res = self._do_call_int(self.funcsym, ll_args)
- if self.restype.c_size < types.slong.c_size:
- # mask res to keep only the bits we are interested in
- res &= ~(-1 << self.restype.c_size*8)
elif RESULT is rffi.DOUBLE:
return self._do_call_float(self.funcsym, ll_args)
elif RESULT is lltype.Void:
Modified: pypy/trunk/pypy/rlib/test/test_libffi.py
==============================================================================
--- pypy/trunk/pypy/rlib/test/test_libffi.py (original)
+++ pypy/trunk/pypy/rlib/test/test_libffi.py Thu Oct 21 17:30:45 2010
@@ -182,9 +182,7 @@
"""
libfoo = self.get_libfoo()
func = (libfoo, 'sum_xy_us', [types.ushort, types.ushort], types.ushort)
- # the ULONG in the next line is not a typo: we really want to test
- # that we get the correct value even if we cast it to a larger type
- res = self.call(func, [32000, 8000], rffi.ULONG)
+ res = self.call(func, [32000, 8000], rffi.USHORT)
assert res == 40000
From arigo at codespeak.net Thu Oct 21 18:05:06 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 21 Oct 2010 18:05:06 +0200 (CEST)
Subject: [pypy-svn] r78181 -
pypy/branch/cleanup-dict-impl/pypy/jit/codewriter
Message-ID: <20101021160506.A9671282BE0@codespeak.net>
Author: arigo
Date: Thu Oct 21 18:05:05 2010
New Revision: 78181
Modified:
pypy/branch/cleanup-dict-impl/pypy/jit/codewriter/jtransform.py
Log:
Support for pypy.rlib.rerased: uses cast_opaque_ptr.
Modified: pypy/branch/cleanup-dict-impl/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/jit/codewriter/jtransform.py Thu Oct 21 18:05:05 2010
@@ -172,6 +172,7 @@
def rewrite_op_same_as(self, op): pass
def rewrite_op_cast_pointer(self, op): pass
+ def rewrite_op_cast_opaque_ptr(self, op): pass # rlib.rerased
def rewrite_op_cast_primitive(self, op): pass
def rewrite_op_cast_bool_to_int(self, op): pass
def rewrite_op_cast_bool_to_uint(self, op): pass
From afa at codespeak.net Thu Oct 21 18:09:39 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 21 Oct 2010 18:09:39 +0200 (CEST)
Subject: [pypy-svn] r78182 - in pypy/trunk/pypy/rpython/lltypesystem: . test
Message-ID: <20101021160939.928EF282BE0@codespeak.net>
Author: afa
Date: Thu Oct 21 18:09:35 2010
New Revision: 78182
Modified:
pypy/trunk/pypy/rpython/lltypesystem/lltype.py
pypy/trunk/pypy/rpython/lltypesystem/test/test_lltype.py
Log:
Attempt to define a "scoped allocator" to use in a 'with' statement.
Modified: pypy/trunk/pypy/rpython/lltypesystem/lltype.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/lltype.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/lltype.py Thu Oct 21 18:09:35 2010
@@ -1868,6 +1868,32 @@
leakfinder.remember_free(p._obj0)
p._obj0._free()
+def _make_scoped_allocator(T):
+ class ScopedAlloc:
+ def __init__(self, n=None, zero=False):
+ self.buf = malloc(T, n=n, flavor='raw', zero=zero)
+
+ def __enter__(self):
+ return self.buf
+
+ def __exit__(self, *args):
+ free(self.buf, flavor='raw')
+
+ ScopedAlloc.__name__ = 'ScopedAlloc_%s' % (T,)
+ return ScopedAlloc
+_make_scoped_allocator._annspecialcase_ = 'specialize:memo'
+
+def scoped_alloc(T, n=None, zero=False):
+ """Returns a context manager which handles allocation and
+ deallocation of temporary memory. Use it in a with statement::
+
+ with scoped_alloc(Array(Signed), 1) as array:
+ ...use array...
+ ...it's freed now.
+ """
+ return _make_scoped_allocator(T)(n=n, zero=zero)
+scoped_alloc._annspecialcase_ = 'specialize:arg(0)'
+
def functionptr(TYPE, name, **attrs):
if not isinstance(TYPE, FuncType):
raise TypeError, "functionptr() for FuncTypes only"
Modified: pypy/trunk/pypy/rpython/lltypesystem/test/test_lltype.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/test/test_lltype.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/test/test_lltype.py Thu Oct 21 18:09:35 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
import py
from pypy.rpython.lltypesystem.lltype import *
from pypy.rpython.lltypesystem import lltype, rffi
@@ -846,3 +847,9 @@
p2 = malloc(Array(Signed), 1, flavor='raw', track_allocation=False)
free(p2, flavor='raw', track_allocation=False)
# p1 is not freed
+
+ def test_scoped_allocator(self):
+ with scoped_alloc(Array(Signed), 1) as array:
+ array[0] = -42
+ x = array[0]
+ assert x == -42
From hakanardo at codespeak.net Thu Oct 21 21:31:19 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Thu, 21 Oct 2010 21:31:19 +0200 (CEST)
Subject: [pypy-svn] r78187 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101021193119.D624A282BDC@codespeak.net>
Author: hakanardo
Date: Thu Oct 21 21:31:17 2010
New Revision: 78187
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_exception.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_jitdriver.py
Log:
Some tests ajusted to reflect the new optimization
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py Thu Oct 21 21:31:17 2010
@@ -1453,7 +1453,8 @@
return x
res = self.meta_interp(f, [299], listops=True)
assert res == f(299)
- self.check_loops(guard_class=0, guard_value=3)
+ self.check_loops(guard_class=0, guard_value=1)
+ self.check_loops(guard_class=0, guard_value=4, everywhere=True)
def test_merge_guardnonnull_guardclass(self):
from pypy.rlib.objectmodel import instantiate
@@ -1482,7 +1483,10 @@
res = self.meta_interp(f, [299], listops=True)
assert res == f(299)
self.check_loops(guard_class=0, guard_nonnull=0,
- guard_nonnull_class=2, guard_isnull=1)
+ guard_nonnull_class=1, guard_isnull=0)
+ self.check_loops(guard_class=0, guard_nonnull=0,
+ guard_nonnull_class=3, guard_isnull=1,
+ everywhere=True)
def test_merge_guardnonnull_guardvalue(self):
from pypy.rlib.objectmodel import instantiate
@@ -1509,8 +1513,11 @@
return x
res = self.meta_interp(f, [299], listops=True)
assert res == f(299)
- self.check_loops(guard_class=0, guard_nonnull=0, guard_value=2,
- guard_nonnull_class=0, guard_isnull=1)
+ self.check_loops(guard_class=0, guard_nonnull=0, guard_value=1,
+ guard_nonnull_class=0, guard_isnull=0)
+ self.check_loops(guard_class=0, guard_nonnull=0, guard_value=3,
+ guard_nonnull_class=0, guard_isnull=1,
+ everywhere=True)
def test_merge_guardnonnull_guardvalue_2(self):
from pypy.rlib.objectmodel import instantiate
@@ -1537,8 +1544,11 @@
return x
res = self.meta_interp(f, [299], listops=True)
assert res == f(299)
- self.check_loops(guard_class=0, guard_nonnull=0, guard_value=2,
- guard_nonnull_class=0, guard_isnull=1)
+ self.check_loops(guard_class=0, guard_nonnull=0, guard_value=1,
+ guard_nonnull_class=0, guard_isnull=0)
+ self.check_loops(guard_class=0, guard_nonnull=0, guard_value=3,
+ guard_nonnull_class=0, guard_isnull=1,
+ everywhere=True)
def test_merge_guardnonnull_guardclass_guardvalue(self):
from pypy.rlib.objectmodel import instantiate
@@ -1568,8 +1578,11 @@
return x
res = self.meta_interp(f, [399], listops=True)
assert res == f(399)
- self.check_loops(guard_class=0, guard_nonnull=0, guard_value=3,
- guard_nonnull_class=0, guard_isnull=1)
+ self.check_loops(guard_class=0, guard_nonnull=0, guard_value=1,
+ guard_nonnull_class=0, guard_isnull=0)
+ self.check_loops(guard_class=0, guard_nonnull=0, guard_value=4,
+ guard_nonnull_class=0, guard_isnull=1,
+ everywhere=True)
def test_residual_call_doesnt_lose_info(self):
myjitdriver = JitDriver(greens = [], reds = ['x', 'y', 'l'])
@@ -1595,7 +1608,8 @@
y.v = g(y.v) - y.v/y.v + lc/l[0] - 1
return y.v
res = self.meta_interp(f, [20], listops=True)
- self.check_loops(getfield_gc=1, getarrayitem_gc=0)
+ self.check_loops(getfield_gc=0, getarrayitem_gc=0)
+ self.check_loops(getfield_gc=1, getarrayitem_gc=0, everywhere=True)
def test_guard_isnull_nonnull(self):
myjitdriver = JitDriver(greens = [], reds = ['x', 'res'])
@@ -1625,7 +1639,7 @@
assert res == 42
self.check_loops(guard_nonnull=1, guard_isnull=1)
- def test_loop_invariant(self):
+ def test_loop_invariant1(self):
myjitdriver = JitDriver(greens = [], reds = ['x', 'res'])
class A(object):
pass
@@ -1650,7 +1664,8 @@
return res
res = self.meta_interp(g, [21])
assert res == 3 * 21
- self.check_loops(call=1)
+ self.check_loops(call=0)
+ self.check_loops(call=1, everywhere=True)
def test_bug_optimizeopt_mutates_ops(self):
myjitdriver = JitDriver(greens = [], reds = ['x', 'res', 'const', 'a'])
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py Thu Oct 21 21:31:17 2010
@@ -5,7 +5,7 @@
from pypy.jit.metainterp.compile import ResumeGuardDescr
from pypy.jit.metainterp.compile import ResumeGuardCountersInt
from pypy.jit.metainterp.compile import compile_tmp_callback
-from pypy.jit.metainterp import optimize, jitprof, typesystem, compile
+from pypy.jit.metainterp import optimize_nopspec, jitprof, typesystem, compile
from pypy.jit.metainterp.test.oparser import parse
from pypy.jit.metainterp.test.test_optimizefindnode import LLtypeMixin
@@ -41,9 +41,12 @@
pass
class FakeState:
- optimize_loop = staticmethod(optimize.optimize_loop)
+ optimize_loop = staticmethod(optimize_nopspec.optimize_loop)
debug_level = 0
+ def attach_unoptimized_bridge_from_interp(*args):
+ pass
+
class FakeGlobalData:
loopnumbering = 0
@@ -87,11 +90,11 @@
loop_tokens = []
loop_token = compile_new_loop(metainterp, loop_tokens, [], 0)
assert loop_tokens == [loop_token]
- assert loop_token.number == 1
- assert staticdata.globaldata.loopnumbering == 2
+ assert loop_token.number == 2
+ assert staticdata.globaldata.loopnumbering == 3
#
- assert len(cpu.seen) == 1
- assert cpu.seen[0][2] == loop_token
+ assert len(cpu.seen) == 2
+ assert cpu.seen[1][2] == loop_token
#
del cpu.seen[:]
metainterp = FakeMetaInterp()
@@ -105,7 +108,7 @@
assert loop_token_2 is loop_token
assert loop_tokens == [loop_token]
assert len(cpu.seen) == 0
- assert staticdata.globaldata.loopnumbering == 2
+ assert staticdata.globaldata.loopnumbering == 3
def test_resume_guard_counters():
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_exception.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_exception.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_exception.py Thu Oct 21 21:31:17 2010
@@ -516,10 +516,9 @@
assert res == -1
self.check_tree_loop_count(2) # the loop and the entry path
# we get:
- # ENTER - compile the new loop
- # ENTER - compile the entry bridge
+ # ENTER - compile the new loop and the entry bridge
# ENTER - compile the leaving path (raising MyError)
- self.check_enter_count(3)
+ self.check_enter_count(2)
def test_bridge_from_interpreter_exc_2(self):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_jitdriver.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_jitdriver.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_jitdriver.py Thu Oct 21 21:31:17 2010
@@ -42,7 +42,7 @@
# at the generated machine code
self.check_loop_count(5)
self.check_tree_loop_count(4) # 2 x loop, 2 x enter bridge
- self.check_enter_count(7)
+ self.check_enter_count(5)
def test_inline(self):
# this is not an example of reasonable code: loop1() is unrolled
From trundle at codespeak.net Fri Oct 22 00:00:17 2010
From: trundle at codespeak.net (trundle at codespeak.net)
Date: Fri, 22 Oct 2010 00:00:17 +0200 (CEST)
Subject: [pypy-svn] r78188 - pypy/branch/fast-forward/pypy/module/exceptions
Message-ID: <20101021220017.7CCB1282BDC@codespeak.net>
Author: trundle
Date: Fri Oct 22 00:00:15 2010
New Revision: 78188
Modified:
pypy/branch/fast-forward/pypy/module/exceptions/__init__.py
pypy/branch/fast-forward/pypy/module/exceptions/interp_exceptions.py
Log:
Add BufferError, make GeneratorExit inherit from BaseException and update
exception hierarchy in docstring.
test_pep352 passes now.
Modified: pypy/branch/fast-forward/pypy/module/exceptions/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/exceptions/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/exceptions/__init__.py Fri Oct 22 00:00:15 2010
@@ -9,6 +9,7 @@
'AssertionError' : 'interp_exceptions.W_AssertionError',
'AttributeError' : 'interp_exceptions.W_AttributeError',
'BaseException' : 'interp_exceptions.W_BaseException',
+ 'BufferError' : 'interp_exceptions.W_BufferError',
'BytesWarning' : 'interp_exceptions.W_BytesWarning',
'DeprecationWarning' : 'interp_exceptions.W_DeprecationWarning',
'EOFError' : 'interp_exceptions.W_EOFError',
Modified: pypy/branch/fast-forward/pypy/module/exceptions/interp_exceptions.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/exceptions/interp_exceptions.py (original)
+++ pypy/branch/fast-forward/pypy/module/exceptions/interp_exceptions.py Fri Oct 22 00:00:15 2010
@@ -23,10 +23,11 @@
BaseException
+-- SystemExit
+-- KeyboardInterrupt
+ +-- GeneratorExit
+-- Exception
- +-- GeneratorExit
+-- StopIteration
+-- StandardError
+ | +-- BufferError
| +-- ArithmeticError
| | +-- FloatingPointError
| | +-- OverflowError
@@ -55,20 +56,20 @@
| +-- SystemError
| +-- TypeError
| +-- ValueError
- | | +-- UnicodeError
- | | +-- UnicodeDecodeError
- | | +-- UnicodeEncodeError
- | | +-- UnicodeTranslateError
+ | +-- UnicodeError
+ | +-- UnicodeDecodeError
+ | +-- UnicodeEncodeError
+ | +-- UnicodeTranslateError
+-- Warning
- +-- BytesWarning
+-- DeprecationWarning
+-- PendingDeprecationWarning
+-- RuntimeWarning
+-- SyntaxWarning
+-- UserWarning
+-- FutureWarning
- +-- ImportWarning
- +-- UnicodeWarning
+ +-- ImportWarning
+ +-- UnicodeWarning
+ +-- BytesWarning
"""
from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root
@@ -262,12 +263,15 @@
W_Exception = _new_exception('Exception', W_BaseException,
"""Common base class for all non-exit exceptions.""")
-W_GeneratorExit = _new_exception('GeneratorExit', W_Exception,
+W_GeneratorExit = _new_exception('GeneratorExit', W_BaseException,
"""Request that a generator exit.""")
W_StandardError = _new_exception('StandardError', W_Exception,
"""Base class for all standard Python exceptions.""")
+W_BufferError = _new_exception('BufferError', W_StandardError,
+ """Buffer error.""")
+
W_ValueError = _new_exception('ValueError', W_StandardError,
"""Inappropriate argument value (of correct type).""")
From afa at codespeak.net Fri Oct 22 09:57:57 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 22 Oct 2010 09:57:57 +0200 (CEST)
Subject: [pypy-svn] r78189 - pypy/branch/fast-forward/pypy/module/exceptions
Message-ID: <20101022075757.C057D36E0CB@codespeak.net>
Author: afa
Date: Fri Oct 22 09:57:54 2010
New Revision: 78189
Modified:
pypy/branch/fast-forward/pypy/module/exceptions/interp_exceptions.py
Log:
Untabify
Modified: pypy/branch/fast-forward/pypy/module/exceptions/interp_exceptions.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/exceptions/interp_exceptions.py (original)
+++ pypy/branch/fast-forward/pypy/module/exceptions/interp_exceptions.py Fri Oct 22 09:57:54 2010
@@ -67,9 +67,9 @@
+-- SyntaxWarning
+-- UserWarning
+-- FutureWarning
- +-- ImportWarning
- +-- UnicodeWarning
- +-- BytesWarning
+ +-- ImportWarning
+ +-- UnicodeWarning
+ +-- BytesWarning
"""
from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root
From dan at codespeak.net Fri Oct 22 10:24:52 2010
From: dan at codespeak.net (dan at codespeak.net)
Date: Fri, 22 Oct 2010 10:24:52 +0200 (CEST)
Subject: [pypy-svn] r78190 - pypy/branch/ootype-virtualrefs
Message-ID: <20101022082452.4D064282BDC@codespeak.net>
Author: dan
Date: Fri Oct 22 10:24:50 2010
New Revision: 78190
Added:
pypy/branch/ootype-virtualrefs/ (props changed)
- copied from r78189, pypy/trunk/
Log:
Hopefully this branch will lead to fixing ootype translation.
From dan at codespeak.net Fri Oct 22 10:32:43 2010
From: dan at codespeak.net (dan at codespeak.net)
Date: Fri, 22 Oct 2010 10:32:43 +0200 (CEST)
Subject: [pypy-svn] r78191 - in pypy/branch/ootype-virtualrefs/pypy: rlib
translator/goal
Message-ID: <20101022083243.0C7BD282BDC@codespeak.net>
Author: dan
Date: Fri Oct 22 10:32:41 2010
New Revision: 78191
Modified:
pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py
pypy/branch/ootype-virtualrefs/pypy/translator/goal/targetpypystandalone.py
Log:
SomeVRef now has a more correct Repr on ootypesystem
Modified: pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py
==============================================================================
--- pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py (original)
+++ pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py Fri Oct 22 10:32:41 2010
@@ -24,7 +24,10 @@
return self.s_instance
def rtyper_makerepr(self, rtyper):
- return vrefrepr
+ if rtyper.type_system.name == 'lltypesystem':
+ return vrefrepr
+ elif rtyper.type_system.name == 'ootypesystem':
+ return oovrefrepr
def rtyper_makekey(self):
return self.__class__,
@@ -54,4 +57,10 @@
" prebuilt virtual_ref")
return lltype.nullptr(OBJECTPTR.TO)
+from pypy.rpython.ootypesystem.rclass import OBJECT
+
+class OOVRefRepr(VRefRepr):
+ lowleveltype = OBJECT
+
vrefrepr = VRefRepr()
+oovrefrepr = OOVRefRepr()
Modified: pypy/branch/ootype-virtualrefs/pypy/translator/goal/targetpypystandalone.py
==============================================================================
--- pypy/branch/ootype-virtualrefs/pypy/translator/goal/targetpypystandalone.py (original)
+++ pypy/branch/ootype-virtualrefs/pypy/translator/goal/targetpypystandalone.py Fri Oct 22 10:32:41 2010
@@ -105,7 +105,6 @@
print 'Translation to cli and jvm is known to be broken at the moment'
print 'Please try the "cli-jit" branch at:'
print 'http://codespeak.net/svn/pypy/branch/cli-jit/'
- sys.exit(1)
self.translateconfig = translateconfig
# set up the objspace optimizations based on the --opt argument
From dan at codespeak.net Fri Oct 22 10:46:09 2010
From: dan at codespeak.net (dan at codespeak.net)
Date: Fri, 22 Oct 2010 10:46:09 +0200 (CEST)
Subject: [pypy-svn] r78192 - pypy/branch/ootype-virtualrefs/pypy/rlib/test
Message-ID: <20101022084609.A84F7282BDC@codespeak.net>
Author: dan
Date: Fri Oct 22 10:46:07 2010
New Revision: 78192
Modified:
pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py
Log:
Made tests run on both lltypesystem and ootypesystem.
Modified: pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py
==============================================================================
--- pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py (original)
+++ pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py Fri Oct 22 10:46:07 2010
@@ -8,6 +8,7 @@
from pypy.rpython.lltypesystem.rclass import OBJECTPTR
from pypy.rpython.lltypesystem import lltype
+from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
class X(object):
pass
@@ -79,37 +80,44 @@
assert s.s_instance.can_be_None
assert s.s_instance.classdef == a.bookkeeper.getuniqueclassdef(X)
-def test_rtype_1():
- def f():
- return virtual_ref(X())
- x = interpret(f, [])
- assert lltype.typeOf(x) == OBJECTPTR
+class BaseTestVRef(BaseRtypingTest):
+ def test_rtype_1(self):
+ def f():
+ return virtual_ref(X())
+ x = self.interpret(f, [])
+ self.is_of_instance_type(x)
-def test_rtype_2():
- def f():
- x1 = X()
- vref = virtual_ref(x1)
- x2 = vref()
- virtual_ref_finish(x2)
- return x2
- x = interpret(f, [])
- assert lltype.castable(OBJECTPTR, lltype.typeOf(x)) > 0
+ def test_rtype_2(self):
+ def f():
+ x1 = X()
+ vref = virtual_ref(x1)
+ x2 = vref()
+ virtual_ref_finish(x2)
+ return x2
+ x = self.interpret(f, [])
+ assert lltype.castable(OBJECTPTR, lltype.typeOf(x)) > 0
+
+ def test_rtype_3(self):
+ def f(n):
+ if n > 0:
+ return virtual_ref(Y())
+ else:
+ return non_virtual_ref(Z())
+ x = self.interpret(f, [-5])
+ assert lltype.typeOf(x) == OBJECTPTR
+
+ def test_rtype_4(self):
+ def f(n):
+ if n > 0:
+ return virtual_ref(X())
+ else:
+ return vref_None
+ x = self.interpret(f, [-5])
+ assert lltype.typeOf(x) == OBJECTPTR
+ assert not x
-def test_rtype_3():
- def f(n):
- if n > 0:
- return virtual_ref(Y())
- else:
- return non_virtual_ref(Z())
- x = interpret(f, [-5])
- assert lltype.typeOf(x) == OBJECTPTR
+class TestLLtype(BaseTestVRef, LLRtypeMixin):
+ pass
-def test_rtype_4():
- def f(n):
- if n > 0:
- return virtual_ref(X())
- else:
- return vref_None
- x = interpret(f, [-5])
- assert lltype.typeOf(x) == OBJECTPTR
- assert not x
+class TestOOtype(BaseTestVRef, OORtypeMixin):
+ pass
From dan at codespeak.net Fri Oct 22 11:45:43 2010
From: dan at codespeak.net (dan at codespeak.net)
Date: Fri, 22 Oct 2010 11:45:43 +0200 (CEST)
Subject: [pypy-svn] r78194 - pypy/branch/ootype-virtualrefs/pypy/rlib/test
Message-ID: <20101022094543.5B813282BDC@codespeak.net>
Author: dan
Date: Fri Oct 22 11:45:41 2010
New Revision: 78194
Modified:
pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py
Log:
Fixed the tests a bit more.
Modified: pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py
==============================================================================
--- pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py (original)
+++ pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py Fri Oct 22 11:45:41 2010
@@ -85,7 +85,7 @@
def f():
return virtual_ref(X())
x = self.interpret(f, [])
- self.is_of_instance_type(x)
+ assert self.is_of_instance_type(x)
def test_rtype_2(self):
def f():
@@ -104,7 +104,7 @@
else:
return non_virtual_ref(Z())
x = self.interpret(f, [-5])
- assert lltype.typeOf(x) == OBJECTPTR
+ assert self.is_of_instance_type(x)
def test_rtype_4(self):
def f(n):
@@ -113,7 +113,7 @@
else:
return vref_None
x = self.interpret(f, [-5])
- assert lltype.typeOf(x) == OBJECTPTR
+ assert self.is_of_instance_type(x)
assert not x
class TestLLtype(BaseTestVRef, LLRtypeMixin):
From arigo at codespeak.net Fri Oct 22 13:09:22 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 22 Oct 2010 13:09:22 +0200 (CEST)
Subject: [pypy-svn] r78195 - in pypy/trunk/pypy/module: array pypyjit/test
Message-ID: <20101022110922.EE9D5282BDC@codespeak.net>
Author: arigo
Date: Fri Oct 22 13:09:20 2010
New Revision: 78195
Modified:
pypy/trunk/pypy/module/array/interp_array.py
pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
Log:
On 64-bit machines, consider 'I' to fit in a Signed too.
Modified: pypy/trunk/pypy/module/array/interp_array.py
==============================================================================
--- pypy/trunk/pypy/module/array/interp_array.py (original)
+++ pypy/trunk/pypy/module/array/interp_array.py Fri Oct 22 13:09:20 2010
@@ -323,7 +323,7 @@
idx, stop, step = space.decode_index(w_idx, self.len)
assert step == 0
item = self.buffer[idx]
- if mytype.typecode in 'bBhHil':
+ if mytype.canoverflow: # if it fits in a Signed
item = rffi.cast(lltype.Signed, item)
elif mytype.typecode == 'f':
item = float(item)
Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Fri Oct 22 13:09:20 2010
@@ -837,6 +837,9 @@
# XXX this test is way too imprecise in what it is actually testing
# it should count the number of guards instead
for tc, maxops in zip('ilILd', (67, 67, 69, 69, 61)):
+ print
+ print '='*65
+ print '='*20, 'running test for tc=%r' % (tc,), '='*20
res = 73574560
if tc in 'IL':
res = long(res)
From arigo at codespeak.net Fri Oct 22 13:27:37 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 22 Oct 2010 13:27:37 +0200 (CEST)
Subject: [pypy-svn] r78196 -
pypy/branch/cleanup-dict-impl/pypy/module/pypyjit/test
Message-ID: <20101022112737.777D9282BDC@codespeak.net>
Author: arigo
Date: Fri Oct 22 13:27:36 2010
New Revision: 78196
Modified:
pypy/branch/cleanup-dict-impl/pypy/module/pypyjit/test/test_pypy_c.py
Log:
Dump the loop as loaded from the log file, instead of dumping it
after being parsed and reformatted (the descrs are lost).
Modified: pypy/branch/cleanup-dict-impl/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/module/pypyjit/test/test_pypy_c.py Fri Oct 22 13:27:36 2010
@@ -136,9 +136,10 @@
assert opslogfile.check()
log = logparser.parse_log_file(str(opslogfile))
parts = logparser.extract_category(log, 'jit-log-opt-')
+ self.rawloops = [part for part in parts
+ if not from_entry_bridge(part, parts)]
# skip entry bridges, they can contain random things
- self.loops = [parse(part, no_namespace=True) for part in parts
- if not from_entry_bridge(part, parts)]
+ self.loops = [parse(part, no_namespace=True) for part in self.rawloops]
self.sliced_loops = [] # contains all bytecodes of all loops
self.total_ops = 0
for loop in self.loops:
@@ -162,12 +163,11 @@
return [ops for ops in self.sliced_loops if ops.bytecode == name]
def print_loops(self):
- for loop in self.loops:
+ for rawloop in self.rawloops:
print
print '@' * 79
print
- for op in loop.operations:
- print op
+ print rawloop.rstrip()
print
print '@' * 79
From cfbolz at codespeak.net Fri Oct 22 13:50:51 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 22 Oct 2010 13:50:51 +0200 (CEST)
Subject: [pypy-svn] r78197 - pypy/extradoc/talk/pepm2011
Message-ID: <20101022115051.AE9E4282BDC@codespeak.net>
Author: cfbolz
Date: Fri Oct 22 13:50:50 2010
New Revision: 78197
Modified:
pypy/extradoc/talk/pepm2011/paper.bib
Log:
capitalization of things, use dls paper for spur
Modified: pypy/extradoc/talk/pepm2011/paper.bib
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.bib (original)
+++ pypy/extradoc/talk/pepm2011/paper.bib Fri Oct 22 13:50:50 2010
@@ -1,6 +1,6 @@
?
@article{choi_escape_1999,
- title = {Escape analysis for Java},
+ title = {Escape analysis for {J}ava},
volume = {34},
url = {http://portal.acm.org/citation.cfm?id=320386},
doi = {10.1145/320385.320386},
@@ -59,22 +59,8 @@
pages = {328--347}
},
- at techreport{michael_bebenita_spur:_2010,
- title = {{SPUR:} A {Trace-Based} {JIT} Compiler for {CIL}},
- abstract = {Tracing just-in-time compilers {(TJITs)} determine frequently executed traces (hot paths and loops) in running programs and focus their optimization effort by emitting optimized machine code specialized to these traces. Prior work has established this strategy to be especially beneficial for dynamic languages such as {JavaScript,} where the {TJIT} interfaces with the interpreter and produces machine code from the {JavaScript} trace.
-
-This direct coupling with a {JavaScript} interpreter makes it difficult to harness the power of a {TJIT} for other components that are not written in {JavaScript,} e.g., the {DOM} implementation or the layout engine inside a browser. Furthermore, if a {TJIT} is tied to a particular high-level language interpreter, it is difficult to reuse it for other input languages as the optimizations are likely targeted at specific idioms of the source language.
-
-To address these issues, we designed and implemented a {TJIT} for Microsoft?s Common Intermediate Language {CIL} (the target language of C\#, {VisualBasic,} F\#, and many other languages). Working on {CIL} enables {TJIT} optimizations for any program compiled to this platform. In addition, to validate that the performance gains of a {TJIT} for {JavaScript} do not depend on specific idioms of {JavaScript} that are lost in the translation to {CIL,} we provide a performance evaluation of our {JavaScript} runtime which translates {JavaScript} to {CIL} and then runs on top of our {CIL} {TJIT.}},
- number = {{MSR-TR-2010-27}},
- institution = {Microsoft Research},
- author = {Michael Bebenita and Florian Brandner and Manuel Fahndrich and Francesco Logozzo and Wolfram Schulte and Nikolai Tillmann and Herman Venter},
- month = mar,
- year = {2010},
-},
-
@article{bruno_blanchet_escape_2003,
- title = {Escape analysis for Java: Theory and practice},
+ title = {Escape analysis for {J}ava: Theory and practice},
volume = {25},
shorttitle = {Escape analysis for {Java{\textless}sup{\textgreater}TM{\textless}/sup{\textgreater}}},
url = {http://portal.acm.org/citation.cfm?id=945885.945886},
@@ -89,7 +75,7 @@
@incollection{mazur_practical_2001,
series = {Lecture Notes in Computer Science},
- title = {Practical Aspects for a Working Compile Time Garbage Collection System for Mercury},
+ title = {Practical Aspects for a Working Compile Time Garbage Collection System for {M}ercury},
volume = {2237},
url = {http://dx.doi.org/10.1007/3-540-45635-X_15},
abstract = {Compile-time garbage collection {(CTGC)} is still a very uncommon feature within compilers. In previous work we have developed a compile-time structure reuse system for Mercury, a logic programming language. This system indicates which datastructures can safely be reused at run-time. As preliminary experiments were promising, we have continued this work and have now a working and well performing near-to-ship {CTGC-system} built into the Melbourne Mercury Compiler {(MMC).} In this paper we present the multiple design decisions leading to this system, we report the results of using {CTGC} for a set of benchmarks, including a real-world program, and finally we discuss further possible improvements. Benchmarks show substantial memory savings and a noticeable reduction in execution time.},
@@ -114,7 +100,7 @@
},
@phdthesis{cuni_high_2010,
- title = {High performance implementation of Python for {CLI/.NET} with {JIT} compiler generation for dynamic languages.},
+ title = {High performance implementation of {P}ython for {CLI/.NET} with {JIT} compiler generation for dynamic languages.},
school = {Dipartimento di Informatica e Scienze {dell'Informazione,} University of Genova},
author = {Antonio Cuni},
year = {2010},
@@ -123,7 +109,7 @@
@inproceedings{carl_friedrich_bolz_towards_2010,
address = {Hagenberg, Austria},
- title = {Towards a Jitting {VM} for Prolog execution},
+ title = {Towards a Jitting {VM} for {P}rolog execution},
isbn = {978-1-4503-0132-9},
url = {http://portal.acm.org/citation.cfm?id=1836102},
doi = {10.1145/1836089.1836102},
@@ -135,6 +121,35 @@
pages = {99--108}
},
+ at inproceedings{mogensen_constructor_1993,
+ address = {Copenhagen, Denmark},
+ title = {Constructor specialization},
+ isbn = {0-89791-594-1},
+ url = {http://portal.acm.org/citation.cfm?id=154633},
+ doi = {10.1145/154630.154633},
+ abstract = {In the section on ?challenging problems? in the proceedings from the first international workshop on partial evaluation and mixed computation {[BEJ88]} a question is stated: {?Can} {PE} be used to generate new specialized data types, in a way analogous to generating specialized functions?. Since then little has been done to address this problem. In {[Lau89],} new types are indeed generated, but they are all simpler versions of the types in the original program. It is, e.g. not possible to have types with more constructors than the types in the original program. I propose to alleviate this by means of constructor specialization. Constructors are specialized with respect to the static parts of their arguments, just like residual functions. I show how this is done and argue that it makes it possible to get good results from partial evaluation in cases where the traditional methods fail to produce satisfactory results. The discussion is centered around a small subset of Standard {ML,} but the idea applies equally well to other languages having user defined constructors, e.g. Haskell and Prolog.},
+ booktitle = {Proceedings of the 1993 {ACM} {SIGPLAN} symposium on Partial evaluation and semantics-based program manipulation},
+ publisher = {{ACM}},
+ author = {Torben ? Mogensen},
+ year = {1993},
+ pages = {22--32}
+},
+
+ at inproceedings{bebenita_spur:_2010,
+ address = {{Reno/Tahoe,} Nevada, {USA}},
+ title = {{SPUR:} a trace-based {JIT} compiler for {CIL}},
+ isbn = {978-1-4503-0203-6},
+ shorttitle = {{SPUR}},
+ url = {http://portal.acm.org/citation.cfm?id=1869459.1869517&coll=GUIDE&dl=GUIDE&type=series&idx=SERIES318&part=series&WantType=Proceedings&title=OOPSLA%2FSPLASH&CFID=106280261&CFTOKEN=29377718},
+ doi = {10.1145/1869459.1869517},
+ abstract = {Tracing just-in-time compilers {(TJITs)} determine frequently executed traces (hot paths and loops) in running programs and focus their optimization effort by emitting optimized machine code specialized to these traces. Prior work has established this strategy to be especially beneficial for dynamic languages such as {JavaScript,} where the {TJIT} interfaces with the interpreter and produces machine code from the {JavaScript} trace.},
+ booktitle = {Proceedings of the {ACM} international conference on Object oriented programming systems languages and applications},
+ publisher = {{ACM}},
+ author = {Michael Bebenita and Florian Brandner and Manuel Fahndrich and Francesco Logozzo and Wolfram Schulte and Nikolai Tillmann and Herman Venter},
+ year = {2010},
+ pages = {708--725}
+},
+
@inproceedings{chang_tracing_2009,
address = {Washington, {DC,} {USA}},
title = {Tracing for Web 3.0: Trace Compilation for the Next Generation Web Applications},
@@ -166,7 +181,7 @@
},
@article{futamura_partial_1999,
- title = {Partial Evaluation of Computation Process - An Approach to a {Compiler-Compiler}},
+ title = {Partial Evaluation of Computation Process - An Approach to a Compiler-Compiler},
volume = {12},
url = {http://citeseer.ist.psu.edu/futamura99partial.html},
number = {4},
@@ -201,7 +216,7 @@
},
@article{blanchet_escape_1999,
- title = {Escape Analysis for Object Oriented Languages. Application to Java},
+ title = {Escape Analysis for Object Oriented Languages. Application to {J}ava},
url = {http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.47.2277},
doi = {10.1.1.47.2277},
journal = {In Proceedings of the 14th Annual Conference on {Object-Oriented} Programming Systems, Languages and Applications},
@@ -211,7 +226,7 @@
},
@article{georges_statistically_2007,
- title = {Statistically rigorous java performance evaluation},
+ title = {Statistically rigorous {J}ava performance evaluation},
volume = {42},
url = {http://portal.acm.org/citation.cfm?id=1297105.1297033},
doi = {10.1145/1297105.1297033},
@@ -232,7 +247,7 @@
},
@inproceedings{andreas_gal_trace-based_2009,
- title = {Trace-based {Just-in-Time} Type Specialization for Dynamic Languages},
+ title = {Trace-based Just-in-Time Type Specialization for Dynamic Languages},
booktitle = {{PLDI}},
author = {Andreas Gal and Brendan Eich and Mike Shaver and David Anderson and Blake Kaplan and Graydon Hoare and David Mandelin and Boris Zbarsky and Jason Orendorff and Michael Bebenita and Mason Chang and Michael Franz and Edwin Smith and Rick Reitmaier and Mohammad Haghighat},
year = {2009},
@@ -254,7 +269,7 @@
},
@techreport{mason_chang_efficient_2007,
- title = {Efficient {Just-In-Time} Execution of Dynamically Typed Languages
+ title = {Efficient Just-In-Time Execution of Dynamically Typed Languages
Via Code Specialization Using Precise Runtime Type Inference},
abstract = {Dynamically typed languages such as {JavaScript} present a challenge to just-in-time compilers. In contrast to statically typed languages such as {JVML,} in which there are specific opcodes for common operations on primitive types (such as iadd for integer addition), all operations in dynamically typed language such as {JavaScript} are late-bound. Often enough, types cannot be inferred with certainty ahead of execution. As a result, just-in-time compilers for dynamically typed languages have tended to perform worse than their statically-typed counterparts. We present a new approach to compiling dynamically typed languages in which code traces observed during execution are dynamically specialized for each actually observed run-time type. For most benchmark programs, our prototype {JavaScript} virtual machine outperforms every other {JavaScript} platform known to us.},
number = {{ICS-TR-07-10}},
@@ -316,7 +331,7 @@
@inproceedings{rigo_representation-based_2004,
address = {Verona, Italy},
- title = {Representation-based just-in-time specialization and the Psyco prototype for Python},
+ title = {Representation-based just-in-time specialization and the {P}syco prototype for {P}ython},
isbn = {1-58113-835-0},
url = {http://portal.acm.org/citation.cfm?id=1014010},
doi = {10.1145/1014007.1014010},
@@ -329,7 +344,7 @@
},
@incollection{carl_friedrich_bolz_back_2008,
- title = {Back to the Future in One Week ? Implementing a Smalltalk {VM} in {PyPy}},
+ title = {Back to the Future in One Week ? Implementing a {S}malltalk {VM} in {PyPy}},
url = {http://dx.doi.org/10.1007/978-3-540-89275-5_7},
abstract = {We report on our experiences with the Spy project, including implementation details and benchmark results. Spy is a re-implementation of the Squeak (i.e. Smalltalk-80) {VM} using the {PyPy} toolchain. The {PyPy} project allows code written in {RPython,} a subset of Python, to be translated
to a multitude of different backends and architectures. During the translation, many aspects of the implementation can be
@@ -342,13 +357,3 @@
year = {2008},
pages = {123--139}
}
- at inproceedings{Mogensen:PEPM93,
- author = {Torben Mogensen},
- title = {Constructor Specialization},
- booktitle = {Proceedings of PEPM'93, the ACM Sigplan Symposium on Partial
- Evaluation and Semantics-Based Program Manipulation},
- year = 1993,
- month = {June},
- publisher = {ACM Press},
- pages = {22--32},
-}
\ No newline at end of file
From cfbolz at codespeak.net Fri Oct 22 13:56:31 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 22 Oct 2010 13:56:31 +0200 (CEST)
Subject: [pypy-svn] r78198 - pypy/extradoc/talk/pepm2011
Message-ID: <20101022115631.8B68B282BDC@codespeak.net>
Author: cfbolz
Date: Fri Oct 22 13:56:29 2010
New Revision: 78198
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/math.lyx
pypy/extradoc/talk/pepm2011/paper.tex
Log:
various fixes, as well as two bugs in the math
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/math.lyx
==============================================================================
--- pypy/extradoc/talk/pepm2011/math.lyx (original)
+++ pypy/extradoc/talk/pepm2011/math.lyx Fri Oct 22 13:56:29 2010
@@ -552,7 +552,7 @@
\begin_inset Text
\begin_layout Plain Layout
-\begin_inset Formula ${\displaystyle \frac{v^{*}\in\mathrm{dom}(S),\,\left(v^{*},S\right)\overset{\mathrm{liftfields}}{=\!=\!\Longrightarrow}\left(\mathrm{ops},S^{\prime}\right)}{v^{*},S\overset{\mathrm{lift}}{\Longrightarrow}\left\langle v^{*}=\mathtt{new}\left(T\right)\right\rangle ::ops,S^{\prime}}}$
+\begin_inset Formula ${\displaystyle \frac{v^{*}\in\mathrm{dom}(S),\,\left(v^{*},S\right)\overset{\mathrm{liftfields}}{=\!=\!\Longrightarrow}\left(\mathrm{ops},S^{\prime}\right)}{v^{*},S\overset{\mathrm{lift}}{\Longrightarrow}\left\langle v^{*}=\mathtt{new}\left(\mathrm{type}\left(S\left(v^{*}\right)\right)\right)\right\rangle ::ops,S^{\prime}}}$
\end_inset
@@ -575,7 +575,7 @@
\begin_inset Text
\begin_layout Plain Layout
-\begin_inset Formula ${\displaystyle \frac{\left(S\left(v^{*}\right)_{L},S\setminus\left\{ v^{*}\mapsto S\left(v^{*}\right)\right\} \right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{L},S^{\prime}\right),\,\left(S\left(v^{*}\right)_{R},S^{\prime}\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{R},S^{\prime\prime}\right)}{v^{*},S\overset{\mathrm{liftfields}}{=\!=\!\Longrightarrow}\mathrm{ops}_{L}::ops_{R}::\left\langle \mathtt{set}\left(v^{*},L,S\left(v^{*}\right)_{L}\right),\,\mathtt{set}\left(v^{*},R,S\left(v^{*}\right)_{R}\right)\right\rangle ,S^{\prime}}}$
+\begin_inset Formula ${\displaystyle \frac{\left(S\left(v^{*}\right)_{L},S\setminus\left\{ v^{*}\mapsto S\left(v^{*}\right)\right\} \right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{L},S^{\prime}\right),\,\left(S\left(v^{*}\right)_{R},S^{\prime}\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{R},S^{\prime\prime}\right)}{v^{*},S\overset{\mathrm{liftfields}}{=\!=\!\Longrightarrow}\mathrm{ops}_{L}::ops_{R}::\left\langle \mathtt{set}\left(v^{*},L,S\left(v^{*}\right)_{L}\right),\,\mathtt{set}\left(v^{*},R,S\left(v^{*}\right)_{R}\right)\right\rangle ,S^{\prime\prime}}}$
\end_inset
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 22 13:56:29 2010
@@ -235,7 +235,7 @@
the translation to C, PyPy's tools can generate a tracing just-in-time compiler for the
language that the interpreter is implementing. This process is mostly
automatic; it only needs to be guided by the language implementer using a small number of
-source-code hints. Mostly-automatically generating a JIT compiler has many advantages
+source-code hints in the interpreter. Mostly-automatically generating a JIT compiler has many advantages
over writing one manually, an error-prone and tedious process.
By construction, the generated JIT has the same semantics as the interpreter.
Optimizations can be shared between different languages implemented with PyPy.
@@ -289,7 +289,7 @@
during tracing, because if
it isn't, the rest of the trace would not be valid.
-When generating machine code, every guard is be turned into a quick check to
+When generating machine code, every guard is turned into a quick check to
see whether the assumption still holds. When such a guard is hit during the
execution of the machine code and the assumption does not hold, the execution of
the machine code is stopped, and interpreter continues to run from that point
@@ -297,9 +297,9 @@
loop end condition also takes the form of a guard.
If one specific guard fails often enough, the tracing JIT will generate a new
-trace that starts exactly at the position of the failing guard. The existing
-assembler is patched to jump to the new trace when the guard fails
-\cite{andreas_gal_incremental_2006}. This approach guarantees that all the
+trace that starts exactly at the position of the failing guard
+\cite{andreas_gal_incremental_2006}. The existing assembler is patched to jump
+to the new trace when the guard fails. This approach guarantees that all the
hot paths in the program will eventually be traced and compiled into efficient
code.
@@ -377,7 +377,7 @@
implement the numeric tower needs two method calls per arithmetic operation,
which is costly due to the method dispatch.
-Let us now consider a simple interpreter function \lstinline{f} that uses the
+Let us now consider a simple ``interpreter'' function \lstinline{f} that uses the
object model (see the bottom of Figure~\ref{fig:objmodel}).
The loop in \lstinline{f} iterates \lstinline{y} times, and computes something in the process.
Simply running this function is slow, because there are lots of virtual method
@@ -460,7 +460,8 @@
corresponding to the stack level of the function that contains the traced
operation. The trace is in single-assignment form, meaning that each variable is
assigned a value exactly once. The arguments $p_0$ and $p_1$ of the loop correspond
-to the live variables \lstinline{y} and \lstinline{res} in the original function.
+to the live variables \lstinline{y} and \lstinline{res} in the while-loop of
+the original function.
The operations in the trace correspond to the operations in the RPython program
in Figure~\ref{fig:objmodel}:
@@ -473,11 +474,13 @@
(inlined) method call and is followed by the trace of the called method.
\item \lstinline{int_add} and \lstinline{int_gt} are integer addition and
comparison (``greater than''), respectively.
+ \item \lstinline{guard_true} checks that a boolean is true.
\end{itemize}
The method calls in the trace are always preceded by a \lstinline{guard_class}
operation, to check that the class of the receiver is the same as the one that
-was observed during tracing.\footnote{\lstinline{guard_class} performs a precise
+was observed during tracing.\footnote{\lstinline{guard_class} XXX lstinline too large in footnotes
+performs a precise
class check, not checking for subclasses.} These guards make the trace specific
to the situation where \lstinline{y} is really a \lstinline{BoxedInteger}. When
the trace is turned into machine code and afterwards executed with
@@ -575,16 +578,16 @@
The final trace after optimization can be seen in Figure~\ref{fig:step1} (the
line numbers are the lines of the unoptimized trace where the operation originates).
-To optimize the trace, it is traversed from beginning to end while an output
+To optimize the trace, it is traversed from beginning to end and an output
trace is produced. Every operation in the input trace is either
-removed or put into the output trace. Sometimes new operations need to be
+removed or copied into the output trace. Sometimes new operations need to be
produced as well. The optimizer can only remove operations that manipulate
objects that have been allocated within the trace, while all other operations are copied to the
output trace unchanged.
Looking at the example trace of Figure~\ref{fig:unopt-trace}, the operations
in lines 1--9 are manipulating objects which existed before the trace and that
-are passed in as arguments: therefore the optimizer just puts them into the
+are passed in as arguments: therefore the optimizer just copies them into the
output trace.
The following operations (lines 10--17) are more interesting:
@@ -608,11 +611,11 @@
stored in the fields of the allocated object come from.
In the snippet above, the two \lstinline{new} operations are removed and two
-static objects are constructed. The \lstinline{set} operations manipulate a
-static object, therefore they can be removed as well; their effect is
+static objects are constructed. The \lstinline{set} operations manipulate
+static objects, therefore they can be removed as well; their effect is
remembered in the static objects.
-The static object associated with $p_{5}$ would store the knowledge that it is a
+After the operations the static object associated with $p_{5}$ would store the knowledge that it is a
\lstinline{BoxedInteger} whose \lstinline{intval} field contains $i_{4}$; the
one associated with $p_{6}$ would store that it is a \lstinline{BoxedInteger}
whose \lstinline{intval} field contains the constant -100.
@@ -628,8 +631,8 @@
$i_{9}$ = int_add($i_{7}$, $i_{8}$)
\end{lstlisting}
-The \lstinline{guard_class} operations can be removed, since their argument is a
-static object with the matching type \lstinline{BoxedInteger}. The
+The \lstinline{guard_class} operations can be removed, since their arguments are
+static objects with the matching type \lstinline{BoxedInteger}. The
\lstinline{get} operations can be removed as well, because each of them reads a
field out of a static object. The results of the get operation are replaced with
what the static object stores in these fields: all the occurences of $i_{7}$ and $i_{8}$ in the trace are just
@@ -642,11 +645,11 @@
The rest of the trace from Figure~\ref{fig:unopt-trace} is optimized in a
similar vein. The operations in lines 27--35 produce two more static objects and
-are removed. Those in line 36--39 are just put into the output trace because they
+are removed. Those in line 36--39 are just copied into the output trace because they
manipulate objects that are allocated before the trace. Lines 40--42 are removed
-because they operate on a static object. Line 43 is put into the output trace.
+because they operate on a static object. Line 43 is copied into the output trace.
Lines 44--46 produce a new static object and are removed, lines 48--51 manipulate
-that static object and are removed as well. Lines 52--54 are put into the output
+that static object and are removed as well. Lines 52--54 are copied into the output
trace.
The last operation (line 55) is an interesting case. It is the \lstinline{jump}
@@ -759,16 +762,16 @@
In this section we want to give a formal description of the semantics of the
traces and of the optimizer and liken the optimization to partial evaluation.
-We focus on the operations for manipulating dynamically allocated objects,
+We focus on the operations for manipulating heap allocated objects,
as those are the only ones that are actually optimized. We also consider only
-objects with two fields in this section, generalizing to arbitrary many fields
+objects with two fields $L$ and $R$ in this section, generalizing to arbitrary many fields
is straightforward.
Traces are lists of operations. The operations considered here are
\lstinline{new}, \lstinline{get}, \lstinline{set} and \lstinline{guard_class}.
The values of all
variables are locations (\ie pointers). Locations are mapped to objects, which
-are represented by triples of a type $T$, and two locations that represent the
+are represented by triples $(T,l_1,l_2)$ of a type $T$, and two locations that represent the
fields of the object. When a new object is created, the fields are initialized
to null, but we require that they are initialized to a real
location before being read, otherwise the trace is malformed (this condition is
@@ -776,7 +779,7 @@
We use some abbreviations when dealing with object triples. To read the type of
an object, $\mathrm{type}((T,l_1,l_2))=T$ is used. Reading a field $F$ from an
-object is written $(T,l_1,l_2)_F$ which either returns $l_1$ if $F=L$ or $l_2$
+object is written $(T,l_1,l_2)_F$ which either is $l_1$ if $F=L$ or $l_2$
if $F=R$. To set field $F$ to a new location $l$, we use the notation
$(T,l_1,l_2)!_Fl$, which yields a new triple $(T,l,l_2)$ if $F=L$ or a new
triple $(T,l_1,l)$ if $F=R$.
@@ -827,8 +830,8 @@
\emph{guard} & ${\displaystyle \frac{E(v)\in\mathrm{dom}(S),\,\mathrm{type}(S(E(v)))=T}{\mathtt{guard\_class}(v,T),E,S\overset{\mathrm{opt}}{\Longrightarrow}\left\langle \,\right\rangle ,E,S}}$\tabularnewline[3em]
& ${\displaystyle \frac{E(v)\notin\mathrm{dom}(S)\vee\mathrm{type}(S(E(v)))\neq T,\,\left(E(v),S\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops},S^{\prime}\right)}{\mathtt{guard\_class}(v,T),E,S\overset{\mathrm{opt}}{\Longrightarrow}\mathrm{ops}::\left\langle \mathtt{guard\_class}(E\left(v\right),T)\right\rangle ,E,S^{\prime}}}$\tabularnewline[3em]
\emph{lifting} & ${\displaystyle \frac{v^{*}\notin\mathrm{dom}(S)}{v^{*},S\overset{\mathrm{lift}}{\Longrightarrow}\left\langle \,\right\rangle ,S}}$\tabularnewline[3em]
- & ${\displaystyle \frac{v^{*}\in\mathrm{dom}(S),\,\left(v^{*},S\right)\overset{\mathrm{liftfields}}{=\!=\!\Longrightarrow}\left(\mathrm{ops},S^{\prime}\right)}{v^{*},S\overset{\mathrm{lift}}{\Longrightarrow}\left\langle v^{*}=\mathtt{new}\left(T\right)\right\rangle ::ops,S^{\prime}}}$\tabularnewline[3em]
- & ${\displaystyle \frac{\left(S\left(v^{*}\right)_{L},S\setminus\left\{ v^{*}\mapsto S\left(v^{*}\right)\right\} \right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{L},S^{\prime}\right),\,\left(S\left(v^{*}\right)_{R},S^{\prime}\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{R},S^{\prime\prime}\right)}{v^{*},S\overset{\mathrm{liftfields}}{=\!=\!\Longrightarrow}\mathrm{ops}_{L}::ops_{R}::\left\langle \mathtt{set}\left(v^{*},L,S\left(v^{*}\right)_{L}\right),\,\mathtt{set}\left(v^{*},R,S\left(v^{*}\right)_{R}\right)\right\rangle ,S^{\prime}}}$\tabularnewline[3em]
+ & ${\displaystyle \frac{v^{*}\in\mathrm{dom}(S),\,\left(v^{*},S\right)\overset{\mathrm{liftfields}}{=\!=\!\Longrightarrow}\left(\mathrm{ops},S^{\prime}\right)}{v^{*},S\overset{\mathrm{lift}}{\Longrightarrow}\left\langle v^{*}=\mathtt{new}\left(\mathrm{type}\left(S\left(v^{*}\right)\right)\right)\right\rangle ::ops,S^{\prime}}}$\tabularnewline[3em]
+ & ${\displaystyle \frac{\left(S\left(v^{*}\right)_{L},S\setminus\left\{ v^{*}\mapsto S\left(v^{*}\right)\right\} \right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{L},S^{\prime}\right),\,\left(S\left(v^{*}\right)_{R},S^{\prime}\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{R},S^{\prime\prime}\right)}{v^{*},S\overset{\mathrm{liftfields}}{=\!=\!\Longrightarrow}\mathrm{ops}_{L}::ops_{R}::\left\langle \mathtt{set}\left(v^{*},L,S\left(v^{*}\right)_{L}\right),\,\mathtt{set}\left(v^{*},R,S\left(v^{*}\right)_{R}\right)\right\rangle ,S^{\prime\prime}}}$\tabularnewline[3em]
\end{tabular}
\end{center}
@@ -958,7 +961,7 @@
\subsection{Analysis of the Algorithm}
\label{sub:analysis}
-While we do not offer a formal proof of it, it should be relatively clear
+While we do not offer a formal proof of it, it can argue informally
that the algorithm presented above is sound: it works by delaying (and
often completely removing) some operations. The algorithm runs in a
single pass over the list of operations. We can check that although
@@ -1014,7 +1017,7 @@
To evaluate our allocation removal algorithm, we look at the effectiveness when
used in the generated tracing JIT of PyPy's Python interpreter. This interpreter
is a full implementation of Python 2.5 language semantics and is about 30'000
-lines of code.
+lines of RPython code.
The
benchmarks we used are small-to-medium Python programs, some synthetic
@@ -1058,7 +1061,7 @@
CPython 2.6.6\footnote{\texttt{http://python.org}}, which uses a bytecode-based
interpreter. Furthermore we compared against
Psyco\cite{rigo_representation-based_2004} 1.6,
-an extension to CPython which is a
+a (hand-written) extension module to CPython which is a
just-in-time compiler that produces machine code at run-time. It is not based
on traces. Finally, we used two versions of PyPy's Python interpreter (revision
77823 of SVN trunk\footnote{\texttt{http://codespeak.net/svn/pypy/trunk}}): one
@@ -1178,17 +1181,17 @@
loop to only allocate it once, instead of every iteration. No details are given
for this optimization. The fact that the object is still allocated and needs to
be written to means that only the allocations are optimized away, but not the
-reads and writes out of/into the object.
+reads out of and writes into the object.
SPUR, a tracing JIT for C\# seems to be able to remove allocations in a similar
way to the approach described here, as hinted at in the technical report
-\cite{michael_bebenita_spur:_2010}. However, no details for the approach and its
+\cite{bebenita_spur:_2010}. However, no details for the approach and its
implementation are given.
Psyco \cite{rigo_representation-based_2004} is a (non-tracing) JIT for Python
that implements a more ad-hoc version of the allocation removal described here.
Our static objects could be related to what are called \emph{virtual} objects
-in Psyco. It is a hand-written extension module for CPython. Historically,
+in Psyco. Historically,
PyPy's JIT can be seen as some successor of Psyco for a general context (one of
the authors of this paper is the author of Psyco).
From david at codespeak.net Fri Oct 22 13:59:45 2010
From: david at codespeak.net (david at codespeak.net)
Date: Fri, 22 Oct 2010 13:59:45 +0200 (CEST)
Subject: [pypy-svn] r78199 - in
pypy/branch/arm-backend/pypy/jit/backend/arm: . test
Message-ID: <20101022115945.7632C282BDC@codespeak.net>
Author: david
Date: Fri Oct 22 13:59:44 2010
New Revision: 78199
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
Log:
Add instr. to call coprocessor
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py Fri Oct 22 13:59:44 2010
@@ -1,6 +1,6 @@
from pypy.jit.backend.arm import conditions as cond
from pypy.jit.backend.arm import instructions
-
+# move table lookup out of generated functions
def define_load_store_func(name, table):
# XXX W and P bits are not encoded yet
n = (0x1 << 26
@@ -92,6 +92,19 @@
| reg_operation(rd, rn, rm, imm, s, shifttype))
return f
+def define_supervisor_and_coproc(name, table):
+ n = (0x3 << 26 | (table['op1'] & 0x3F) << 20 | (table['op'] & 0x1) << 4)
+ def f(self, coproc, opc1, rt, crn, crm, opc2=0, cond=cond.AL):
+ self.write32(n
+ | cond << 28
+ | (opc1 & 0x7) << 21
+ | (crn & 0xF) << 16
+ | (rt & 0xF) << 12
+ | (coproc & 0xF) << 8
+ | (opc2 & 0x7) << 5
+ | (crm & 0xF))
+ return f
+
def imm_operation(rt, rn, imm):
return ((rn & 0xFF) << 16
| (rt & 0xFF) << 12
@@ -119,3 +132,6 @@
for key, val in instructions.data_proc_imm.iteritems():
define_instruction(define_data_proc_imm, key, val, target)
+
+ for key, val in instructions.supervisor_and_coproc.iteritems():
+ define_instruction(define_supervisor_and_coproc, key, val, target)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py Fri Oct 22 13:59:44 2010
@@ -52,3 +52,7 @@
'BIC_ri': {'op': 0x1C, 'rncond':'', 'result':True, 'base':True},
'MVN_ri': {'op': 0x1E, 'rncond':'', 'result':True, 'base':False},
}
+
+supervisor_and_coproc = {
+ 'MCR': {'op1': 0x20, 'op': 1, 'rn':0, 'coproc':0},
+}
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Fri Oct 22 13:59:44 2010
@@ -114,6 +114,11 @@
self.cb.CMP(r.r3, 123)
self.assert_equal('CMP r3, #123')
+ def test_mcr(self):
+ self.cb.MCR(15, 0, r.r1, 7, 10,0)
+
+ self.assert_equal('MCR P15, 0, r1, c7, c10, 0')
+
class TestInstrCodeBuilderForGeneratedInstr(ASMTest):
def setup_method(self, ffuu_method):
From cfbolz at codespeak.net Fri Oct 22 14:02:38 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 22 Oct 2010 14:02:38 +0200 (CEST)
Subject: [pypy-svn] r78200 - pypy/extradoc/talk/pepm2011
Message-ID: <20101022120238.D4DA936E0CB@codespeak.net>
Author: cfbolz
Date: Fri Oct 22 14:02:37 2010
New Revision: 78200
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.bib
pypy/extradoc/talk/pepm2011/paper.tex
Log:
fix citation
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.bib
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.bib (original)
+++ pypy/extradoc/talk/pepm2011/paper.bib Fri Oct 22 14:02:37 2010
@@ -130,7 +130,7 @@
abstract = {In the section on ?challenging problems? in the proceedings from the first international workshop on partial evaluation and mixed computation {[BEJ88]} a question is stated: {?Can} {PE} be used to generate new specialized data types, in a way analogous to generating specialized functions?. Since then little has been done to address this problem. In {[Lau89],} new types are indeed generated, but they are all simpler versions of the types in the original program. It is, e.g. not possible to have types with more constructors than the types in the original program. I propose to alleviate this by means of constructor specialization. Constructors are specialized with respect to the static parts of their arguments, just like residual functions. I show how this is done and argue that it makes it possible to get good results from partial evaluation in cases where the traditional methods fail to produce satisfactory results. The discussion is centered around a small subset of Standard {ML,} but the idea applies equally well to other languages having user defined constructors, e.g. Haskell and Prolog.},
booktitle = {Proceedings of the 1993 {ACM} {SIGPLAN} symposium on Partial evaluation and semantics-based program manipulation},
publisher = {{ACM}},
- author = {Torben ? Mogensen},
+ author = {Torben Mogensen},
year = {1993},
pages = {22--32}
},
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 22 14:02:37 2010
@@ -765,10 +765,9 @@
We focus on the operations for manipulating heap allocated objects,
as those are the only ones that are actually optimized. We also consider only
objects with two fields $L$ and $R$ in this section, generalizing to arbitrary many fields
-is straightforward.
-
-Traces are lists of operations. The operations considered here are
+is straightforward. Traces are lists of operations. The operations considered here are
\lstinline{new}, \lstinline{get}, \lstinline{set} and \lstinline{guard_class}.
+
The values of all
variables are locations (\ie pointers). Locations are mapped to objects, which
are represented by triples $(T,l_1,l_2)$ of a type $T$, and two locations that represent the
@@ -814,6 +813,8 @@
continues without changing heap and environment. Otherwise, execution is
stopped.
+% subsection Operational Semantics of Traces (end)
+
\subsection{Optimizing Traces}
\label{sub:formalopt}
@@ -1210,7 +1211,7 @@
residual program. This is similar to what our optimization does for an
imperative language.
In functional programming this idea was introduced as
- constructor specialisation by Mogensen \cite{Mogensen:PEPM93}.
+ constructor specialisation by Mogensen \cite{mogensen_constructor_1993}.
%xxx:
%partially static data structures: kenichi asai's thesis?
From cfbolz at codespeak.net Fri Oct 22 14:06:46 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 22 Oct 2010 14:06:46 +0200 (CEST)
Subject: [pypy-svn] r78201 - pypy/extradoc/talk/pepm2011
Message-ID: <20101022120646.C5138282BDC@codespeak.net>
Author: cfbolz
Date: Fri Oct 22 14:06:45 2010
New Revision: 78201
Modified:
pypy/extradoc/talk/pepm2011/escape-tracing.pdf
pypy/extradoc/talk/pepm2011/paper.tex
Log:
kill XXX, remove line overflow
Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf
==============================================================================
Binary files. No diff available.
Modified: pypy/extradoc/talk/pepm2011/paper.tex
==============================================================================
--- pypy/extradoc/talk/pepm2011/paper.tex (original)
+++ pypy/extradoc/talk/pepm2011/paper.tex Fri Oct 22 14:06:45 2010
@@ -477,9 +477,9 @@
\item \lstinline{guard_true} checks that a boolean is true.
\end{itemize}
-The method calls in the trace are always preceded by a \lstinline{guard_class}
+Method calls in the trace are preceded by a \lstinline{guard_class}
operation, to check that the class of the receiver is the same as the one that
-was observed during tracing.\footnote{\lstinline{guard_class} XXX lstinline too large in footnotes
+was observed during tracing.\footnote{\lstinline{guard_class}
performs a precise
class check, not checking for subclasses.} These guards make the trace specific
to the situation where \lstinline{y} is really a \lstinline{BoxedInteger}. When
From afa at codespeak.net Fri Oct 22 14:12:02 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 22 Oct 2010 14:12:02 +0200 (CEST)
Subject: [pypy-svn] r78202 - pypy/trunk/pypy/translator/c/test
Message-ID: <20101022121202.4C92B36E0CB@codespeak.net>
Author: afa
Date: Fri Oct 22 14:12:00 2010
New Revision: 78202
Modified:
pypy/trunk/pypy/translator/c/test/test_typed.py
Log:
Show how variable calls to __exit__ confuse the annotator
Modified: pypy/trunk/pypy/translator/c/test/test_typed.py
==============================================================================
--- pypy/trunk/pypy/translator/c/test/test_typed.py (original)
+++ pypy/trunk/pypy/translator/c/test/test_typed.py Fri Oct 22 14:12:00 2010
@@ -838,15 +838,16 @@
state.append('raised')
state.append('release')
- def func():
+ def func(n):
try:
with C('hello') as c:
state.append(c.name)
- raise ValueError
+ if 1:
+ raise ValueError
except ValueError:
pass
return ', '.join(state)
- f = self.getcompiled(func, [])
- res = f()
+ f = self.getcompiled(func, [int])
+ res = f(1)
assert res == 'acquire, hello, raised, release'
From afa at codespeak.net Fri Oct 22 14:14:14 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 22 Oct 2010 14:14:14 +0200 (CEST)
Subject: [pypy-svn] r78203 - pypy/trunk/pypy/translator/c/test
Message-ID: <20101022121414.5160B282BDC@codespeak.net>
Author: afa
Date: Fri Oct 22 14:14:12 2010
New Revision: 78203
Modified:
pypy/trunk/pypy/translator/c/test/test_typed.py
Log:
The actual (failing) test
Modified: pypy/trunk/pypy/translator/c/test/test_typed.py
==============================================================================
--- pypy/trunk/pypy/translator/c/test/test_typed.py (original)
+++ pypy/trunk/pypy/translator/c/test/test_typed.py Fri Oct 22 14:14:12 2010
@@ -842,7 +842,7 @@
try:
with C('hello') as c:
state.append(c.name)
- if 1:
+ if n:
raise ValueError
except ValueError:
pass
From cfbolz at codespeak.net Fri Oct 22 14:24:57 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 22 Oct 2010 14:24:57 +0200 (CEST)
Subject: [pypy-svn] r78204 - pypy/extradoc/talk/pepm2011
Message-ID: <20101022122457.5EB3E282BDC@codespeak.net>
Author: cfbolz
Date: Fri Oct 22 14:24:56 2010
New Revision: 78204
Added:
pypy/extradoc/talk/pepm2011/bolz-allocation-removal.pdf (contents, props changed)
Log:
paper as submitted
Added: pypy/extradoc/talk/pepm2011/bolz-allocation-removal.pdf
==============================================================================
Binary file. No diff available.
From arigo at codespeak.net Fri Oct 22 14:28:41 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 22 Oct 2010 14:28:41 +0200 (CEST)
Subject: [pypy-svn] r78205 - pypy/build/bot2/pypybuildbot
Message-ID: <20101022122841.0CFE3282BDC@codespeak.net>
Author: arigo
Date: Fri Oct 22 14:28:40 2010
New Revision: 78205
Modified:
pypy/build/bot2/pypybuildbot/summary.py
Log:
Increase the size of the cache. Right now it needs 84 entries;
it used to be 60, so let's bump it to 150.
Modified: pypy/build/bot2/pypybuildbot/summary.py
==============================================================================
--- pypy/build/bot2/pypybuildbot/summary.py (original)
+++ pypy/build/bot2/pypybuildbot/summary.py Fri Oct 22 14:28:40 2010
@@ -295,7 +295,7 @@
N = 5
-outcome_set_cache = RevisionOutcomeSetCache(10*(N+1))
+outcome_set_cache = RevisionOutcomeSetCache(25*(N+1))
def colsizes(namekeys):
From afa at codespeak.net Fri Oct 22 14:49:31 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 22 Oct 2010 14:49:31 +0200 (CEST)
Subject: [pypy-svn] r78207 - in pypy/trunk/pypy: interpreter objspace/flow
Message-ID: <20101022124931.F235C282BDC@codespeak.net>
Author: afa
Date: Fri Oct 22 14:49:30 2010
New Revision: 78207
Modified:
pypy/trunk/pypy/interpreter/pyopcode.py
pypy/trunk/pypy/objspace/flow/flowcontext.py
Log:
Fix the test: in RPython, the 'exc_type' parameter of __exit__ is not usable.
'exc_value' probably contains everything you need.
Modified: pypy/trunk/pypy/interpreter/pyopcode.py
==============================================================================
--- pypy/trunk/pypy/interpreter/pyopcode.py (original)
+++ pypy/trunk/pypy/interpreter/pyopcode.py Fri Oct 22 14:49:30 2010
@@ -164,6 +164,9 @@
next_instr = block.handle(self, unroller)
return next_instr
+ def call_contextmanager_exit_function(self, w_func, w_typ, w_val, w_tb):
+ return self.space.call_function(w_func, w_typ, w_val, w_tb)
+
@jit.unroll_safe
def dispatch_bytecode(self, co_code, next_instr, ec):
space = self.space
@@ -886,7 +889,7 @@
self.pushvalue(self.space.w_None)
self.pushvalue(self.space.w_None)
elif self.pycode.magic >= 0xa0df28c:
- # Implemementation since 2.5a0: 62092 (changed WITH_CLEANUP opcode)
+ # Implementation since 2.5a0: 62092 (changed WITH_CLEANUP opcode)
w_exitfunc = self.popvalue()
w_unroller = self.peekvalue(2)
else:
@@ -896,18 +899,20 @@
if isinstance(unroller, SApplicationException):
operr = unroller.operr
w_traceback = self.space.wrap(operr.application_traceback)
- w_result = self.space.call_function(w_exitfunc,
- operr.w_type,
- operr.get_w_value(self.space),
- w_traceback)
+ w_result = self.call_contextmanager_exit_function(
+ w_exitfunc,
+ operr.w_type,
+ operr.get_w_value(self.space),
+ w_traceback)
if self.space.is_true(w_result):
# __exit__() returned True -> Swallow the exception.
self.settopvalue(self.space.w_None, 2)
else:
- self.space.call_function(w_exitfunc,
- self.space.w_None,
- self.space.w_None,
- self.space.w_None)
+ self.call_contextmanager_exit_function(
+ w_exitfunc,
+ self.space.w_None,
+ self.space.w_None,
+ self.space.w_None)
@jit.unroll_safe
def call_function(self, oparg, w_star=None, w_starstar=None):
Modified: pypy/trunk/pypy/objspace/flow/flowcontext.py
==============================================================================
--- pypy/trunk/pypy/objspace/flow/flowcontext.py (original)
+++ pypy/trunk/pypy/objspace/flow/flowcontext.py Fri Oct 22 14:49:30 2010
@@ -502,3 +502,10 @@
raise operr
return pyframe.PyFrame.handle_operation_error(self, ec, operr,
*args, **kwds)
+
+ def call_contextmanager_exit_function(self, w_func, w_typ, w_val, w_tb):
+ # The annotator won't allow to merge exception types with None.
+ # XXX return an object which will break translation when it is used
+ w_typ = self.space.w_None
+ return self.space.call_function(w_func, w_typ, w_val, w_tb)
+
From arigo at codespeak.net Fri Oct 22 14:55:00 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 22 Oct 2010 14:55:00 +0200 (CEST)
Subject: [pypy-svn] r78208 - in pypy/trunk/pypy: jit/codewriter
jit/metainterp jit/metainterp/test rlib
Message-ID: <20101022125500.41E85282BDC@codespeak.net>
Author: arigo
Date: Fri Oct 22 14:54:58 2010
New Revision: 78208
Modified:
pypy/trunk/pypy/jit/codewriter/jtransform.py
pypy/trunk/pypy/jit/metainterp/test/test_jitdriver.py
pypy/trunk/pypy/jit/metainterp/warmspot.py
pypy/trunk/pypy/rlib/jit.py
Log:
Add the ability to deactivate some jitdrivers.
Far better solution than somehow hacking in the source to
remove the calls to jitdriver.jit_merge_point().
Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/trunk/pypy/jit/codewriter/jtransform.py Fri Oct 22 14:54:58 2010
@@ -831,6 +831,8 @@
def rewrite_op_jit_marker(self, op):
key = op.args[0].value
jitdriver = op.args[1].value
+ if not jitdriver.active:
+ return []
return getattr(self, 'handle_jit_marker__%s' % key)(op, jitdriver)
def handle_jit_marker__jit_merge_point(self, op, jitdriver):
Modified: pypy/trunk/pypy/jit/metainterp/test/test_jitdriver.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/test_jitdriver.py (original)
+++ pypy/trunk/pypy/jit/metainterp/test/test_jitdriver.py Fri Oct 22 14:54:58 2010
@@ -72,6 +72,33 @@
# we expect no loop at all for 'loop1': it should always be inlined
self.check_tree_loop_count(2) # 1 x loop, 1 x enter bridge
+ def test_inactive_jitdriver(self):
+ myjitdriver1 = JitDriver(greens=[], reds=['n', 'm'],
+ get_printable_location = getloc1)
+ myjitdriver2 = JitDriver(greens=['g'], reds=['r'],
+ get_printable_location = getloc2)
+ #
+ myjitdriver1.active = False # <===
+ #
+ def loop1(n, m):
+ while n > 0:
+ myjitdriver1.can_enter_jit(n=n, m=m)
+ myjitdriver1.jit_merge_point(n=n, m=m)
+ n -= m
+ return n
+ #
+ def loop2(g, r):
+ while r > 0:
+ myjitdriver2.can_enter_jit(g=g, r=r)
+ myjitdriver2.jit_merge_point(g=g, r=r)
+ r += loop1(r, g) + (-1)
+ return r
+ #
+ res = self.meta_interp(loop2, [4, 40], repeat=7, inline=True)
+ assert res == loop2(4, 40)
+ # we expect no int_sub, but a residual call
+ self.check_loops(int_sub=0, call=1)
+
class TestLLtype(MultipleJitDriversTests, LLJitMixin):
pass
Modified: pypy/trunk/pypy/jit/metainterp/warmspot.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/warmspot.py (original)
+++ pypy/trunk/pypy/jit/metainterp/warmspot.py Fri Oct 22 14:54:58 2010
@@ -110,7 +110,8 @@
for i in range(len(block.operations)):
op = block.operations[i]
if (op.opname == 'jit_marker' and
- op.args[0].value == marker_name):
+ op.args[0].value == marker_name and
+ op.args[1].value.active): # the jitdriver
results.append((graph, block, i))
return results
Modified: pypy/trunk/pypy/rlib/jit.py
==============================================================================
--- pypy/trunk/pypy/rlib/jit.py (original)
+++ pypy/trunk/pypy/rlib/jit.py Fri Oct 22 14:54:58 2010
@@ -274,8 +274,9 @@
several independent JITting interpreters in it.
"""
+ active = True # if set to False, this JitDriver is ignored
virtualizables = []
-
+
def __init__(self, greens=None, reds=None, virtualizables=None,
get_jitcell_at=None, set_jitcell_at=None,
get_printable_location=None, confirm_enter_jit=None,
From arigo at codespeak.net Fri Oct 22 14:55:52 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 22 Oct 2010 14:55:52 +0200 (CEST)
Subject: [pypy-svn] r78209 - pypy/trunk/pypy/rlib/rsre
Message-ID: <20101022125552.B7AF536E0CB@codespeak.net>
Author: arigo
Date: Fri Oct 22 14:55:51 2010
New Revision: 78209
Modified:
pypy/trunk/pypy/rlib/rsre/rsre_jit.py
Log:
Temporarily(?) deactive the rsre jit drivers,
to see the benchmark run results tomorrow.
Modified: pypy/trunk/pypy/rlib/rsre/rsre_jit.py
==============================================================================
--- pypy/trunk/pypy/rlib/rsre/rsre_jit.py (original)
+++ pypy/trunk/pypy/rlib/rsre/rsre_jit.py Fri Oct 22 14:55:51 2010
@@ -2,6 +2,7 @@
class RSreJitDriver(JitDriver):
+ active = False # XXX temporary?
def __init__(self, name, debugprint, **kwds):
JitDriver.__init__(self, **kwds)
From arigo at codespeak.net Fri Oct 22 15:47:24 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 22 Oct 2010 15:47:24 +0200 (CEST)
Subject: [pypy-svn] r78210 - pypy/build/bot2/pypybuildbot
Message-ID: <20101022134724.505F3282BDC@codespeak.net>
Author: arigo
Date: Fri Oct 22 15:47:22 2010
New Revision: 78210
Modified:
pypy/build/bot2/pypybuildbot/master.py
pypy/build/bot2/pypybuildbot/summary.py
Log:
Raaaaaaah. Grrr grr gr. Killed the lines in master.py.
Modified: pypy/build/bot2/pypybuildbot/master.py
==============================================================================
--- pypy/build/bot2/pypybuildbot/master.py (original)
+++ pypy/build/bot2/pypybuildbot/master.py Fri Oct 22 15:47:22 2010
@@ -317,14 +317,3 @@
'buildbotURL': 'http://codespeak.net:%d/'%(httpPortNumber),
'projectURL': 'http://codespeak.net/pypy/',
'projectName': 'PyPy'}
-
-CACHESIZE = 80 # cache size for build outcomes
-
-estimated = (sum([len(_sched.listBuilderNames())
- for _sched in BuildmasterConfig['schedulers']]) * 6)
-
-if estimated > CACHESIZE:
- raise ValueError("master.py CACHESIZE (%d) is too small considered"
- " a builder*scheduler combinations based estimate (%d)"
- % (CACHESIZE, estimated))
-summary.outcome_set_cache.cachesize = CACHESIZE
Modified: pypy/build/bot2/pypybuildbot/summary.py
==============================================================================
--- pypy/build/bot2/pypybuildbot/summary.py (original)
+++ pypy/build/bot2/pypybuildbot/summary.py Fri Oct 22 15:47:22 2010
@@ -295,7 +295,7 @@
N = 5
-outcome_set_cache = RevisionOutcomeSetCache(25*(N+1))
+outcome_set_cache = RevisionOutcomeSetCache(32*(N+1))
def colsizes(namekeys):
From arigo at codespeak.net Fri Oct 22 16:05:04 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 22 Oct 2010 16:05:04 +0200 (CEST)
Subject: [pypy-svn] r78211 - pypy/trunk/pypy/module/array
Message-ID: <20101022140504.6BA6436E35D@codespeak.net>
Author: arigo
Date: Fri Oct 22 16:05:03 2010
New Revision: 78211
Modified:
pypy/trunk/pypy/module/array/interp_array.py
Log:
Revert r78195: CPython returns longs for arrays of 'I' even on 64-bit machines,
and I don't want to consider the deep implications of that.
Modified: pypy/trunk/pypy/module/array/interp_array.py
==============================================================================
--- pypy/trunk/pypy/module/array/interp_array.py (original)
+++ pypy/trunk/pypy/module/array/interp_array.py Fri Oct 22 16:05:03 2010
@@ -323,7 +323,7 @@
idx, stop, step = space.decode_index(w_idx, self.len)
assert step == 0
item = self.buffer[idx]
- if mytype.canoverflow: # if it fits in a Signed
+ if mytype.typecode in 'bBhHil':
item = rffi.cast(lltype.Signed, item)
elif mytype.typecode == 'f':
item = float(item)
From arigo at codespeak.net Fri Oct 22 16:09:58 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 22 Oct 2010 16:09:58 +0200 (CEST)
Subject: [pypy-svn] r78212 - pypy/trunk/pypy/module/pypyjit/test
Message-ID: <20101022140958.CE98D282BDC@codespeak.net>
Author: arigo
Date: Fri Oct 22 16:09:57 2010
New Revision: 78212
Modified:
pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
Log:
70 operations look fine on 64-bit machines.
Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Fri Oct 22 16:09:57 2010
@@ -836,7 +836,7 @@
def test_array_intimg(self):
# XXX this test is way too imprecise in what it is actually testing
# it should count the number of guards instead
- for tc, maxops in zip('ilILd', (67, 67, 69, 69, 61)):
+ for tc, maxops in zip('ilILd', (67, 67, 70, 70, 61)):
print
print '='*65
print '='*20, 'running test for tc=%r' % (tc,), '='*20
From arigo at codespeak.net Fri Oct 22 16:13:08 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 22 Oct 2010 16:13:08 +0200 (CEST)
Subject: [pypy-svn] r78213 - pypy/branch/cleanup-dict-impl/pypy/objspace/std
Message-ID: <20101022141308.E225E282BDC@codespeak.net>
Author: arigo
Date: Fri Oct 22 16:13:07 2010
New Revision: 78213
Modified:
pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py
Log:
Simplify the code for getting the correct class.
It should also help the JIT get rid of a few operations.
Modified: pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py Fri Oct 22 16:13:07 2010
@@ -1,4 +1,4 @@
-from pypy.rlib import jit, objectmodel
+from pypy.rlib import jit, objectmodel, debug
from pypy.interpreter.baseobjspace import W_Root
from pypy.objspace.std.dictmultiobject import W_DictMultiObject
@@ -10,6 +10,9 @@
# attribute shapes
NUM_DIGITS = 4
+NUM_DIGITS_POW2 = 1 << NUM_DIGITS
+# note: we use "x * NUM_DIGITS_POW2" instead of "x << NUM_DIGITS" because
+# we want to propagate knowledge that the result cannot be negative
class AbstractAttribute(object):
_immutable_fields_ = ['w_cls']
@@ -69,8 +72,10 @@
attr = self._get_new_attr(selector[0], selector[1])
oldattr = obj._get_mapdict_map()
if not jit.we_are_jitted():
- oldattr._size_estimate += attr.size_estimate() - oldattr.size_estimate()
- assert oldattr.size_estimate() >= oldattr.length()
+ size_est = (oldattr._size_estimate + attr.size_estimate()
+ - oldattr.size_estimate())
+ assert size_est >= (oldattr.length() * NUM_DIGITS_POW2)
+ oldattr._size_estimate = size_est
if attr.length() > obj._mapdict_storage_length():
# note that attr.size_estimate() is always at least attr.length()
new_storage = [None] * attr.size_estimate()
@@ -188,7 +193,7 @@
self.selector = selector
self.position = back.length()
self.back = back
- self._size_estimate = self.length() << NUM_DIGITS
+ self._size_estimate = self.length() * NUM_DIGITS_POW2
def _copy_attr(self, obj, new_obj):
w_value = self.read(obj, self.selector)
@@ -384,12 +389,11 @@
map = w_type.terminator
classes = memo_get_subclass_of_correct_size(space, cls)
size = map.size_estimate()
- if not size:
- size = 1
- try:
- return classes[size - 1]
- except IndexError:
- return classes[-1]
+ debug.check_nonneg(size)
+ if size < len(classes):
+ return classes[size]
+ else:
+ return classes[len(classes)-1]
get_subclass_of_correct_size._annspecialcase_ = "specialize:arg(1)"
NUM_SUBCLASSES = 10 # XXX tweak this number
@@ -403,6 +407,7 @@
result = []
for i in range(1, NUM_SUBCLASSES+1):
result.append(_make_subclass_size_n(supercls, i))
+ result.insert(0, result[0]) # for 0, use the same class as for 1
_subclass_cache[key] = result
return result
memo_get_subclass_of_correct_size._annspecialcase_ = "specialize:memo"
From afa at codespeak.net Fri Oct 22 16:31:52 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 22 Oct 2010 16:31:52 +0200 (CEST)
Subject: [pypy-svn] r78214 - in pypy/trunk/pypy: objspace/flow
translator/c/test
Message-ID: <20101022143152.6739B282BDC@codespeak.net>
Author: afa
Date: Fri Oct 22 16:31:50 2010
New Revision: 78214
Modified:
pypy/trunk/pypy/objspace/flow/flowcontext.py
pypy/trunk/pypy/translator/c/test/test_typed.py
Log:
Replace the exc_typ argument of the __exit__ method with an object
that is likely to break annotation if used in any way
(except maybe with the 'is' operator)
Modified: pypy/trunk/pypy/objspace/flow/flowcontext.py
==============================================================================
--- pypy/trunk/pypy/objspace/flow/flowcontext.py (original)
+++ pypy/trunk/pypy/objspace/flow/flowcontext.py Fri Oct 22 16:31:50 2010
@@ -504,8 +504,10 @@
*args, **kwds)
def call_contextmanager_exit_function(self, w_func, w_typ, w_val, w_tb):
- # The annotator won't allow to merge exception types with None.
- # XXX return an object which will break translation when it is used
- w_typ = self.space.w_None
+ if w_typ is not self.space.w_None:
+ # The annotator won't allow to merge exception types with None.
+ # Replace it with an object which will break translation when used
+ # (except maybe with 'exc_typ is None')
+ w_typ = self.space.wrap(self.space)
return self.space.call_function(w_func, w_typ, w_val, w_tb)
Modified: pypy/trunk/pypy/translator/c/test/test_typed.py
==============================================================================
--- pypy/trunk/pypy/translator/c/test/test_typed.py (original)
+++ pypy/trunk/pypy/translator/c/test/test_typed.py Fri Oct 22 16:31:50 2010
@@ -833,21 +833,32 @@
def __enter__(self):
state.append('acquire')
return self
- def __exit__(self, *args):
- if args[1] is not None:
+ def __exit__(self, typ, value, tb):
+ if typ is not None:
+ if value is None:
+ raise RuntimeError('test failed')
state.append('raised')
+ else:
+ if value is not None:
+ raise RuntimeError('test failed')
state.append('release')
def func(n):
+ del state[:]
try:
with C('hello') as c:
state.append(c.name)
- if n:
+ if n == 1:
raise ValueError
- except ValueError:
+ elif n == 2:
+ raise TypeError
+ except (ValueError, TypeError):
pass
return ', '.join(state)
f = self.getcompiled(func, [int])
+ res = f(0)
+ assert res == 'acquire, hello, release'
res = f(1)
assert res == 'acquire, hello, raised, release'
-
+ res = f(2)
+ assert res == 'acquire, hello, raised, release'
From arigo at codespeak.net Fri Oct 22 16:50:18 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 22 Oct 2010 16:50:18 +0200 (CEST)
Subject: [pypy-svn] r78215 - pypy/trunk/pypy/translator/goal
Message-ID: <20101022145018.0B000282BDC@codespeak.net>
Author: arigo
Date: Fri Oct 22 16:50:17 2010
New Revision: 78215
Modified:
pypy/trunk/pypy/translator/goal/app_main.py
Log:
Bah. There is also the bug that we cannot combine options,
like "pypy-c -iE"
Modified: pypy/trunk/pypy/translator/goal/app_main.py
==============================================================================
--- pypy/trunk/pypy/translator/goal/app_main.py (original)
+++ pypy/trunk/pypy/translator/goal/app_main.py Fri Oct 22 16:50:17 2010
@@ -255,7 +255,7 @@
break
elif arg == '-u':
unbuffered = True
- elif arg == '-O':
+ elif arg == '-O' or arg == '-OO':
pass
elif arg == '--version' or arg == '-V':
print "Python", sys.version
From arigo at codespeak.net Fri Oct 22 17:12:00 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 22 Oct 2010 17:12:00 +0200 (CEST)
Subject: [pypy-svn] r78216 - pypy/trunk/pypy/translator/goal
Message-ID: <20101022151200.F0C2F36E363@codespeak.net>
Author: arigo
Date: Fri Oct 22 17:11:59 2010
New Revision: 78216
Modified:
pypy/trunk/pypy/translator/goal/app_main.py
Log:
Print an error message here.
Modified: pypy/trunk/pypy/translator/goal/app_main.py
==============================================================================
--- pypy/trunk/pypy/translator/goal/app_main.py (original)
+++ pypy/trunk/pypy/translator/goal/app_main.py Fri Oct 22 17:11:59 2010
@@ -386,8 +386,9 @@
if python_startup:
try:
startup = open(python_startup).read()
- except IOError:
- pass
+ except IOError, e:
+ print >> sys.stderr, "Could not open PYTHONSTARTUP"
+ print >> sys.stderr, "IOError:", e
else:
def run_it():
co_python_startup = compile(startup,
From arigo at codespeak.net Fri Oct 22 17:13:03 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 22 Oct 2010 17:13:03 +0200 (CEST)
Subject: [pypy-svn] r78217 - pypy/trunk/pypy/translator/goal
Message-ID: <20101022151303.7F644282BDC@codespeak.net>
Author: arigo
Date: Fri Oct 22 17:13:02 2010
New Revision: 78217
Modified:
pypy/trunk/pypy/translator/goal/app_main.py
Log:
Use the proper gc-friendly style of explicitly close()ing the file.
Modified: pypy/trunk/pypy/translator/goal/app_main.py
==============================================================================
--- pypy/trunk/pypy/translator/goal/app_main.py (original)
+++ pypy/trunk/pypy/translator/goal/app_main.py Fri Oct 22 17:13:02 2010
@@ -385,7 +385,9 @@
python_startup = os.getenv('PYTHONSTARTUP')
if python_startup:
try:
- startup = open(python_startup).read()
+ f = open(python_startup)
+ startup = f.read()
+ f.close()
except IOError, e:
print >> sys.stderr, "Could not open PYTHONSTARTUP"
print >> sys.stderr, "IOError:", e
From fijal at codespeak.net Fri Oct 22 18:19:57 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Fri, 22 Oct 2010 18:19:57 +0200 (CEST)
Subject: [pypy-svn] r78218 - in pypy/trunk/pypy: jit/backend/test
jit/backend/x86/test jit/metainterp/test jit/tool
module/pypyjit/test
Message-ID: <20101022161957.01D29282BDC@codespeak.net>
Author: fijal
Date: Fri Oct 22 18:19:52 2010
New Revision: 78218
Added:
pypy/trunk/pypy/jit/tool/oparser.py
- copied, changed from r78182, pypy/trunk/pypy/jit/metainterp/test/oparser.py
Removed:
pypy/trunk/pypy/jit/metainterp/test/oparser.py
Modified:
pypy/trunk/pypy/jit/backend/test/runner_test.py
pypy/trunk/pypy/jit/backend/x86/test/test_gc_integration.py
pypy/trunk/pypy/jit/backend/x86/test/test_regalloc.py
pypy/trunk/pypy/jit/backend/x86/test/test_runner.py
pypy/trunk/pypy/jit/metainterp/test/test_compile.py
pypy/trunk/pypy/jit/metainterp/test/test_logger.py
pypy/trunk/pypy/jit/metainterp/test/test_oparser.py
pypy/trunk/pypy/jit/metainterp/test/test_optimizefindnode.py
pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py
pypy/trunk/pypy/jit/tool/loopviewer.py
pypy/trunk/pypy/jit/tool/showstats.py
pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
Log:
Refactor a bit oparser. Also move it to tool instead of metainterp/test
(it's rather widely used by now)
Modified: pypy/trunk/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/trunk/pypy/jit/backend/test/runner_test.py Fri Oct 22 18:19:52 2010
@@ -9,7 +9,7 @@
ConstObj, BoxFloat, ConstFloat)
from pypy.jit.metainterp.resoperation import ResOperation, rop
from pypy.jit.metainterp.typesystem import deref
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi, rclass
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.annlowlevel import llhelper
Modified: pypy/trunk/pypy/jit/backend/x86/test/test_gc_integration.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/test/test_gc_integration.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/test/test_gc_integration.py Fri Oct 22 18:19:52 2010
@@ -12,7 +12,7 @@
from pypy.jit.backend.detect_cpu import getcpuclass
from pypy.jit.backend.x86.regalloc import RegAlloc
from pypy.jit.backend.x86.arch import WORD, FRAME_FIXED_SIZE
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.rpython.lltypesystem import lltype, llmemory, rffi
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.lltypesystem import rclass, rstr
Modified: pypy/trunk/pypy/jit/backend/x86/test/test_regalloc.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/test/test_regalloc.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/test/test_regalloc.py Fri Oct 22 18:19:52 2010
@@ -11,7 +11,7 @@
from pypy.jit.backend.x86.regalloc import RegAlloc, X86RegisterManager,\
FloatConstants, is_comparison_or_ovf_op
from pypy.jit.backend.x86.arch import IS_X86_32, IS_X86_64
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.rpython.lltypesystem import lltype, llmemory, rffi
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.lltypesystem import rclass, rstr
Modified: pypy/trunk/pypy/jit/backend/x86/test/test_runner.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/test/test_runner.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/test/test_runner.py Fri Oct 22 18:19:52 2010
@@ -10,7 +10,7 @@
from pypy.jit.metainterp.resoperation import rop
from pypy.jit.metainterp.executor import execute
from pypy.jit.backend.test.runner_test import LLtypeBackendTest
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.tool.udir import udir
import ctypes
import sys
Modified: pypy/trunk/pypy/jit/metainterp/test/test_compile.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/test_compile.py (original)
+++ pypy/trunk/pypy/jit/metainterp/test/test_compile.py Fri Oct 22 18:19:52 2010
@@ -6,8 +6,8 @@
from pypy.jit.metainterp.compile import ResumeGuardCountersInt
from pypy.jit.metainterp.compile import compile_tmp_callback
from pypy.jit.metainterp import optimize, jitprof, typesystem, compile
-from pypy.jit.metainterp.test.oparser import parse
from pypy.jit.metainterp.test.test_optimizefindnode import LLtypeMixin
+from pypy.jit.tool.oparser import parse
def test_insert_loop_token():
Modified: pypy/trunk/pypy/jit/metainterp/test/test_logger.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/test_logger.py (original)
+++ pypy/trunk/pypy/jit/metainterp/test/test_logger.py Fri Oct 22 18:19:52 2010
@@ -1,6 +1,6 @@
import sys
from pypy.rlib import debug
-from pypy.jit.metainterp.test.oparser import pure_parse
+from pypy.jit.tool.oparser import pure_parse
from pypy.jit.metainterp import logger
from pypy.jit.metainterp.typesystem import llhelper
from StringIO import StringIO
Modified: pypy/trunk/pypy/jit/metainterp/test/test_oparser.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/test_oparser.py (original)
+++ pypy/trunk/pypy/jit/metainterp/test/test_oparser.py Fri Oct 22 18:19:52 2010
@@ -1,7 +1,7 @@
from pypy.rpython.lltypesystem import lltype, llmemory
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.jit.metainterp.resoperation import rop
from pypy.jit.metainterp.history import AbstractDescr, BoxInt, LoopToken,\
BoxFloat
Modified: pypy/trunk/pypy/jit/metainterp/test/test_optimizefindnode.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/test_optimizefindnode.py (original)
+++ pypy/trunk/pypy/jit/metainterp/test/test_optimizefindnode.py Fri Oct 22 18:19:52 2010
@@ -18,7 +18,7 @@
from pypy.jit.metainterp.specnode import ConstantSpecNode
from pypy.jit.codewriter.effectinfo import EffectInfo
from pypy.jit.codewriter.heaptracker import register_known_gctype, adr2int
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
def test_sort_descrs():
class PseudoDescr(AbstractDescr):
Modified: pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py Fri Oct 22 18:19:52 2010
@@ -12,7 +12,7 @@
from pypy.jit.metainterp.jitprof import EmptyProfiler
from pypy.jit.metainterp import executor, compile, resume, history
from pypy.jit.metainterp.resoperation import rop, opname, ResOperation
-from pypy.jit.metainterp.test.oparser import pure_parse
+from pypy.jit.tool.oparser import pure_parse
##class FakeFrame(object):
## parent_resumedata_snapshot = None
Modified: pypy/trunk/pypy/jit/tool/loopviewer.py
==============================================================================
--- pypy/trunk/pypy/jit/tool/loopviewer.py (original)
+++ pypy/trunk/pypy/jit/tool/loopviewer.py Fri Oct 22 18:19:52 2010
@@ -9,7 +9,7 @@
import optparse
from pprint import pprint
from pypy.tool import logparser
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.jit.metainterp.history import ConstInt
from pypy.rpython.lltypesystem import llmemory, lltype
Copied: pypy/trunk/pypy/jit/tool/oparser.py (from r78182, pypy/trunk/pypy/jit/metainterp/test/oparser.py)
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/oparser.py (original)
+++ pypy/trunk/pypy/jit/tool/oparser.py Fri Oct 22 18:19:52 2010
@@ -184,24 +184,9 @@
self.newvar(arg)
return self.vars[arg]
- def parse_op(self, line):
- num = line.find('(')
- if num == -1:
- raise ParseError("invalid line: %s" % line)
- opname = line[:num]
- try:
- opnum = getattr(rop, opname.upper())
- except AttributeError:
- if opname == 'escape':
- opnum = ESCAPE_OP.OPNUM
- else:
- raise ParseError("unknown op: %s" % opname)
- endnum = line.rfind(')')
- if endnum == -1:
- raise ParseError("invalid line: %s" % line)
+ def parse_args(self, opname, argspec):
args = []
descr = None
- argspec = line[num + 1:endnum]
if argspec.strip():
if opname == 'debug_merge_point':
allargs = [argspec]
@@ -219,6 +204,24 @@
args.append(self.getvar(arg))
except KeyError:
raise ParseError("Unknown var: %s" % arg)
+ return args, descr
+
+ def parse_op(self, line):
+ num = line.find('(')
+ if num == -1:
+ raise ParseError("invalid line: %s" % line)
+ opname = line[:num]
+ try:
+ opnum = getattr(rop, opname.upper())
+ except AttributeError:
+ if opname == 'escape':
+ opnum = ESCAPE_OP.OPNUM
+ else:
+ raise ParseError("unknown op: %s" % opname)
+ endnum = line.rfind(')')
+ if endnum == -1:
+ raise ParseError("invalid line: %s" % line)
+ args, descr = self.parse_args(opname, line[num + 1:endnum])
if rop._GUARD_FIRST <= opnum <= rop._GUARD_LAST:
i = line.find('[', endnum) + 1
j = line.find(']', i)
Modified: pypy/trunk/pypy/jit/tool/showstats.py
==============================================================================
--- pypy/trunk/pypy/jit/tool/showstats.py (original)
+++ pypy/trunk/pypy/jit/tool/showstats.py Fri Oct 22 18:19:52 2010
@@ -4,7 +4,7 @@
import autopath
import sys, py
from pypy.tool import logparser
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.jit.metainterp.resoperation import rop
from pypy.rpython.lltypesystem import lltype, llmemory
Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Fri Oct 22 18:19:52 2010
@@ -131,7 +131,7 @@
return result
def parse_loops(self, opslogfile):
- from pypy.jit.metainterp.test.oparser import parse
+ from pypy.jit.tool.oparser import parse
from pypy.tool import logparser
assert opslogfile.check()
log = logparser.parse_log_file(str(opslogfile))
From dan at codespeak.net Fri Oct 22 19:32:41 2010
From: dan at codespeak.net (dan at codespeak.net)
Date: Fri, 22 Oct 2010 19:32:41 +0200 (CEST)
Subject: [pypy-svn] r78223 - pypy/branch/ootype-virtualrefs/pypy/rlib/test
Message-ID: <20101022173241.5D9BA282BDC@codespeak.net>
Author: dan
Date: Fri Oct 22 19:32:39 2010
New Revision: 78223
Modified:
pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py
Log:
Properly test interp() results.
Modified: pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py
==============================================================================
--- pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py (original)
+++ pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py Fri Oct 22 19:32:39 2010
@@ -6,6 +6,7 @@
from pypy.annotation.annrpython import RPythonAnnotator
from pypy.rpython.test.test_llinterp import interpret
from pypy.rpython.lltypesystem.rclass import OBJECTPTR
+from pypy.rpython.ootypesystem.rclass import OBJECT
from pypy.rpython.lltypesystem import lltype
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
@@ -85,7 +86,7 @@
def f():
return virtual_ref(X())
x = self.interpret(f, [])
- assert self.is_of_instance_type(x)
+ assert lltype.typeOf(x) == self.OBJECTTYPE
def test_rtype_2(self):
def f():
@@ -95,7 +96,7 @@
virtual_ref_finish(x2)
return x2
x = self.interpret(f, [])
- assert lltype.castable(OBJECTPTR, lltype.typeOf(x)) > 0
+ assert self.castable(self.OBJECTTYPE, x)
def test_rtype_3(self):
def f(n):
@@ -104,7 +105,7 @@
else:
return non_virtual_ref(Z())
x = self.interpret(f, [-5])
- assert self.is_of_instance_type(x)
+ assert lltype.typeOf(x) == self.OBJECTTYPE
def test_rtype_4(self):
def f(n):
@@ -113,11 +114,15 @@
else:
return vref_None
x = self.interpret(f, [-5])
- assert self.is_of_instance_type(x)
+ assert lltype.typeOf(x) == self.OBJECTTYPE
assert not x
class TestLLtype(BaseTestVRef, LLRtypeMixin):
- pass
+ OBJECTTYPE = OBJECTPTR
+ def castable(self, TO, var):
+ return lltype.castable(TO, lltype.typeOf(var)) > 0
class TestOOtype(BaseTestVRef, OORtypeMixin):
- pass
+ OBJECTTYPE = OBJECT
+ def castable(self, TO, var):
+ return ootype.isSubclass(TO, lltype.typeOf(var))
From fijal at codespeak.net Fri Oct 22 19:46:58 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Fri, 22 Oct 2010 19:46:58 +0200 (CEST)
Subject: [pypy-svn] r78224 - in pypy/trunk/pypy/rpython: lltypesystem test
Message-ID: <20101022174658.DA3F5282BDC@codespeak.net>
Author: fijal
Date: Fri Oct 22 19:46:57 2010
New Revision: 78224
Modified:
pypy/trunk/pypy/rpython/lltypesystem/lltype.py
pypy/trunk/pypy/rpython/test/test_llinterp.py
Log:
A test and a fix
Modified: pypy/trunk/pypy/rpython/lltypesystem/lltype.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/lltype.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/lltype.py Fri Oct 22 19:46:57 2010
@@ -1871,7 +1871,10 @@
def _make_scoped_allocator(T):
class ScopedAlloc:
def __init__(self, n=None, zero=False):
- self.buf = malloc(T, n=n, flavor='raw', zero=zero)
+ if n is None:
+ self.buf = malloc(T, flavor='raw', zero=zero)
+ else:
+ self.buf = malloc(T, n, flavor='raw', zero=zero)
def __enter__(self):
return self.buf
Modified: pypy/trunk/pypy/rpython/test/test_llinterp.py
==============================================================================
--- pypy/trunk/pypy/rpython/test/test_llinterp.py (original)
+++ pypy/trunk/pypy/rpython/test/test_llinterp.py Fri Oct 22 19:46:57 2010
@@ -645,3 +645,16 @@
res = interpret(f, [])
assert hlstr(res) == 'acquire, use, raised, release'
+
+def test_scoped_allocator():
+ from pypy.rpython.lltypesystem.lltype import scoped_alloc, Array, Signed
+ T = Array(Signed)
+
+ def f():
+ x = 0
+ with scoped_alloc(T, 1) as array:
+ array[0] = -42
+ x = array[0]
+ assert x == -42
+
+ res = interpret(f, [])
From afa at codespeak.net Fri Oct 22 20:14:22 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 22 Oct 2010 20:14:22 +0200 (CEST)
Subject: [pypy-svn] r78225 - in pypy/trunk/pypy: module/_winreg
rpython/lltypesystem
Message-ID: <20101022181422.00056282BDC@codespeak.net>
Author: afa
Date: Fri Oct 22 20:14:21 2010
New Revision: 78225
Modified:
pypy/trunk/pypy/module/_winreg/interp_winreg.py
pypy/trunk/pypy/rpython/lltypesystem/rffi.py
Log:
Start using the "with" statement to manage temporary allocations.
Only the _winreg module for now... it's now 60 lines shorter!
Modified: pypy/trunk/pypy/module/_winreg/interp_winreg.py
==============================================================================
--- pypy/trunk/pypy/module/_winreg/interp_winreg.py (original)
+++ pypy/trunk/pypy/module/_winreg/interp_winreg.py Fri Oct 22 20:14:21 2010
@@ -212,13 +212,10 @@
subkey = None
else:
subkey = space.str_w(w_subkey)
- dataptr = rffi.str2charp(value)
- try:
+ with rffi.scoped_str2charp(value) as dataptr:
ret = rwinreg.RegSetValue(hkey, subkey, rwinreg.REG_SZ, dataptr, len(value))
- finally:
- rffi.free_charp(dataptr)
- if ret != 0:
- raiseWindowsError(space, ret, 'RegSetValue')
+ if ret != 0:
+ raiseWindowsError(space, ret, 'RegSetValue')
SetValue.unwrap_spec = [ObjSpace, W_Root, W_Root, int, str]
def QueryValue(space, w_hkey, w_subkey):
@@ -237,23 +234,15 @@
subkey = None
else:
subkey = space.str_w(w_subkey)
- bufsize_p = lltype.malloc(rwin32.PLONG.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.PLONG.TO, 1) as bufsize_p:
ret = rwinreg.RegQueryValue(hkey, subkey, None, bufsize_p)
if ret != 0:
raiseWindowsError(space, ret, 'RegQueryValue')
- buf = lltype.malloc(rffi.CCHARP.TO, bufsize_p[0], flavor='raw')
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, bufsize_p[0]) as buf:
ret = rwinreg.RegQueryValue(hkey, subkey, buf, bufsize_p)
if ret != 0:
raiseWindowsError(space, ret, 'RegQueryValue')
return space.wrap(rffi.charp2strn(buf, bufsize_p[0] - 1))
- finally:
- lltype.free(buf, flavor='raw')
- finally:
- lltype.free(bufsize_p, flavor='raw')
- if ret != 0:
- raiseWindowsError(space, ret, 'RegQueryValue')
QueryValue.unwrap_spec = [ObjSpace, W_Root, W_Root]
def convert_to_regdata(space, w_value, typ):
@@ -412,16 +401,14 @@
value_name is a string indicating the value to query"""
hkey = hkey_w(w_hkey, space)
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
- retDataSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retDataSize:
ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword, null_dword,
None, retDataSize)
if ret != 0:
raiseWindowsError(space, ret, 'RegQueryValueEx')
- databuf = lltype.malloc(rffi.CCHARP.TO, retDataSize[0], flavor='raw')
- try:
- retType = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+
+ with lltype.scoped_alloc(rffi.CCHARP.TO, retDataSize[0]) as databuf:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType:
ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword,
retType, databuf, retDataSize)
@@ -432,12 +419,6 @@
retDataSize[0], retType[0]),
space.wrap(retType[0]),
])
- finally:
- lltype.free(retType, flavor='raw')
- finally:
- lltype.free(databuf, flavor='raw')
- finally:
- lltype.free(retDataSize, flavor='raw')
QueryValueEx.unwrap_spec = [ObjSpace, W_Root, str]
@@ -454,14 +435,11 @@
The return value is the handle of the opened key.
If the function fails, an exception is raised."""
hkey = hkey_w(w_hkey, space)
- rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey:
ret = rwinreg.RegCreateKey(hkey, subkey, rethkey)
if ret != 0:
raiseWindowsError(space, ret, 'CreateKey')
return space.wrap(W_HKEY(rethkey[0]))
- finally:
- lltype.free(rethkey, flavor='raw')
CreateKey.unwrap_spec = [ObjSpace, W_Root, str]
def DeleteKey(space, w_hkey, subkey):
@@ -504,14 +482,11 @@
The result is a new handle to the specified key
If the function fails, an EnvironmentError exception is raised."""
hkey = hkey_w(w_hkey, space)
- rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey:
ret = rwinreg.RegOpenKeyEx(hkey, subkey, res, sam, rethkey)
if ret != 0:
raiseWindowsError(space, ret, 'RegOpenKeyEx')
return space.wrap(W_HKEY(rethkey[0]))
- finally:
- lltype.free(rethkey, flavor='raw')
OpenKey.unwrap_spec = [ObjSpace, W_Root, str, int, rffi.r_uint]
def EnumValue(space, w_hkey, index):
@@ -531,10 +506,8 @@
hkey = hkey_w(w_hkey, space)
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
- retValueSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
- retDataSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retValueSize:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retDataSize:
ret = rwinreg.RegQueryInfoKey(
hkey, None, null_dword, null_dword,
null_dword, null_dword, null_dword,
@@ -546,14 +519,9 @@
retValueSize[0] += 1
retDataSize[0] += 1
- valuebuf = lltype.malloc(rffi.CCHARP.TO, retValueSize[0],
- flavor='raw')
- try:
- databuf = lltype.malloc(rffi.CCHARP.TO, retDataSize[0],
- flavor='raw')
- try:
- retType = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, retValueSize[0]) as valuebuf:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, retDataSize[0]) as databuf:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType:
ret = rwinreg.RegEnumValue(
hkey, index, valuebuf, retValueSize,
null_dword, retType, databuf, retDataSize)
@@ -566,16 +534,6 @@
retDataSize[0], retType[0]),
space.wrap(retType[0]),
])
- finally:
- lltype.free(retType, flavor='raw')
- finally:
- lltype.free(databuf, flavor='raw')
- finally:
- lltype.free(valuebuf, flavor='raw')
- finally:
- lltype.free(retDataSize, flavor='raw')
- finally:
- lltype.free(retValueSize, flavor='raw')
EnumValue.unwrap_spec = [ObjSpace, W_Root, int]
@@ -592,10 +550,8 @@
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
# max key name length is 255
- buf = lltype.malloc(rffi.CCHARP.TO, 256, flavor='raw')
- try:
- retValueSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, 256) as buf:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retValueSize:
retValueSize[0] = 256 # includes NULL terminator
ret = rwinreg.RegEnumKeyEx(hkey, index, buf, retValueSize,
null_dword, None, null_dword,
@@ -603,10 +559,6 @@
if ret != 0:
raiseWindowsError(space, ret, 'RegEnumKeyEx')
return space.wrap(rffi.charp2str(buf))
- finally:
- lltype.free(retValueSize, flavor='raw')
- finally:
- lltype.free(buf, flavor='raw')
EnumKey.unwrap_spec = [ObjSpace, W_Root, int]
@@ -621,12 +573,9 @@
A long integer that identifies when the key was last modified (if available)
as 100's of nanoseconds since Jan 1, 1600."""
hkey = hkey_w(w_hkey, space)
- nSubKeys = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
- nValues = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
- ft = lltype.malloc(rwin32.PFILETIME.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as nSubKeys:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as nValues:
+ with lltype.scoped_alloc(rwin32.PFILETIME.TO, 1) as ft:
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
ret = rwinreg.RegQueryInfoKey(
hkey, None, null_dword, null_dword,
@@ -640,12 +589,6 @@
return space.newtuple([space.wrap(nSubKeys[0]),
space.wrap(nValues[0]),
space.wrap(l)])
- finally:
- lltype.free(ft, flavor='raw')
- finally:
- lltype.free(nValues, flavor='raw')
- finally:
- lltype.free(nSubKeys, flavor='raw')
QueryInfoKey.unwrap_spec = [ObjSpace, W_Root]
def str_or_None_w(space, w_obj):
@@ -666,12 +609,9 @@
If the function fails, an EnvironmentError exception is raised."""
machine = str_or_None_w(space, w_machine)
hkey = hkey_w(w_hkey, space)
- rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey:
ret = rwinreg.RegConnectRegistry(machine, hkey, rethkey)
if ret != 0:
raiseWindowsError(space, ret, 'RegConnectRegistry')
return space.wrap(W_HKEY(rethkey[0]))
- finally:
- lltype.free(rethkey, flavor='raw')
ConnectRegistry.unwrap_spec = [ObjSpace, W_Root, W_Root]
Modified: pypy/trunk/pypy/rpython/lltypesystem/rffi.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/rffi.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/rffi.py Fri Oct 22 20:14:21 2010
@@ -933,3 +933,11 @@
"""
return cast(lltype.Signed, getattr(pdst, fieldname))
getintfield._annspecialcase_ = 'specialize:ll_and_arg(1)'
+
+class scoped_str2charp:
+ def __init__(self, value):
+ self.buf = str2charp(value)
+ def __enter__(self):
+ return self.buf
+ def __exit__(self, *args):
+ free_charp(self.buf)
From afa at codespeak.net Fri Oct 22 22:30:52 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 22 Oct 2010 22:30:52 +0200 (CEST)
Subject: [pypy-svn] r78226 - pypy/trunk/pypy/module/bz2
Message-ID: <20101022203052.EDE5F282BDC@codespeak.net>
Author: afa
Date: Fri Oct 22 22:30:49 2010
New Revision: 78226
Modified:
pypy/trunk/pypy/module/bz2/interp_bz2.py
Log:
Use the "with" memory manager in bz2 module
Modified: pypy/trunk/pypy/module/bz2/interp_bz2.py
==============================================================================
--- pypy/trunk/pypy/module/bz2/interp_bz2.py (original)
+++ pypy/trunk/pypy/module/bz2/interp_bz2.py Fri Oct 22 22:30:49 2010
@@ -225,6 +225,11 @@
if self.current_size > 0:
rffi.keep_buffer_alive_until_here(self.raw_buf, self.gc_buf)
+ def __enter__(self):
+ return self
+ def __exit__(self, *args):
+ self.free()
+
# ____________________________________________________________
#
# Make the BZ2File type by internally inheriting from W_File.
@@ -531,33 +536,30 @@
if not self.running:
raise OperationError(self.space.w_ValueError,
self.space.wrap("this object was already flushed"))
-
- out = OutBuffer(self.bzs)
+
in_bufsize = datasize
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(datasize):
- in_buf[i] = data[i]
- try:
-
- self.bzs.c_next_in = in_buf
- rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize)
+ with OutBuffer(self.bzs) as out:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
- while True:
- bzerror = BZ2_bzCompress(self.bzs, BZ_RUN)
- if bzerror != BZ_RUN_OK:
- _catch_bz2_error(self.space, bzerror)
+ for i in range(datasize):
+ in_buf[i] = data[i]
- if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
- break
- elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
+ self.bzs.c_next_in = in_buf
+ rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize)
- res = out.make_result_string()
- return self.space.wrap(res)
- finally:
- lltype.free(in_buf, flavor='raw')
- out.free()
+ while True:
+ bzerror = BZ2_bzCompress(self.bzs, BZ_RUN)
+ if bzerror != BZ_RUN_OK:
+ _catch_bz2_error(self.space, bzerror)
+
+ if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
+ break
+ elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
+
+ res = out.make_result_string()
+ return self.space.wrap(res)
compress.unwrap_spec = ['self', 'bufferstr']
@@ -566,9 +568,8 @@
raise OperationError(self.space.w_ValueError,
self.space.wrap("this object was already flushed"))
self.running = False
-
- out = OutBuffer(self.bzs)
- try:
+
+ with OutBuffer(self.bzs) as out:
while True:
bzerror = BZ2_bzCompress(self.bzs, BZ_FINISH)
if bzerror == BZ_STREAM_END:
@@ -581,8 +582,6 @@
res = out.make_result_string()
return self.space.wrap(res)
- finally:
- out.free()
flush.unwrap_spec = ['self']
W_BZ2Compressor.typedef = TypeDef("BZ2Compressor",
@@ -641,38 +640,37 @@
if not self.running:
raise OperationError(self.space.w_EOFError,
self.space.wrap("end of stream was already found"))
-
+
in_bufsize = len(data)
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(in_bufsize):
- in_buf[i] = data[i]
- out = OutBuffer(self.bzs)
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
+ for i in range(in_bufsize):
+ in_buf[i] = data[i]
self.bzs.c_next_in = in_buf
rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize)
- while True:
- bzerror = BZ2_bzDecompress(self.bzs)
- if bzerror == BZ_STREAM_END:
- if rffi.getintfield(self.bzs, 'c_avail_in') != 0:
- unused = [self.bzs.c_next_in[i] for i in range(rffi.getintfield(self.bzs, 'c_avail_in'))]
- self.unused_data = "".join(unused)
- self.running = False
- break
- if bzerror != BZ_OK:
- _catch_bz2_error(self.space, bzerror)
-
- if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
- break
- elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
+ with OutBuffer(self.bzs) as out:
+ while True:
+ bzerror = BZ2_bzDecompress(self.bzs)
+ if bzerror == BZ_STREAM_END:
+ if rffi.getintfield(self.bzs, 'c_avail_in') != 0:
+ unused = [self.bzs.c_next_in[i]
+ for i in range(
+ rffi.getintfield(self.bzs,
+ 'c_avail_in'))]
+ self.unused_data = "".join(unused)
+ self.running = False
+ break
+ if bzerror != BZ_OK:
+ _catch_bz2_error(self.space, bzerror)
+
+ if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
+ break
+ elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
- res = out.make_result_string()
- return self.space.wrap(res)
- finally:
- lltype.free(in_buf, flavor='raw')
- out.free()
+ res = out.make_result_string()
+ return self.space.wrap(res)
decompress.unwrap_spec = ['self', 'bufferstr']
@@ -695,43 +693,39 @@
if compresslevel < 1 or compresslevel > 9:
raise OperationError(space.w_ValueError,
space.wrap("compresslevel must be between 1 and 9"))
-
- bzs = lltype.malloc(bz_stream.TO, flavor='raw', zero=True)
- in_bufsize = len(data)
- # conforming to bz2 manual, this is large enough to fit compressed
- # data in one shot. We will check it later anyway.
- out = OutBuffer(bzs, in_bufsize + (in_bufsize / 100 + 1) + 600)
-
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(in_bufsize):
- in_buf[i] = data[i]
-
- try:
- bzs.c_next_in = in_buf
- rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
- bzerror = BZ2_bzCompressInit(bzs, compresslevel, 0, 0)
- if bzerror != BZ_OK:
- _catch_bz2_error(space, bzerror)
+ with lltype.scoped_alloc(bz_stream.TO, zero=True) as bzs:
+ in_bufsize = len(data)
+
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
+ for i in range(in_bufsize):
+ in_buf[i] = data[i]
+ bzs.c_next_in = in_buf
+ rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
+
+ # conforming to bz2 manual, this is large enough to fit compressed
+ # data in one shot. We will check it later anyway.
+ with OutBuffer(bzs,
+ in_bufsize + (in_bufsize / 100 + 1) + 600) as out:
+
+ bzerror = BZ2_bzCompressInit(bzs, compresslevel, 0, 0)
+ if bzerror != BZ_OK:
+ _catch_bz2_error(space, bzerror)
- while True:
- bzerror = BZ2_bzCompress(bzs, BZ_FINISH)
- if bzerror == BZ_STREAM_END:
- break
- elif bzerror != BZ_FINISH_OK:
+ while True:
+ bzerror = BZ2_bzCompress(bzs, BZ_FINISH)
+ if bzerror == BZ_STREAM_END:
+ break
+ elif bzerror != BZ_FINISH_OK:
+ BZ2_bzCompressEnd(bzs)
+ _catch_bz2_error(space, bzerror)
+
+ if rffi.getintfield(bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
+
+ res = out.make_result_string()
BZ2_bzCompressEnd(bzs)
- _catch_bz2_error(space, bzerror)
-
- if rffi.getintfield(bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
-
- res = out.make_result_string()
- BZ2_bzCompressEnd(bzs)
- return space.wrap(res)
- finally:
- lltype.free(bzs, flavor='raw')
- lltype.free(in_buf, flavor='raw')
- out.free()
+ return space.wrap(res)
compress.unwrap_spec = [ObjSpace, 'bufferstr', int]
def decompress(space, data):
@@ -744,40 +738,34 @@
if in_bufsize == 0:
return space.wrap("")
- bzs = lltype.malloc(bz_stream.TO, flavor='raw', zero=True)
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(in_bufsize):
- in_buf[i] = data[i]
-
- out = OutBuffer(bzs)
- try:
- bzs.c_next_in = in_buf
- rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
-
- bzerror = BZ2_bzDecompressInit(bzs, 0, 0)
- if bzerror != BZ_OK:
- _catch_bz2_error(space, bzerror)
-
- while True:
- bzerror = BZ2_bzDecompress(bzs)
- if bzerror == BZ_STREAM_END:
- break
- if bzerror != BZ_OK:
- BZ2_bzDecompressEnd(bzs)
- _catch_bz2_error(space, bzerror)
-
- if rffi.getintfield(bzs, 'c_avail_in') == 0:
+ with lltype.scoped_alloc(bz_stream.TO, zero=True) as bzs:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
+ for i in range(in_bufsize):
+ in_buf[i] = data[i]
+ bzs.c_next_in = in_buf
+ rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
+
+ with OutBuffer(bzs) as out:
+ bzerror = BZ2_bzDecompressInit(bzs, 0, 0)
+ if bzerror != BZ_OK:
+ _catch_bz2_error(space, bzerror)
+
+ while True:
+ bzerror = BZ2_bzDecompress(bzs)
+ if bzerror == BZ_STREAM_END:
+ break
+ if bzerror != BZ_OK:
+ BZ2_bzDecompressEnd(bzs)
+ _catch_bz2_error(space, bzerror)
+
+ if rffi.getintfield(bzs, 'c_avail_in') == 0:
+ BZ2_bzDecompressEnd(bzs)
+ raise OperationError(space.w_ValueError, space.wrap(
+ "couldn't find end of stream"))
+ elif rffi.getintfield(bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
+
+ res = out.make_result_string()
BZ2_bzDecompressEnd(bzs)
- raise OperationError(space.w_ValueError,
- space.wrap("couldn't find end of stream"))
- elif rffi.getintfield(bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
-
- res = out.make_result_string()
- BZ2_bzDecompressEnd(bzs)
- return space.wrap(res)
- finally:
- lltype.free(bzs, flavor='raw')
- lltype.free(in_buf, flavor='raw')
- out.free()
+ return space.wrap(res)
decompress.unwrap_spec = [ObjSpace, 'bufferstr']
From afa at codespeak.net Fri Oct 22 22:32:10 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 22 Oct 2010 22:32:10 +0200 (CEST)
Subject: [pypy-svn] r78227 -
pypy/branch/fast-forward/pypy/module/exceptions/test
Message-ID: <20101022203210.D90AB282BDC@codespeak.net>
Author: afa
Date: Fri Oct 22 22:32:09 2010
New Revision: 78227
Modified:
pypy/branch/fast-forward/pypy/module/exceptions/test/test_exc.py
Log:
A failing test that shows that the __doc__ attribute is broken.
Modified: pypy/branch/fast-forward/pypy/module/exceptions/test/test_exc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/exceptions/test/test_exc.py (original)
+++ pypy/branch/fast-forward/pypy/module/exceptions/test/test_exc.py Fri Oct 22 22:32:09 2010
@@ -227,6 +227,7 @@
if isinstance(e, type) and issubclass(e, exceptions.BaseException):
assert e.__doc__, e
assert e.__module__ == 'exceptions', e
+ assert 'run-time' in RuntimeError.__doc__
def test_reduce(self):
from exceptions import LookupError, EnvironmentError
From afa at codespeak.net Fri Oct 22 23:09:54 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 22 Oct 2010 23:09:54 +0200 (CEST)
Subject: [pypy-svn] r78228 - in pypy/branch/fast-forward: dotviewer
lib-python lib_pypy/_ctypes pypy pypy/annotation
pypy/annotation/test pypy/config pypy/doc pypy/doc/config
pypy/interpreter pypy/interpreter/pyparser
pypy/jit/backend/llgraph pypy/jit/backend/llsupport
pypy/jit/backend/llsupport/test pypy/jit/backend/test
pypy/jit/backend/x86 pypy/jit/backend/x86/test
pypy/jit/codewriter pypy/jit/codewriter/test
pypy/jit/metainterp pypy/jit/metainterp/optimizeopt
pypy/jit/metainterp/test pypy/jit/tl pypy/jit/tool
pypy/module/__builtin__ pypy/module/__builtin__/test
pypy/module/_ffi pypy/module/_ffi/test pypy/module/_rawffi
pypy/module/_rawffi/test pypy/module/_socket/test
pypy/module/_sre pypy/module/_weakref pypy/module/_winreg
pypy/module/array pypy/module/bz2 pypy/module/cpyext
pypy/module/cpyext/test pypy/module/gc pypy/module/imp/test
pypy/module/pypyjit pypy/module/pypyjit/test
pypy/module/signal pypy/module/sys
pypy/module/test_lib_pypy/ctypes_tests pypy/module/thread
pypy/objspace/flow pypy/objspace/flow/test pypy/objspace/std
pypy/objspace/std/test pypy/rlib pypy/rlib/rsre
pypy/rlib/rsre/test pypy/rlib/test pypy/rpython
pypy/rpython/lltypesystem pypy/rpython/lltypesystem/test
pypy/rpython/memory pypy/rpython/memory/gc
pypy/rpython/memory/gctransform pypy/rpython/memory/test
pypy/rpython/module pypy/rpython/test pypy/tool
pypy/tool/test pypy/translator pypy/translator/c
pypy/translator/c/gcc pypy/translator/c/gcc/test/elf
pypy/translator/c/gcc/test/elf64 pypy/translator/c/src
pypy/translator/c/test pypy/translator/goal pypy/translator/jvm/test
pypy/translator/oosupport/test_template pypy/translator/platform
Message-ID: <20101022210954.7F7A1282BDC@codespeak.net>
Author: afa
Date: Fri Oct 22 23:09:43 2010
New Revision: 78228
Added:
pypy/branch/fast-forward/pypy/doc/config/objspace.std.withmapdict.txt
- copied unchanged from r78227, pypy/trunk/pypy/doc/config/objspace.std.withmapdict.txt
pypy/branch/fast-forward/pypy/doc/config/objspace.std.withstrbuf.txt
- copied unchanged from r78227, pypy/trunk/pypy/doc/config/objspace.std.withstrbuf.txt
pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._ffi.txt
- copied unchanged from r78227, pypy/trunk/pypy/doc/config/objspace.usemodules._ffi.txt
pypy/branch/fast-forward/pypy/jit/backend/llsupport/ffisupport.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/backend/llsupport/ffisupport.py
pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_ffisupport.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/backend/llsupport/test/test_ffisupport.py
pypy/branch/fast-forward/pypy/jit/metainterp/greenfield.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/metainterp/greenfield.py
pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/fficall.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/metainterp/optimizeopt/fficall.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_fficall.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/metainterp/test/test_fficall.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_greenfield.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/metainterp/test/test_greenfield.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizefficall.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/metainterp/test/test_optimizefficall.py
pypy/branch/fast-forward/pypy/jit/tool/oparser.py
- copied unchanged from r78227, pypy/trunk/pypy/jit/tool/oparser.py
pypy/branch/fast-forward/pypy/module/_ffi/
- copied from r78227, pypy/trunk/pypy/module/_ffi/
pypy/branch/fast-forward/pypy/module/_ffi/__init__.py
- copied unchanged from r78227, pypy/trunk/pypy/module/_ffi/__init__.py
pypy/branch/fast-forward/pypy/module/_ffi/interp_ffi.py
- copied unchanged from r78227, pypy/trunk/pypy/module/_ffi/interp_ffi.py
pypy/branch/fast-forward/pypy/module/_ffi/test/
- copied from r78227, pypy/trunk/pypy/module/_ffi/test/
pypy/branch/fast-forward/pypy/module/_ffi/test/__init__.py
- copied unchanged from r78227, pypy/trunk/pypy/module/_ffi/test/__init__.py
pypy/branch/fast-forward/pypy/module/_ffi/test/test__ffi.py
- copied unchanged from r78227, pypy/trunk/pypy/module/_ffi/test/test__ffi.py
pypy/branch/fast-forward/pypy/objspace/std/mapdict.py
- copied unchanged from r78227, pypy/trunk/pypy/objspace/std/mapdict.py
pypy/branch/fast-forward/pypy/objspace/std/strbufobject.py
- copied unchanged from r78227, pypy/trunk/pypy/objspace/std/strbufobject.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_mapdict.py
- copied unchanged from r78227, pypy/trunk/pypy/objspace/std/test/test_mapdict.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_strbufobject.py
- copied unchanged from r78227, pypy/trunk/pypy/objspace/std/test/test_strbufobject.py
pypy/branch/fast-forward/pypy/rlib/clibffi.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/clibffi.py
pypy/branch/fast-forward/pypy/rlib/libffi.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/libffi.py
pypy/branch/fast-forward/pypy/rlib/rerased.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/rerased.py
pypy/branch/fast-forward/pypy/rlib/rsre/rsre_jit.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/rsre/rsre_jit.py
pypy/branch/fast-forward/pypy/rlib/rsre/test/conftest.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/rsre/test/conftest.py
pypy/branch/fast-forward/pypy/rlib/rsre/test/test_zjit.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/rsre/test/test_zjit.py
pypy/branch/fast-forward/pypy/rlib/test/test_clibffi.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/test/test_clibffi.py
pypy/branch/fast-forward/pypy/rlib/test/test_libffi.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/test/test_libffi.py
pypy/branch/fast-forward/pypy/rlib/test/test_rerased.py
- copied unchanged from r78227, pypy/trunk/pypy/rlib/test/test_rerased.py
pypy/branch/fast-forward/pypy/tool/leakfinder.py
- copied unchanged from r78227, pypy/trunk/pypy/tool/leakfinder.py
pypy/branch/fast-forward/pypy/tool/test/test_leakfinder.py
- copied unchanged from r78227, pypy/trunk/pypy/tool/test/test_leakfinder.py
pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf/track10.s
- copied unchanged from r78227, pypy/trunk/pypy/translator/c/gcc/test/elf/track10.s
pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf/track11.s
- copied unchanged from r78227, pypy/trunk/pypy/translator/c/gcc/test/elf/track11.s
pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf64/track_loadconst.s
- copied unchanged from r78227, pypy/trunk/pypy/translator/c/gcc/test/elf64/track_loadconst.s
pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf64/track_rpyassertfailed.s
- copied unchanged from r78227, pypy/trunk/pypy/translator/c/gcc/test/elf64/track_rpyassertfailed.s
pypy/branch/fast-forward/pypy/translator/c/src/debug_alloc.h
- copied unchanged from r78227, pypy/trunk/pypy/translator/c/src/debug_alloc.h
Removed:
pypy/branch/fast-forward/pypy/jit/metainterp/test/oparser.py
pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_find.py
pypy/branch/fast-forward/pypy/rpython/rspecialcase.py
pypy/branch/fast-forward/pypy/rpython/test/test_rspecialcase.py
Modified:
pypy/branch/fast-forward/dotviewer/drawgraph.py
pypy/branch/fast-forward/lib-python/conftest.py
pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py
pypy/branch/fast-forward/lib_pypy/_ctypes/union.py
pypy/branch/fast-forward/pypy/ (props changed)
pypy/branch/fast-forward/pypy/annotation/builtin.py
pypy/branch/fast-forward/pypy/annotation/model.py
pypy/branch/fast-forward/pypy/annotation/policy.py
pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py
pypy/branch/fast-forward/pypy/annotation/test/test_model.py
pypy/branch/fast-forward/pypy/config/pypyoption.py
pypy/branch/fast-forward/pypy/conftest.py
pypy/branch/fast-forward/pypy/doc/docindex.txt
pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py
pypy/branch/fast-forward/pypy/interpreter/pycode.py
pypy/branch/fast-forward/pypy/interpreter/pyopcode.py
pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py
pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py
pypy/branch/fast-forward/pypy/interpreter/typedef.py
pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py
pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py
pypy/branch/fast-forward/pypy/jit/backend/llsupport/descr.py
pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py
pypy/branch/fast-forward/pypy/jit/backend/llsupport/llmodel.py
pypy/branch/fast-forward/pypy/jit/backend/llsupport/symbolic.py
pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_descr.py
pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py
pypy/branch/fast-forward/pypy/jit/backend/test/test_ll_random.py
pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py
pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py
pypy/branch/fast-forward/pypy/jit/backend/x86/regloc.py
pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py
pypy/branch/fast-forward/pypy/jit/backend/x86/rx86.py
pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_assembler.py
pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_gc_integration.py
pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_regalloc.py
pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py
pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_string.py
pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_zrpy_gc.py
pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py
pypy/branch/fast-forward/pypy/jit/codewriter/call.py
pypy/branch/fast-forward/pypy/jit/codewriter/codewriter.py
pypy/branch/fast-forward/pypy/jit/codewriter/effectinfo.py
pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py
pypy/branch/fast-forward/pypy/jit/codewriter/support.py
pypy/branch/fast-forward/pypy/jit/codewriter/test/test_codewriter.py
pypy/branch/fast-forward/pypy/jit/codewriter/test/test_effectinfo.py
pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py
pypy/branch/fast-forward/pypy/jit/metainterp/blackhole.py
pypy/branch/fast-forward/pypy/jit/metainterp/compile.py
pypy/branch/fast-forward/pypy/jit/metainterp/executor.py
pypy/branch/fast-forward/pypy/jit/metainterp/graphpage.py
pypy/branch/fast-forward/pypy/jit/metainterp/history.py
pypy/branch/fast-forward/pypy/jit/metainterp/jitdriver.py
pypy/branch/fast-forward/pypy/jit/metainterp/optimize_nopspec.py (contents, props changed)
pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/__init__.py
pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/intbounds.py
pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/rewrite.py
pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/string.py
pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/virtualize.py
pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py
pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py
pypy/branch/fast-forward/pypy/jit/metainterp/resume.py
pypy/branch/fast-forward/pypy/jit/metainterp/simple_optimize.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_compile.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_jitdriver.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_loop_nopspec.py (props changed)
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_oparser.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizefindnode.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_string.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_virtualref.py
pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmspot.py
pypy/branch/fast-forward/pypy/jit/metainterp/virtualref.py
pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py
pypy/branch/fast-forward/pypy/jit/metainterp/warmstate.py
pypy/branch/fast-forward/pypy/jit/tl/jittest.py (props changed)
pypy/branch/fast-forward/pypy/jit/tl/pypyjit.py
pypy/branch/fast-forward/pypy/jit/tl/pypyjit_child.py
pypy/branch/fast-forward/pypy/jit/tool/loopviewer.py
pypy/branch/fast-forward/pypy/jit/tool/showstats.py
pypy/branch/fast-forward/pypy/jit/tool/traceviewer.py
pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py
pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py
pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py
pypy/branch/fast-forward/pypy/module/_rawffi/array.py
pypy/branch/fast-forward/pypy/module/_rawffi/callback.py
pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py
pypy/branch/fast-forward/pypy/module/_rawffi/structure.py
pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py
pypy/branch/fast-forward/pypy/module/_socket/test/test_sock_app.py
pypy/branch/fast-forward/pypy/module/_sre/interp_sre.py
pypy/branch/fast-forward/pypy/module/_weakref/interp__weakref.py
pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py
pypy/branch/fast-forward/pypy/module/array/interp_array.py
pypy/branch/fast-forward/pypy/module/bz2/interp_bz2.py
pypy/branch/fast-forward/pypy/module/cpyext/classobject.py
pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py
pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py
pypy/branch/fast-forward/pypy/module/gc/referents.py
pypy/branch/fast-forward/pypy/module/imp/test/test_import.py
pypy/branch/fast-forward/pypy/module/pypyjit/__init__.py
pypy/branch/fast-forward/pypy/module/pypyjit/interp_jit.py
pypy/branch/fast-forward/pypy/module/pypyjit/policy.py
pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py
pypy/branch/fast-forward/pypy/module/signal/interp_signal.py
pypy/branch/fast-forward/pypy/module/sys/__init__.py
pypy/branch/fast-forward/pypy/module/sys/state.py
pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c
pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py
pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py
pypy/branch/fast-forward/pypy/module/thread/ll_thread.py
pypy/branch/fast-forward/pypy/objspace/flow/flowcontext.py
pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py
pypy/branch/fast-forward/pypy/objspace/std/celldict.py
pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py
pypy/branch/fast-forward/pypy/objspace/std/fake.py
pypy/branch/fast-forward/pypy/objspace/std/model.py
pypy/branch/fast-forward/pypy/objspace/std/objspace.py
pypy/branch/fast-forward/pypy/objspace/std/sharingdict.py
pypy/branch/fast-forward/pypy/objspace/std/stringobject.py
pypy/branch/fast-forward/pypy/objspace/std/stringtype.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_shadowtracking.py
pypy/branch/fast-forward/pypy/objspace/std/typeobject.py
pypy/branch/fast-forward/pypy/rlib/_rsocket_rffi.py
pypy/branch/fast-forward/pypy/rlib/jit.py
pypy/branch/fast-forward/pypy/rlib/rgc.py
pypy/branch/fast-forward/pypy/rlib/rmmap.py
pypy/branch/fast-forward/pypy/rlib/rsre/rsre_char.py
pypy/branch/fast-forward/pypy/rlib/rsre/rsre_core.py
pypy/branch/fast-forward/pypy/rlib/rsre/test/test_match.py
pypy/branch/fast-forward/pypy/rlib/rstring.py
pypy/branch/fast-forward/pypy/rlib/test/test_jit.py
pypy/branch/fast-forward/pypy/rlib/test/test_rdynload.py
pypy/branch/fast-forward/pypy/rlib/test/test_rsocket.py
pypy/branch/fast-forward/pypy/rlib/test/test_rstring.py
pypy/branch/fast-forward/pypy/rlib/test/test_rzlib.py
pypy/branch/fast-forward/pypy/rpython/llinterp.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/ll2ctypes.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/llmemory.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/lloperation.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/lltype.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/rbuilder.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/rclass.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_ll2ctypes.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_llmemory.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_lltype.py
pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_rffi.py
pypy/branch/fast-forward/pypy/rpython/memory/gc/minimarkpage.py
pypy/branch/fast-forward/pypy/rpython/memory/gctransform/asmgcroot.py
pypy/branch/fast-forward/pypy/rpython/memory/gctransform/transform.py
pypy/branch/fast-forward/pypy/rpython/memory/gcwrapper.py
pypy/branch/fast-forward/pypy/rpython/memory/support.py
pypy/branch/fast-forward/pypy/rpython/memory/test/test_gc.py
pypy/branch/fast-forward/pypy/rpython/module/ll_time.py
pypy/branch/fast-forward/pypy/rpython/rbuilder.py
pypy/branch/fast-forward/pypy/rpython/rbuiltin.py
pypy/branch/fast-forward/pypy/rpython/rpbc.py
pypy/branch/fast-forward/pypy/rpython/rtyper.py
pypy/branch/fast-forward/pypy/rpython/test/test_llinterp.py
pypy/branch/fast-forward/pypy/rpython/test/test_nongc.py
pypy/branch/fast-forward/pypy/rpython/test/test_rbuilder.py
pypy/branch/fast-forward/pypy/rpython/test/test_rclass.py
pypy/branch/fast-forward/pypy/rpython/test/test_rptr.py
pypy/branch/fast-forward/pypy/translator/c/funcgen.py
pypy/branch/fast-forward/pypy/translator/c/gcc/instruction.py
pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf/track5.s
pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py
pypy/branch/fast-forward/pypy/translator/c/src/g_include.h
pypy/branch/fast-forward/pypy/translator/c/src/main.h
pypy/branch/fast-forward/pypy/translator/c/src/signals.h
pypy/branch/fast-forward/pypy/translator/c/src/stack.h
pypy/branch/fast-forward/pypy/translator/c/test/test_genc.py
pypy/branch/fast-forward/pypy/translator/c/test/test_lltyped.py
pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py
pypy/branch/fast-forward/pypy/translator/c/test/test_standalone.py
pypy/branch/fast-forward/pypy/translator/c/test/test_typed.py
pypy/branch/fast-forward/pypy/translator/goal/ann_override.py
pypy/branch/fast-forward/pypy/translator/goal/app_main.py
pypy/branch/fast-forward/pypy/translator/jvm/test/test_class.py
pypy/branch/fast-forward/pypy/translator/oosupport/test_template/class_.py
pypy/branch/fast-forward/pypy/translator/platform/darwin.py
pypy/branch/fast-forward/pypy/translator/transform.py
Log:
Merge from trunk:
svn merge -r77543:78227 ../trunk/ .
Modified: pypy/branch/fast-forward/dotviewer/drawgraph.py
==============================================================================
--- pypy/branch/fast-forward/dotviewer/drawgraph.py (original)
+++ pypy/branch/fast-forward/dotviewer/drawgraph.py Fri Oct 22 23:09:43 2010
@@ -423,20 +423,43 @@
else:
for line in lines:
raw_line = line.replace('\\l','').replace('\r','') or ' '
- img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
- w, h = img.get_size()
- if w>wmax: wmax = w
- if raw_line.strip():
- if line.endswith('\\l'):
- def cmd(img=img, y=hmax):
- img.draw(xleft, ytop+y)
- elif line.endswith('\r'):
- def cmd(img=img, y=hmax, w=w):
- img.draw(xright-w, ytop+y)
- else:
- def cmd(img=img, y=hmax, w=w):
- img.draw(xcenter-w//2, ytop+y)
+ if '\f' in raw_line: # grayed out parts of the line
+ imgs = []
+ graytext = True
+ h = 16
+ w_total = 0
+ for linepart in raw_line.split('\f'):
+ graytext = not graytext
+ if not linepart.strip():
+ continue
+ if graytext:
+ fgcolor = (128, 160, 160)
+ else:
+ fgcolor = (0, 0, 0)
+ img = TextSnippet(self, linepart, fgcolor, bgcolor)
+ imgs.append((w_total, img))
+ w, h = img.get_size()
+ w_total += w
+ if w_total > wmax: wmax = w_total
+ def cmd(imgs=imgs, y=hmax):
+ for x, img in imgs:
+ img.draw(xleft+x, ytop+y)
commands.append(cmd)
+ else:
+ img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
+ w, h = img.get_size()
+ if w>wmax: wmax = w
+ if raw_line.strip():
+ if line.endswith('\\l'):
+ def cmd(img=img, y=hmax):
+ img.draw(xleft, ytop+y)
+ elif line.endswith('\r'):
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xright-w, ytop+y)
+ else:
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xcenter-w//2, ytop+y)
+ commands.append(cmd)
hmax += h
#hmax += 8
Modified: pypy/branch/fast-forward/lib-python/conftest.py
==============================================================================
--- pypy/branch/fast-forward/lib-python/conftest.py (original)
+++ pypy/branch/fast-forward/lib-python/conftest.py Fri Oct 22 23:09:43 2010
@@ -496,7 +496,12 @@
RegrTest('test_coding.py'),
RegrTest('test_complex_args.py'),
RegrTest('test_contextlib.py', usemodules="thread"),
- RegrTest('test_ctypes.py', usemodules="_rawffi"),
+ # we skip test ctypes, since we adapted it massively in order
+ # to test what we want to support. There are real failures,
+ # but it's about missing features that we don't want to support
+ # now
+ RegrTest('test_ctypes.py', usemodules="_rawffi",
+ skip="missing features that we don't want to support now"),
RegrTest('test_defaultdict.py'),
RegrTest('test_email_renamed.py'),
RegrTest('test_exception_variations.py'),
Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py (original)
+++ pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py Fri Oct 22 23:09:43 2010
@@ -7,7 +7,7 @@
def round_up(size, alignment):
return (size + alignment - 1) & -alignment
-def size_alignment_pos(fields):
+def size_alignment_pos(fields, is_union=False):
import ctypes
size = 0
alignment = 1
@@ -15,14 +15,19 @@
for fieldname, ctype in fields:
fieldsize = ctypes.sizeof(ctype)
fieldalignment = ctypes.alignment(ctype)
- size = round_up(size, fieldalignment)
alignment = max(alignment, fieldalignment)
- pos.append(size)
- size += fieldsize
+ if is_union:
+ pos.append(0)
+ size = max(size, fieldsize)
+ else:
+ size = round_up(size, fieldalignment)
+ pos.append(size)
+ size += fieldsize
size = round_up(size, alignment)
return size, alignment, pos
-def names_and_fields(_fields_, superclass, zero_offset=False, anon=None):
+def names_and_fields(_fields_, superclass, zero_offset=False, anon=None,
+ is_union=False):
if isinstance(_fields_, tuple):
_fields_ = list(_fields_)
for _, tp in _fields_:
@@ -36,7 +41,7 @@
rawfields = [(name, ctype._ffishape)
for name, ctype in all_fields]
if not zero_offset:
- _, _, pos = size_alignment_pos(all_fields)
+ _, _, pos = size_alignment_pos(all_fields, is_union)
else:
pos = [0] * len(all_fields)
fields = {}
@@ -73,8 +78,8 @@
# ________________________________________________________________
-def _set_shape(tp, rawfields):
- tp._ffistruct = _rawffi.Structure(rawfields)
+def _set_shape(tp, rawfields, is_union=False):
+ tp._ffistruct = _rawffi.Structure(rawfields, is_union)
tp._ffiargshape = tp._ffishape = (tp._ffistruct, 1)
tp._fficompositesize = tp._ffistruct.size
@@ -92,13 +97,14 @@
raise AttributeError("Structure or union cannot contain itself")
self._names, rawfields, self._fieldtypes = names_and_fields(
value, self.__bases__[0], False,
- self.__dict__.get('_anonymous_', None))
+ self.__dict__.get('_anonymous_', None), self._is_union)
_CDataMeta.__setattr__(self, '_fields_', value)
- _set_shape(self, rawfields)
+ _set_shape(self, rawfields, self._is_union)
return
_CDataMeta.__setattr__(self, name, value)
-class StructureMeta(_CDataMeta):
+class StructOrUnionMeta(_CDataMeta):
+
def __new__(self, name, cls, typedict):
res = type.__new__(self, name, cls, typedict)
if '_fields_' in typedict:
@@ -109,8 +115,8 @@
raise AttributeError("Anonymous field not found")
res._names, rawfields, res._fieldtypes = names_and_fields(
typedict['_fields_'], cls[0], False,
- typedict.get('_anonymous_', None))
- _set_shape(res, rawfields)
+ typedict.get('_anonymous_', None), self._is_union)
+ _set_shape(res, rawfields, self._is_union)
return res
@@ -150,8 +156,8 @@
res.__dict__['_index'] = -1
return res
-class Structure(_CData):
- __metaclass__ = StructureMeta
+class StructOrUnion(_CData):
+ __metaclass__ = StructOrUnionMeta
def __new__(cls, *args, **kwds):
if not hasattr(cls, '_ffistruct'):
@@ -213,3 +219,10 @@
def _get_buffer_value(self):
return self._buffer.buffer
+
+
+class StructureMeta(StructOrUnionMeta):
+ _is_union = False
+
+class Structure(StructOrUnion):
+ __metaclass__ = StructureMeta
Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/union.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/_ctypes/union.py (original)
+++ pypy/branch/fast-forward/lib_pypy/_ctypes/union.py Fri Oct 22 23:09:43 2010
@@ -1,118 +1,7 @@
+from _ctypes import structure
+class UnionMeta(structure.StructOrUnionMeta):
+ _is_union = True
-import _rawffi
-from _ctypes.basics import _CData, _CDataMeta, store_reference, keepalive_key
-from _ctypes.basics import ensure_objects
-from _ctypes.structure import round_up, names_and_fields, struct_getattr,\
- struct_setattr
-
-
-def _set_shape(tp):
- size = tp._sizeofinstances()
- alignment = tp._alignmentofinstances()
- tp._ffiopaque = _rawffi.Structure((size, alignment)) # opaque
- tp._ffiargshape = tp._ffishape = (tp._ffiopaque, 1)
- tp._fficompositesize = tp._ffiopaque.size
- # we need to create an array of size one for each
- # of our elements
- tp._ffiarrays = {}
- for name, field in tp._fieldtypes.iteritems():
- tp._ffiarrays[name] = _rawffi.Array(field.ctype._ffishape)
-
-class UnionMeta(_CDataMeta):
- def __new__(self, name, cls, typedict):
- res = type.__new__(self, name, cls, typedict)
- if '_fields_' in typedict:
- res._names, rawfields, res._fieldtypes = names_and_fields(
- typedict['_fields_'], cls[0], True,
- typedict.get('_anonymous_', None))
- _set_shape(res)
-
- def __init__(self): # don't allow arguments by now
- if not hasattr(self, '_ffiarrays'):
- raise TypeError("Cannot instantiate union, has no type")
- # malloc size
- size = self.__class__._sizeofinstances()
- self.__dict__['_objects'] = {}
- self.__dict__['_buffer'] = self._ffiopaque(autofree=True)
- res.__init__ = __init__
- return res
-
- def _sizeofinstances(self):
- if not hasattr(self, '_size_'):
- self._size_ = max([field.size for field in
- self._fieldtypes.values()] + [0])
- return self._size_
-
- def _alignmentofinstances(self):
- from ctypes import alignment
- if not hasattr(self, '_alignment_'):
- self._alignment_ = max([alignment(field.ctype) for field in
- self._fieldtypes.values()] + [1])
- return self._alignment_
-
- __getattr__ = struct_getattr
-
- def __setattr__(self, name, value):
- if name == '_fields_':
- if self.__dict__.get('_fields_', None):
- raise AttributeError("_fields_ is final")
- if self in [v for k, v in value]:
- raise AttributeError("Union cannot contain itself")
- self._names, rawfields, self._fieldtypes = names_and_fields(
- value, self.__bases__[0], True,
- self.__dict__.get('_anonymous_', None))
- _CDataMeta.__setattr__(self, '_fields_', value)
- _set_shape(self)
- _CDataMeta.__setattr__(self, name, value)
-
- def _CData_output(self, resarray, base=None, index=-1):
- res = self.__new__(self)
- ffiopaque = self._ffiopaque.fromaddress(resarray.buffer)
- res.__dict__['_buffer'] = ffiopaque
- res.__dict__['_base'] = base
- res.__dict__['_index'] = index
- return res
-
- def _CData_retval(self, resbuffer):
- res = self.__new__(self)
- res.__dict__['_buffer'] = resbuffer
- res.__dict__['_base'] = None
- res.__dict__['_index'] = -1
- return res
-
-
-class Union(_CData):
+class Union(structure.StructOrUnion):
__metaclass__ = UnionMeta
-
- def __getattr__(self, name):
- try:
- field = self._fieldtypes[name]
- except KeyError:
- raise AttributeError(name)
- fieldtype = field.ctype
- val = self._ffiarrays[name].fromaddress(self._buffer.buffer, 1)
- offset = field.num
- return fieldtype._CData_output(val, self, offset)
-
- def __setattr__(self, name, value):
- try:
- field = self._fieldtypes[name]
- except KeyError:
- raise AttributeError(name)
- fieldtype = field.ctype
- cobj = fieldtype.from_param(value)
- if ensure_objects(cobj) is not None:
- key = keepalive_key(field.num)
- store_reference(self, key, cobj._objects)
- arg = cobj._get_buffer_value()
- if fieldtype._fficompositesize is not None:
- from ctypes import memmove
- dest = self._buffer.buffer
- memmove(dest, arg, fieldtype._fficompositesize)
- else:
- buf = self._ffiarrays[name].fromaddress(self._buffer.buffer, 1)
- buf[0] = arg
-
- def _get_buffer_value(self):
- return self._buffer.buffer
Modified: pypy/branch/fast-forward/pypy/annotation/builtin.py
==============================================================================
--- pypy/branch/fast-forward/pypy/annotation/builtin.py (original)
+++ pypy/branch/fast-forward/pypy/annotation/builtin.py Fri Oct 22 23:09:43 2010
@@ -423,7 +423,7 @@
from pypy.annotation.model import SomePtr
from pypy.rpython.lltypesystem import lltype
-def malloc(s_T, s_n=None, s_flavor=None, s_zero=None):
+def malloc(s_T, s_n=None, s_flavor=None, s_zero=None, s_track_allocation=None):
assert (s_n is None or s_n.knowntype == int
or issubclass(s_n.knowntype, pypy.rlib.rarithmetic.base_int))
assert s_T.is_constant()
@@ -438,13 +438,15 @@
r = SomePtr(lltype.typeOf(p))
else:
assert s_flavor.is_constant()
+ assert s_track_allocation is None or s_track_allocation.is_constant()
# not sure how to call malloc() for the example 'p' in the
# presence of s_extraargs
r = SomePtr(lltype.Ptr(s_T.const))
return r
-def free(s_p, s_flavor):
+def free(s_p, s_flavor, s_track_allocation=None):
assert s_flavor.is_constant()
+ assert s_track_allocation is None or s_track_allocation.is_constant()
# same problem as in malloc(): some flavors are not easy to
# malloc-by-example
#T = s_p.ll_ptrtype.TO
Modified: pypy/branch/fast-forward/pypy/annotation/model.py
==============================================================================
--- pypy/branch/fast-forward/pypy/annotation/model.py (original)
+++ pypy/branch/fast-forward/pypy/annotation/model.py Fri Oct 22 23:09:43 2010
@@ -584,11 +584,11 @@
NUMBER = object()
annotation_to_ll_map = [
+ (SomeSingleFloat(), lltype.SingleFloat),
(s_None, lltype.Void), # also matches SomeImpossibleValue()
(s_Bool, lltype.Bool),
(SomeInteger(knowntype=r_ulonglong), NUMBER),
(SomeFloat(), lltype.Float),
- (SomeSingleFloat(), lltype.SingleFloat),
(SomeLongFloat(), lltype.LongFloat),
(SomeChar(), lltype.Char),
(SomeUnicodeCodePoint(), lltype.UniChar),
Modified: pypy/branch/fast-forward/pypy/annotation/policy.py
==============================================================================
--- pypy/branch/fast-forward/pypy/annotation/policy.py (original)
+++ pypy/branch/fast-forward/pypy/annotation/policy.py Fri Oct 22 23:09:43 2010
@@ -1,4 +1,4 @@
-# base annotation policy for overrides and specialization
+# base annotation policy for specialization
from pypy.annotation.specialize import default_specialize as default
from pypy.annotation.specialize import specialize_argvalue, specialize_argtype, specialize_arglistitemtype
from pypy.annotation.specialize import memo
@@ -41,7 +41,7 @@
if directive is None:
return pol.default_specialize
- # specialize|override:name[(args)]
+ # specialize[(args)]
directive_parts = directive.split('(', 1)
if len(directive_parts) == 1:
[name] = directive_parts
@@ -60,14 +60,6 @@
except AttributeError:
raise AttributeError("%r specialize tag not defined in annotation"
"policy %s" % (name, pol))
- if directive.startswith('override:'):
- # different signature: override__xyz(*args_s)
- if parms:
- raise Exception, "override:* specialisations don't support parameters"
- def specialize_override(funcdesc, args_s):
- funcdesc.overridden = True
- return specializer(*args_s)
- return specialize_override
else:
if not parms:
return specializer
@@ -92,9 +84,5 @@
from pypy.rpython.annlowlevel import LowLevelAnnotatorPolicy
return LowLevelAnnotatorPolicy.specialize__ll_and_arg(*args)
- def override__ignore(pol, *args):
- bk = getbookkeeper()
- return bk.immutablevalue(None)
-
class StrictAnnotatorPolicy(AnnotatorPolicy):
allow_someobjects = False
Modified: pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py
==============================================================================
--- pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py (original)
+++ pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py Fri Oct 22 23:09:43 2010
@@ -766,28 +766,6 @@
s = a.build_types(f, [list])
assert s.classdef is a.bookkeeper.getuniqueclassdef(IndexError) # KeyError ignored because l is a list
- def test_overrides(self):
- excs = []
- def record_exc(e):
- """NOT_RPYTHON"""
- excs.append(sys.exc_info)
- record_exc._annspecialcase_ = "override:record_exc"
- def g():
- pass
- def f():
- try:
- g()
- except Exception, e:
- record_exc(e)
- class MyAnnotatorPolicy(policy.AnnotatorPolicy):
-
- def override__record_exc(pol, s_e):
- return a.bookkeeper.immutablevalue(None)
-
- a = self.RPythonAnnotator(policy=MyAnnotatorPolicy())
- s = a.build_types(f, [])
- assert s.const is None
-
def test_freeze_protocol(self):
class Stuff:
def __init__(self, flag):
@@ -3359,6 +3337,26 @@
s = a.build_types(f, [])
assert isinstance(s, annmodel.SomeChar)
+ def test_context_manager(self):
+ class C:
+ def __init__(self):
+ pass
+ def __enter__(self):
+ self.x = 1
+ def __exit__(self, *args):
+ self.x = 3
+ def f():
+ c = C()
+ with c:
+ pass
+ return c.x
+
+ a = self.RPythonAnnotator()
+ s = a.build_types(f, [])
+ assert isinstance(s, annmodel.SomeInteger)
+ # not a constant: both __enter__ and __exit__ have been annotated
+ assert not s.is_constant()
+
def g(n):
return [0,1,2,n]
Modified: pypy/branch/fast-forward/pypy/annotation/test/test_model.py
==============================================================================
--- pypy/branch/fast-forward/pypy/annotation/test/test_model.py (original)
+++ pypy/branch/fast-forward/pypy/annotation/test/test_model.py Fri Oct 22 23:09:43 2010
@@ -128,7 +128,7 @@
assert isinstance(s_p, SomeOOInstance) and s_p.ootype == C
def test_annotation_to_lltype():
- from pypy.rlib.rarithmetic import r_uint
+ from pypy.rlib.rarithmetic import r_uint, r_singlefloat
s_i = SomeInteger()
s_pos = SomeInteger(nonneg=True)
s_1 = SomeInteger(nonneg=True); s_1.const = 1
@@ -151,6 +151,9 @@
C = ootype.Instance('C', ROOT, {})
ref = SomeOOInstance(C)
assert annotation_to_lltype(ref) == C
+ s_singlefloat = SomeSingleFloat()
+ s_singlefloat.const = r_singlefloat(0.0)
+ assert annotation_to_lltype(s_singlefloat) == lltype.SingleFloat
def test_ll_union():
PS1 = lltype.Ptr(lltype.GcStruct('s'))
Modified: pypy/branch/fast-forward/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/fast-forward/pypy/config/pypyoption.py (original)
+++ pypy/branch/fast-forward/pypy/config/pypyoption.py Fri Oct 22 23:09:43 2010
@@ -73,9 +73,9 @@
}
module_import_dependencies = {
- # no _rawffi if importing pypy.rlib.libffi raises ImportError
+ # no _rawffi if importing pypy.rlib.clibffi raises ImportError
# or CompilationError
- "_rawffi" : ["pypy.rlib.libffi"],
+ "_rawffi" : ["pypy.rlib.clibffi"],
"zlib" : ["pypy.rlib.rzlib"],
"bz2" : ["pypy.module.bz2.interp_bz2"],
@@ -198,6 +198,9 @@
BoolOption("withstrslice", "use strings optimized for slicing",
default=False),
+ BoolOption("withstrbuf", "use strings optimized for addition (ver 2)",
+ default=False),
+
BoolOption("withprebuiltchar",
"use prebuilt single-character string objects",
default=False),
@@ -210,7 +213,8 @@
BoolOption("withrope", "use ropes as the string implementation",
default=False,
requires=[("objspace.std.withstrslice", False),
- ("objspace.std.withstrjoin", False)],
+ ("objspace.std.withstrjoin", False),
+ ("objspace.std.withstrbuf", False)],
suggests=[("objspace.std.withprebuiltchar", True),
("objspace.std.sharesmallstr", True)]),
@@ -238,6 +242,16 @@
default=False,
requires=[("objspace.std.withshadowtracking", False)]),
+ BoolOption("withmapdict",
+ "make instances really small but slow without the JIT",
+ default=False,
+ requires=[("objspace.std.withshadowtracking", False),
+ ("objspace.std.withinlineddict", False),
+ ("objspace.std.withsharingdict", False),
+ ("objspace.std.getattributeshortcut", True),
+ ("objspace.std.withtypeversion", True),
+ ]),
+
BoolOption("withrangelist",
"enable special range list implementation that does not "
"actually create the full list until the resulting "
@@ -343,7 +357,7 @@
config.objspace.std.suggest(withprebuiltint=True)
config.objspace.std.suggest(withrangelist=True)
config.objspace.std.suggest(withprebuiltchar=True)
- config.objspace.std.suggest(withinlineddict=True)
+ config.objspace.std.suggest(withmapdict=True)
config.objspace.std.suggest(withstrslice=True)
config.objspace.std.suggest(withstrjoin=True)
# xxx other options? ropes maybe?
@@ -359,6 +373,7 @@
# extra optimizations with the JIT
if level == 'jit':
config.objspace.std.suggest(withcelldict=True)
+ #config.objspace.std.suggest(withmapdict=True)
def enable_allworkingmodules(config):
Modified: pypy/branch/fast-forward/pypy/conftest.py
==============================================================================
--- pypy/branch/fast-forward/pypy/conftest.py (original)
+++ pypy/branch/fast-forward/pypy/conftest.py Fri Oct 22 23:09:43 2010
@@ -7,6 +7,7 @@
from inspect import isclass, getmro
from pypy.tool.udir import udir
from pypy.tool.autopath import pypydir
+from pypy.tool import leakfinder
# pytest settings
pytest_plugins = "resultlog",
@@ -358,7 +359,14 @@
def runtest(self):
try:
- super(IntTestFunction, self).runtest()
+ leakfinder.start_tracking_allocations()
+ try:
+ super(IntTestFunction, self).runtest()
+ finally:
+ if leakfinder.TRACK_ALLOCATIONS:
+ leaks = leakfinder.stop_tracking_allocations(False)
+ else:
+ leaks = None # stop_tracking_allocations() already called
except OperationError, e:
check_keyboard_interrupt(e)
raise
@@ -377,6 +385,8 @@
_pygame_imported = True
assert option.view, ("should not invoke Pygame "
"if conftest.option.view is False")
+ if leaks: # check for leaks, but only if the test passed so far
+ raise leakfinder.MallocMismatch(leaks)
class AppTestFunction(PyPyTestFunction):
def _prunetraceback(self, traceback):
Modified: pypy/branch/fast-forward/pypy/doc/docindex.txt
==============================================================================
--- pypy/branch/fast-forward/pypy/doc/docindex.txt (original)
+++ pypy/branch/fast-forward/pypy/doc/docindex.txt Fri Oct 22 23:09:43 2010
@@ -84,7 +84,7 @@
PyPy's own tests `summary`_, daily updated, run through BuildBot infrastructure.
You can also find CPython's compliance tests run with compiled ``pypy-c``
-exeuctables there.
+executables there.
information dating from early 2007:
Modified: pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py Fri Oct 22 23:09:43 2010
@@ -168,6 +168,20 @@
def _call_builtin_destructor(self):
pass # method overridden in typedef.py
+ # hooks that the mapdict implementations needs:
+ def _get_mapdict_map(self):
+ return None
+ def _set_mapdict_map(self, map):
+ raise NotImplementedError
+ def _mapdict_read_storage(self, index):
+ raise NotImplementedError
+ def _mapdict_write_storage(self, index, value):
+ raise NotImplementedError
+ def _mapdict_storage_length(self):
+ raise NotImplementedError
+ def _set_mapdict_storage_and_map(self, storage, map):
+ raise NotImplementedError
+
class Wrappable(W_Root):
"""A subclass of Wrappable is an internal, interpreter-level class
Modified: pypy/branch/fast-forward/pypy/interpreter/pycode.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pycode.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pycode.py Fri Oct 22 23:09:43 2010
@@ -117,6 +117,10 @@
self._compute_flatcall()
+ if self.space.config.objspace.std.withmapdict:
+ from pypy.objspace.std.mapdict import init_mapdict_cache
+ init_mapdict_cache(self)
+
def _freeze_(self):
if (self.magic == cpython_magic and
'__pypy__' not in sys.builtin_module_names):
Modified: pypy/branch/fast-forward/pypy/interpreter/pyopcode.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyopcode.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyopcode.py Fri Oct 22 23:09:43 2010
@@ -164,6 +164,9 @@
next_instr = block.handle(self, unroller)
return next_instr
+ def call_contextmanager_exit_function(self, w_func, w_typ, w_val, w_tb):
+ return self.space.call_function(w_func, w_typ, w_val, w_tb)
+
@jit.unroll_safe
def dispatch_bytecode(self, co_code, next_instr, ec):
space = self.space
@@ -710,9 +713,14 @@
def LOAD_ATTR(self, nameindex, next_instr):
"obj.attributename"
- w_attributename = self.getname_w(nameindex)
w_obj = self.popvalue()
- w_value = self.space.getattr(w_obj, w_attributename)
+ if (self.space.config.objspace.std.withmapdict
+ and not jit.we_are_jitted()):
+ from pypy.objspace.std.mapdict import LOAD_ATTR_caching
+ w_value = LOAD_ATTR_caching(self.getcode(), w_obj, nameindex)
+ else:
+ w_attributename = self.getname_w(nameindex)
+ w_value = self.space.getattr(w_obj, w_attributename)
self.pushvalue(w_value)
LOAD_ATTR._always_inline_ = True
@@ -904,28 +912,47 @@
self.pushvalue(w_result)
def WITH_CLEANUP(self, oparg, next_instr):
- self.dropvalues(2)
- w_unroller = self.popvalue()
+ # see comment in END_FINALLY for stack state
+ # This opcode changed a lot between CPython versions
+ if (self.pycode.magic >= 0xa0df2ef
+ # Implementation since 2.7a0: 62191 (introduce SETUP_WITH)
+ or self.pycode.magic >= 0xa0df2d1):
+ # implementation since 2.6a1: 62161 (WITH_CLEANUP optimization)
+ self.popvalue()
+ self.popvalue()
+ w_unroller = self.popvalue()
+ w_exitfunc = self.popvalue()
+ self.pushvalue(w_unroller)
+ self.pushvalue(self.space.w_None)
+ self.pushvalue(self.space.w_None)
+ elif self.pycode.magic >= 0xa0df28c:
+ # Implementation since 2.5a0: 62092 (changed WITH_CLEANUP opcode)
+ w_exitfunc = self.popvalue()
+ w_unroller = self.peekvalue(2)
+ else:
+ raise NotImplementedError("WITH_CLEANUP for CPython <= 2.4")
+
unroller = self.space.interpclass_w(w_unroller)
w_exit = self.popvalue()
is_app_exc = (unroller is not None and
isinstance(unroller, SApplicationException))
if is_app_exc:
operr = unroller.operr
- w_type = operr.w_type
- w_value = operr.get_w_value(self.space)
- w_tb = self.space.wrap(operr.application_traceback)
- else:
- w_type = w_value = w_tb = self.space.w_None
- w_suppress = self.space.call_function(w_exit, w_type, w_value, w_tb)
- if is_app_exc and self.space.is_true(w_suppress):
- self.pushvalue(self.space.w_None)
- self.pushvalue(self.space.w_None)
- self.pushvalue(self.space.w_None)
+ w_traceback = self.space.wrap(operr.application_traceback)
+ w_suppress = self.call_contextmanager_exit_function(
+ w_exitfunc,
+ operr.w_type,
+ operr.get_w_value(self.space),
+ w_traceback)
+ if self.space.is_true(w_suppress):
+ # __exit__() returned True -> Swallow the exception.
+ self.settopvalue(self.space.w_None, 2)
else:
- self.pushvalue(w_unroller)
- self.pushvalue(w_value)
- self.pushvalue(w_type)
+ self.call_contextmanager_exit_function(
+ w_exitfunc,
+ self.space.w_None,
+ self.space.w_None,
+ self.space.w_None)
@jit.unroll_safe
def call_function(self, oparg, w_star=None, w_starstar=None):
@@ -1345,10 +1372,10 @@
unroller.operr.normalize_exception(frame.space)
return FinallyBlock.really_handle(self, frame, unroller)
-
block_classes = {'SETUP_LOOP': LoopBlock,
'SETUP_EXCEPT': ExceptBlock,
- 'SETUP_FINALLY': FinallyBlock}
+ 'SETUP_FINALLY': FinallyBlock,
+ 'SETUP_WITH': WithBlock}
### helpers written at the application-level ###
# Some of these functions are expected to be generally useful if other
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py Fri Oct 22 23:09:43 2010
@@ -268,7 +268,8 @@
i += 1
import StringIO
print "states = ["
- for state in dfa.states:
+ for numstate, state in enumerate(dfa.states):
+ print " #", numstate
s = StringIO.StringIO()
i = 0
for k, v in sorted(state.items()):
@@ -284,14 +285,17 @@
s.write(', ')
s.write('},')
i = 0
- for line in textwrap.wrap(s.getvalue(), width=35):
+ if len(state) <= 4:
+ text = [s.getvalue()]
+ else:
+ text = textwrap.wrap(s.getvalue(), width=36)
+ for line in text:
line = line.replace('::', ': ')
if i == 0:
print ' {' + line
else:
print ' ' + line
i += 1
- print
print " ]"
print "%s = automata.%s(states, accepts)" % (name, dfa_class)
print
@@ -302,16 +306,8 @@
endDFAMap = makePyEndDFAMap()
output("double3DFA", "NonGreedyDFA", endDFAMap['"""'])
output("single3DFA", "NonGreedyDFA", endDFAMap["'''"])
- output("doubleDFA", "DFA", endDFAMap['"'])
output("singleDFA", "DFA", endDFAMap["'"])
- print "endDFAs = {\"'\" : singleDFA,"
- print " '\"' : doubleDFA,"
- print " 'r' : None,"
- print " 'R' : None,"
- print " 'u' : None,"
- print " 'U' : None,"
- print " 'b' : None,"
- print " 'B' : None}"
+ output("doubleDFA", "DFA", endDFAMap['"'])
# ______________________________________________________________________
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py Fri Oct 22 23:09:43 2010
@@ -30,6 +30,7 @@
True, False, False, False, False, True, False,
False, False, True]
states = [
+ # 0
{'\t': 0, '\n': 13, '\x0c': 0,
'\r': 14, ' ': 0, '!': 10, '"': 16,
'#': 18, '%': 12, '&': 12, "'": 15,
@@ -55,7 +56,7 @@
'u': 2, 'v': 1, 'w': 1, 'x': 1,
'y': 1, 'z': 1, '{': 13, '|': 12,
'}': 13, '~': 13},
-
+ # 1
{'0': 1, '1': 1, '2': 1, '3': 1,
'4': 1, '5': 1, '6': 1, '7': 1,
'8': 1, '9': 1, 'A': 1, 'B': 1,
@@ -72,7 +73,7 @@
'p': 1, 'q': 1, 'r': 1, 's': 1,
't': 1, 'u': 1, 'v': 1, 'w': 1,
'x': 1, 'y': 1, 'z': 1},
-
+ # 2
{'"': 16, "'": 15, '0': 1, '1': 1,
'2': 1, '3': 1, '4': 1, '5': 1,
'6': 1, '7': 1, '8': 1, '9': 1,
@@ -90,7 +91,7 @@
'r': 3, 's': 1, 't': 1, 'u': 1,
'v': 1, 'w': 1, 'x': 1, 'y': 1,
'z': 1},
-
+ # 3
{'"': 16, "'": 15, '0': 1, '1': 1,
'2': 1, '3': 1, '4': 1, '5': 1,
'6': 1, '7': 1, '8': 1, '9': 1,
@@ -108,215 +109,188 @@
'r': 1, 's': 1, 't': 1, 'u': 1,
'v': 1, 'w': 1, 'x': 1, 'y': 1,
'z': 1},
-
+ # 4
{'.': 24, '0': 21, '1': 21, '2': 21,
'3': 21, '4': 21, '5': 21, '6': 21,
'7': 21, '8': 23, '9': 23, 'B': 22,
'E': 25, 'J': 13, 'L': 13, 'O': 20,
'X': 19, 'b': 22, 'e': 25, 'j': 13,
'l': 13, 'o': 20, 'x': 19},
-
+ # 5
{'.': 24, '0': 5, '1': 5, '2': 5,
'3': 5, '4': 5, '5': 5, '6': 5,
'7': 5, '8': 5, '9': 5, 'E': 25,
'J': 13, 'L': 13, 'e': 25, 'j': 13,
'l': 13},
-
+ # 6
{'0': 26, '1': 26, '2': 26, '3': 26,
'4': 26, '5': 26, '6': 26, '7': 26,
'8': 26, '9': 26},
-
+ # 7
{'*': 12, '=': 13},
-
+ # 8
{'=': 13, '>': 12},
-
+ # 9
{'<': 12, '=': 13, '>': 13},
-
+ # 10
{'=': 13},
-
+ # 11
{'/': 12, '=': 13},
-
+ # 12
{'=': 13},
-
+ # 13
{},
-
+ # 14
{'\n': 13},
-
- {automata.DEFAULT: 30, '\n': 27,
- "'": 28, '\\': 29},
-
- {automata.DEFAULT: 33, '\n': 27,
- '"': 31, '\\': 32},
-
+ # 15
+ {automata.DEFAULT: 30, '\n': 27, "'": 28, '\\': 29},
+ # 16
+ {automata.DEFAULT: 33, '\n': 27, '"': 31, '\\': 32},
+ # 17
{'\n': 13, '\r': 14},
-
- {automata.DEFAULT: 18, '\n': 27,
- '\r': 27},
-
+ # 18
+ {automata.DEFAULT: 18, '\n': 27, '\r': 27},
+ # 19
{'0': 19, '1': 19, '2': 19, '3': 19,
'4': 19, '5': 19, '6': 19, '7': 19,
'8': 19, '9': 19, 'A': 19, 'B': 19,
'C': 19, 'D': 19, 'E': 19, 'F': 19,
'L': 13, 'a': 19, 'b': 19, 'c': 19,
- 'd': 19, 'e': 19, 'f': 19,
- 'l': 13},
-
+ 'd': 19, 'e': 19, 'f': 19, 'l': 13},
+ # 20
{'0': 20, '1': 20, '2': 20, '3': 20,
'4': 20, '5': 20, '6': 20, '7': 20,
'L': 13, 'l': 13},
-
+ # 21
{'.': 24, '0': 21, '1': 21, '2': 21,
'3': 21, '4': 21, '5': 21, '6': 21,
'7': 21, '8': 23, '9': 23, 'E': 25,
'J': 13, 'L': 13, 'e': 25, 'j': 13,
'l': 13},
-
- {'0': 22, '1': 22, 'L': 13,
- 'l': 13},
-
+ # 22
+ {'0': 22, '1': 22, 'L': 13, 'l': 13},
+ # 23
{'.': 24, '0': 23, '1': 23, '2': 23,
'3': 23, '4': 23, '5': 23, '6': 23,
'7': 23, '8': 23, '9': 23, 'E': 25,
'J': 13, 'e': 25, 'j': 13},
-
+ # 24
{'0': 24, '1': 24, '2': 24, '3': 24,
'4': 24, '5': 24, '6': 24, '7': 24,
'8': 24, '9': 24, 'E': 34, 'J': 13,
'e': 34, 'j': 13},
-
+ # 25
{'+': 35, '-': 35, '0': 36, '1': 36,
'2': 36, '3': 36, '4': 36, '5': 36,
- '6': 36, '7': 36, '8': 36,
- '9': 36},
-
+ '6': 36, '7': 36, '8': 36, '9': 36},
+ # 26
{'0': 26, '1': 26, '2': 26, '3': 26,
'4': 26, '5': 26, '6': 26, '7': 26,
'8': 26, '9': 26, 'E': 34, 'J': 13,
'e': 34, 'j': 13},
-
+ # 27
{},
-
+ # 28
{"'": 13},
-
- {automata.DEFAULT: 37, '\n': 13,
- '\r': 14},
-
- {automata.DEFAULT: 30, '\n': 27,
- "'": 13, '\\': 29},
-
+ # 29
+ {automata.DEFAULT: 37, '\n': 13, '\r': 14},
+ # 30
+ {automata.DEFAULT: 30, '\n': 27, "'": 13, '\\': 29},
+ # 31
{'"': 13},
-
- {automata.DEFAULT: 38, '\n': 13,
- '\r': 14},
-
- {automata.DEFAULT: 33, '\n': 27,
- '"': 13, '\\': 32},
-
+ # 32
+ {automata.DEFAULT: 38, '\n': 13, '\r': 14},
+ # 33
+ {automata.DEFAULT: 33, '\n': 27, '"': 13, '\\': 32},
+ # 34
{'+': 39, '-': 39, '0': 40, '1': 40,
'2': 40, '3': 40, '4': 40, '5': 40,
- '6': 40, '7': 40, '8': 40,
- '9': 40},
-
+ '6': 40, '7': 40, '8': 40, '9': 40},
+ # 35
{'0': 36, '1': 36, '2': 36, '3': 36,
'4': 36, '5': 36, '6': 36, '7': 36,
'8': 36, '9': 36},
-
+ # 36
{'0': 36, '1': 36, '2': 36, '3': 36,
'4': 36, '5': 36, '6': 36, '7': 36,
- '8': 36, '9': 36, 'J': 13,
- 'j': 13},
-
- {automata.DEFAULT: 37, '\n': 27,
- "'": 13, '\\': 29},
-
- {automata.DEFAULT: 38, '\n': 27,
- '"': 13, '\\': 32},
-
+ '8': 36, '9': 36, 'J': 13, 'j': 13},
+ # 37
+ {automata.DEFAULT: 37, '\n': 27, "'": 13, '\\': 29},
+ # 38
+ {automata.DEFAULT: 38, '\n': 27, '"': 13, '\\': 32},
+ # 39
{'0': 40, '1': 40, '2': 40, '3': 40,
'4': 40, '5': 40, '6': 40, '7': 40,
'8': 40, '9': 40},
-
+ # 40
{'0': 40, '1': 40, '2': 40, '3': 40,
'4': 40, '5': 40, '6': 40, '7': 40,
- '8': 40, '9': 40, 'J': 13,
- 'j': 13},
-
+ '8': 40, '9': 40, 'J': 13, 'j': 13},
]
pseudoDFA = automata.DFA(states, accepts)
accepts = [False, False, False, False, False, True]
states = [
- {automata.DEFAULT: 0, '"': 1,
- '\\': 2},
-
- {automata.DEFAULT: 4, '"': 3,
- '\\': 2},
-
+ # 0
+ {automata.DEFAULT: 0, '"': 1, '\\': 2},
+ # 1
+ {automata.DEFAULT: 4, '"': 3, '\\': 2},
+ # 2
{automata.DEFAULT: 4},
-
- {automata.DEFAULT: 4, '"': 5,
- '\\': 2},
-
- {automata.DEFAULT: 4, '"': 1,
- '\\': 2},
-
- {automata.DEFAULT: 4, '"': 5,
- '\\': 2},
-
+ # 3
+ {automata.DEFAULT: 4, '"': 5, '\\': 2},
+ # 4
+ {automata.DEFAULT: 4, '"': 1, '\\': 2},
+ # 5
+ {automata.DEFAULT: 4, '"': 5, '\\': 2},
]
double3DFA = automata.NonGreedyDFA(states, accepts)
accepts = [False, False, False, False, False, True]
states = [
- {automata.DEFAULT: 0, "'": 1,
- '\\': 2},
-
- {automata.DEFAULT: 4, "'": 3,
- '\\': 2},
-
+ # 0
+ {automata.DEFAULT: 0, "'": 1, '\\': 2},
+ # 1
+ {automata.DEFAULT: 4, "'": 3, '\\': 2},
+ # 2
{automata.DEFAULT: 4},
-
- {automata.DEFAULT: 4, "'": 5,
- '\\': 2},
-
- {automata.DEFAULT: 4, "'": 1,
- '\\': 2},
-
- {automata.DEFAULT: 4, "'": 5,
- '\\': 2},
-
+ # 3
+ {automata.DEFAULT: 4, "'": 5, '\\': 2},
+ # 4
+ {automata.DEFAULT: 4, "'": 1, '\\': 2},
+ # 5
+ {automata.DEFAULT: 4, "'": 5, '\\': 2},
]
single3DFA = automata.NonGreedyDFA(states, accepts)
accepts = [False, True, False, False]
states = [
- {automata.DEFAULT: 0, '"': 1,
- '\\': 2},
-
+ # 0
+ {automata.DEFAULT: 0, "'": 1, '\\': 2},
+ # 1
{},
-
+ # 2
{automata.DEFAULT: 3},
-
- {automata.DEFAULT: 3, '"': 1,
- '\\': 2},
-
+ # 3
+ {automata.DEFAULT: 3, "'": 1, '\\': 2},
]
-doubleDFA = automata.DFA(states, accepts)
+singleDFA = automata.DFA(states, accepts)
accepts = [False, True, False, False]
states = [
- {automata.DEFAULT: 0, "'": 1,
- '\\': 2},
-
+ # 0
+ {automata.DEFAULT: 0, '"': 1, '\\': 2},
+ # 1
{},
-
+ # 2
{automata.DEFAULT: 3},
-
- {automata.DEFAULT: 3, "'": 1,
- '\\': 2},
-
+ # 3
+ {automata.DEFAULT: 3, '"': 1, '\\': 2},
]
-singleDFA = automata.DFA(states, accepts)
+doubleDFA = automata.DFA(states, accepts)
+
+#_______________________________________________________________________
+# End of automatically generated DFA's
endDFAs = {"'" : singleDFA,
'"' : doubleDFA,
Modified: pypy/branch/fast-forward/pypy/interpreter/typedef.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/typedef.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/typedef.py Fri Oct 22 23:09:43 2010
@@ -127,6 +127,13 @@
typedef = cls.typedef
if wants_dict and typedef.hasdict:
wants_dict = False
+ if config.objspace.std.withmapdict and not typedef.hasdict:
+ # mapdict only works if the type does not already have a dict
+ if wants_del:
+ parentcls = get_unique_interplevel_subclass(config, cls, True, True,
+ False, True)
+ return _usersubclswithfeature(config, parentcls, "del")
+ return _usersubclswithfeature(config, cls, "user", "dict", "weakref", "slots")
# Forest of if's - see the comment above.
if wants_del:
if wants_dict:
@@ -180,10 +187,20 @@
def add(Proto):
for key, value in Proto.__dict__.items():
- if not key.startswith('__') or key == '__del__':
+ if (not key.startswith('__') and not key.startswith('_mixin_')
+ or key == '__del__'):
+ if hasattr(value, "func_name"):
+ value = func_with_new_name(value, value.func_name)
body[key] = value
+ if (config.objspace.std.withmapdict and "dict" in features):
+ from pypy.objspace.std.mapdict import BaseMapdictObject, ObjectMixin
+ add(BaseMapdictObject)
+ add(ObjectMixin)
+ features = ()
+
if "user" in features: # generic feature needed by all subcls
+
class Proto(object):
user_overridden_class = True
@@ -249,6 +266,9 @@
wantdict = False
if wantdict:
+ base_user_setup = supercls.user_setup.im_func
+ if "user_setup" in body:
+ base_user_setup = body["user_setup"]
class Proto(object):
def getdict(self):
return self.w__dict__
@@ -257,11 +277,9 @@
self.w__dict__ = check_new_dictionary(space, w_dict)
def user_setup(self, space, w_subtype):
- self.space = space
- self.w__class__ = w_subtype
self.w__dict__ = space.newdict(
instance=True, classofinstance=w_subtype)
- self.user_setup_slots(w_subtype.nslots)
+ base_user_setup(self, space, w_subtype)
def setclass(self, space, w_subtype):
# only used by descr_set___class__
Modified: pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py Fri Oct 22 23:09:43 2010
@@ -10,7 +10,7 @@
BoxInt, BoxPtr, BoxObj, BoxFloat,
REF, INT, FLOAT)
from pypy.jit.codewriter import heaptracker
-from pypy.rpython.lltypesystem import lltype, llmemory, rclass, rstr
+from pypy.rpython.lltypesystem import lltype, llmemory, rclass, rstr, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.module.support import LLSupport, OOSupport
from pypy.rpython.llinterp import LLException
@@ -305,12 +305,12 @@
loop = _from_opaque(loop)
loop.operations.append(Operation(opnum))
-def compile_add_descr(loop, ofs, type):
+def compile_add_descr(loop, ofs, type, arg_types):
from pypy.jit.backend.llgraph.runner import Descr
loop = _from_opaque(loop)
op = loop.operations[-1]
assert isinstance(type, str) and len(type) == 1
- op.descr = Descr(ofs, type)
+ op.descr = Descr(ofs, type, arg_types=arg_types)
def compile_add_loop_token(loop, descr):
if we_are_translated():
@@ -801,7 +801,7 @@
else:
raise TypeError(x)
try:
- return _do_call_common(func, args_in_order)
+ return _do_call_common(func, args_in_order, calldescr)
except LLException, lle:
_last_exception = lle
d = {'v': None,
@@ -1018,6 +1018,9 @@
if isinstance(TYPE, lltype.Ptr):
if isinstance(x, (int, long, llmemory.AddressAsInt)):
x = llmemory.cast_int_to_adr(x)
+ if TYPE is rffi.VOIDP:
+ # assume that we want a "C-style" cast, without typechecking the value
+ return rffi.cast(TYPE, x)
return llmemory.cast_adr_to_ptr(x, TYPE)
elif TYPE == llmemory.Address:
if isinstance(x, (int, long, llmemory.AddressAsInt)):
@@ -1411,10 +1414,26 @@
def do_call_pushfloat(x):
_call_args_f.append(x)
-def _do_call_common(f, args_in_order=None):
+kind2TYPE = {
+ 'i': lltype.Signed,
+ 'f': lltype.Float,
+ 'v': lltype.Void,
+ }
+
+def _do_call_common(f, args_in_order=None, calldescr=None):
ptr = llmemory.cast_int_to_adr(f).ptr
- FUNC = lltype.typeOf(ptr).TO
- ARGS = FUNC.ARGS
+ PTR = lltype.typeOf(ptr)
+ if PTR == rffi.VOIDP:
+ # it's a pointer to a C function, so we don't have a precise
+ # signature: create one from the descr
+ ARGS = map(kind2TYPE.get, calldescr.arg_types)
+ RESULT = kind2TYPE[calldescr.typeinfo]
+ FUNC = lltype.FuncType(ARGS, RESULT)
+ func_to_call = rffi.cast(lltype.Ptr(FUNC), ptr)
+ else:
+ FUNC = PTR.TO
+ ARGS = FUNC.ARGS
+ func_to_call = ptr._obj._callable
args = cast_call_args(ARGS, _call_args_i, _call_args_r, _call_args_f,
args_in_order)
del _call_args_i[:]
@@ -1426,7 +1445,7 @@
result = llinterp.eval_graph(ptr._obj.graph, args)
# ^^^ may raise, in which case we get an LLException
else:
- result = ptr._obj._callable(*args)
+ result = func_to_call(*args)
return result
def do_call_void(f):
Modified: pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py Fri Oct 22 23:09:43 2010
@@ -154,7 +154,7 @@
llimpl.compile_add(c, op.getopnum())
descr = op.getdescr()
if isinstance(descr, Descr):
- llimpl.compile_add_descr(c, descr.ofs, descr.typeinfo)
+ llimpl.compile_add_descr(c, descr.ofs, descr.typeinfo, descr.arg_types)
if isinstance(descr, history.LoopToken) and op.getopnum() != rop.JUMP:
llimpl.compile_add_loop_token(c, descr)
if self.is_oo and isinstance(descr, (OODescr, MethDescr)):
@@ -297,6 +297,18 @@
return self.getdescr(0, token[0], extrainfo=extrainfo,
arg_types=''.join(arg_types))
+ def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo=None):
+ from pypy.jit.backend.llsupport.ffisupport import get_ffi_type_kind
+ arg_types = []
+ for arg in ffi_args:
+ kind = get_ffi_type_kind(arg)
+ if kind != history.VOID:
+ arg_types.append(kind)
+ reskind = get_ffi_type_kind(ffi_result)
+ return self.getdescr(0, reskind, extrainfo=extrainfo,
+ arg_types=''.join(arg_types))
+
+
def grab_exc_value(self):
return llimpl.grab_exc_value()
Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/descr.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llsupport/descr.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/descr.py Fri Oct 22 23:09:43 2010
@@ -82,6 +82,7 @@
_is_pointer_field = False # unless overridden by GcPtrFieldDescr
_is_float_field = False # unless overridden by FloatFieldDescr
+ _is_field_signed = False # unless overridden by XxxFieldDescr
def is_pointer_field(self):
return self._is_pointer_field
@@ -89,6 +90,9 @@
def is_float_field(self):
return self._is_float_field
+ def is_field_signed(self):
+ return self._is_field_signed
+
def repr_of_descr(self):
return '<%s %s %s>' % (self._clsname, self.name, self.offset)
@@ -105,7 +109,7 @@
def getFieldDescrClass(TYPE):
return getDescrClass(TYPE, BaseFieldDescr, GcPtrFieldDescr,
NonGcPtrFieldDescr, 'Field', 'get_field_size',
- '_is_float_field')
+ '_is_float_field', '_is_field_signed')
def get_field_descr(gccache, STRUCT, fieldname):
cache = gccache._cache_field
@@ -144,6 +148,7 @@
_is_array_of_pointers = False # unless overridden by GcPtrArrayDescr
_is_array_of_floats = False # unless overridden by FloatArrayDescr
+ _is_item_signed = False # unless overridden by XxxArrayDescr
def is_array_of_pointers(self):
return self._is_array_of_pointers
@@ -151,6 +156,9 @@
def is_array_of_floats(self):
return self._is_array_of_floats
+ def is_item_signed(self):
+ return self._is_item_signed
+
def repr_of_descr(self):
return '<%s>' % self._clsname
@@ -186,12 +194,12 @@
def getArrayDescrClass(ARRAY):
return getDescrClass(ARRAY.OF, BaseArrayDescr, GcPtrArrayDescr,
NonGcPtrArrayDescr, 'Array', 'get_item_size',
- '_is_array_of_floats')
+ '_is_array_of_floats', '_is_item_signed')
def getArrayNoLengthDescrClass(ARRAY):
return getDescrClass(ARRAY.OF, BaseArrayNoLengthDescr, GcPtrArrayNoLengthDescr,
NonGcPtrArrayNoLengthDescr, 'ArrayNoLength', 'get_item_size',
- '_is_array_of_floats')
+ '_is_array_of_floats', '_is_item_signed')
def get_array_descr(gccache, ARRAY):
cache = gccache._cache_array
@@ -242,6 +250,9 @@
def get_result_size(self, translate_support_code):
raise NotImplementedError
+ def is_result_signed(self):
+ return False # unless overridden
+
def create_call_stub(self, rtyper, RESULT):
def process(c):
arg = 'args_%s[%d]' % (c, seen[c])
@@ -307,6 +318,30 @@
_return_type = history.INT
call_stub = staticmethod(lambda func, args_i, args_r, args_f: 0)
+ _is_result_signed = False # can be overridden in XxxCallDescr
+ def is_result_signed(self):
+ return self._is_result_signed
+
+class DynamicIntCallDescr(BaseIntCallDescr):
+ """
+ calldescr that works for every integer type, by explicitly passing it the
+ size of the result. Used only by get_call_descr_dynamic
+ """
+ _clsname = 'DynamicIntCallDescr'
+
+ def __init__(self, arg_classes, result_size, result_sign, extrainfo=None):
+ BaseIntCallDescr.__init__(self, arg_classes, extrainfo)
+ assert isinstance(result_sign, bool)
+ self._result_size = chr(result_size)
+ self._result_sign = result_sign
+
+ def get_result_size(self, translate_support_code):
+ return ord(self._result_size)
+
+ def is_result_signed(self):
+ return self._result_sign
+
+
class NonGcPtrCallDescr(BaseIntCallDescr):
_clsname = 'NonGcPtrCallDescr'
def get_result_size(self, translate_support_code):
@@ -341,7 +376,8 @@
return FloatCallDescr
return getDescrClass(RESULT, BaseIntCallDescr, GcPtrCallDescr,
NonGcPtrCallDescr, 'Call', 'get_result_size',
- Ellipsis) # <= floatattrname should not be used here
+ Ellipsis, # <= floatattrname should not be used here
+ '_is_result_signed')
def get_call_descr(gccache, ARGS, RESULT, extrainfo=None):
arg_classes = []
@@ -368,7 +404,8 @@
# ____________________________________________________________
def getDescrClass(TYPE, BaseDescr, GcPtrDescr, NonGcPtrDescr,
- nameprefix, methodname, floatattrname, _cache={}):
+ nameprefix, methodname, floatattrname, signedattrname,
+ _cache={}):
if isinstance(TYPE, lltype.Ptr):
if TYPE.TO._gckind == 'gc':
return GcPtrDescr
@@ -388,6 +425,8 @@
#
if TYPE is lltype.Float:
setattr(Descr, floatattrname, True)
+ elif TYPE is not lltype.Bool and rffi.cast(TYPE, -1) == -1:
+ setattr(Descr, signedattrname, True)
#
_cache[nameprefix, TYPE] = Descr
return Descr
Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py Fri Oct 22 23:09:43 2010
@@ -158,7 +158,7 @@
# used to avoid too many duplications in the GCREF_LISTs.
self.hashtable = lltype.malloc(self.HASHTABLE,
self.HASHTABLE_SIZE+1,
- flavor='raw')
+ flavor='raw', track_allocation=False)
dummy = lltype.direct_ptradd(lltype.direct_arrayitems(self.hashtable),
self.HASHTABLE_SIZE)
dummy = llmemory.cast_ptr_to_adr(dummy)
@@ -252,14 +252,15 @@
def _enlarge_gcmap(self):
newlength = 250 + self._gcmap_maxlength * 2
- newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw')
+ newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw',
+ track_allocation=False)
oldgcmap = self._gcmap
for i in range(self._gcmap_curlength):
newgcmap[i] = oldgcmap[i]
self._gcmap = newgcmap
self._gcmap_maxlength = newlength
if oldgcmap:
- lltype.free(oldgcmap, flavor='raw')
+ lltype.free(oldgcmap, flavor='raw', track_allocation=False)
def get_basic_shape(self, is_64_bit=False):
# XXX: Should this code even really know about stack frame layout of
@@ -308,7 +309,8 @@
# them inside bigger arrays) and we never try to share them.
length = len(shape)
compressed = lltype.malloc(self.CALLSHAPE_ARRAY, length,
- flavor='raw')
+ flavor='raw',
+ track_allocation=False) # memory leak
for i in range(length):
compressed[length-1-i] = rffi.cast(rffi.UCHAR, shape[i])
return llmemory.cast_ptr_to_adr(compressed)
Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/llmodel.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llsupport/llmodel.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/llmodel.py Fri Oct 22 23:09:43 2010
@@ -17,6 +17,7 @@
from pypy.jit.backend.llsupport.descr import get_call_descr
from pypy.jit.backend.llsupport.descr import BaseIntCallDescr, GcPtrCallDescr
from pypy.jit.backend.llsupport.descr import FloatCallDescr, VoidCallDescr
+from pypy.jit.backend.llsupport.ffisupport import get_call_descr_dynamic
from pypy.rpython.annlowlevel import cast_instance_to_base_ptr
@@ -82,7 +83,8 @@
# read back by the machine code reading at the address given by
# pos_exception() and pos_exc_value().
_exception_emulator = lltype.malloc(rffi.CArray(lltype.Signed), 2,
- zero=True, flavor='raw')
+ zero=True, flavor='raw',
+ immortal=True)
self._exception_emulator = _exception_emulator
def _store_exception(lle):
@@ -210,7 +212,8 @@
assert isinstance(fielddescr, BaseFieldDescr)
ofs = fielddescr.offset
size = fielddescr.get_field_size(self.translate_support_code)
- return ofs, size
+ sign = fielddescr.is_field_signed()
+ return ofs, size, sign
unpack_fielddescr_size._always_inline_ = True
def arraydescrof(self, A):
@@ -225,12 +228,16 @@
assert isinstance(arraydescr, BaseArrayDescr)
ofs = arraydescr.get_base_size(self.translate_support_code)
size = arraydescr.get_item_size(self.translate_support_code)
- return ofs, size
+ sign = arraydescr.is_item_signed()
+ return ofs, size, sign
unpack_arraydescr_size._always_inline_ = True
def calldescrof(self, FUNC, ARGS, RESULT, extrainfo=None):
return get_call_descr(self.gc_ll_descr, ARGS, RESULT, extrainfo)
+ def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo=None):
+ return get_call_descr_dynamic(ffi_args, ffi_result, extrainfo)
+
def get_overflow_error(self):
ovf_vtable = self.cast_adr_to_int(self._ovf_error_vtable)
ovf_inst = lltype.cast_opaque_ptr(llmemory.GCREF,
@@ -252,15 +259,21 @@
@specialize.argtype(2)
def bh_getarrayitem_gc_i(self, arraydescr, gcref, itemindex):
- ofs, size = self.unpack_arraydescr_size(arraydescr)
+ ofs, size, sign = self.unpack_arraydescr_size(arraydescr)
# --- start of GC unsafe code (no GC operation!) ---
items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for STYPE, UTYPE, itemsize in unroll_basic_sizes:
if size == itemsize:
- items = rffi.cast(rffi.CArrayPtr(TYPE), items)
- val = items[itemindex]
+ if sign:
+ items = rffi.cast(rffi.CArrayPtr(STYPE), items)
+ val = items[itemindex]
+ val = rffi.cast(lltype.Signed, val)
+ else:
+ items = rffi.cast(rffi.CArrayPtr(UTYPE), items)
+ val = items[itemindex]
+ val = rffi.cast(lltype.Signed, val)
# --- end of GC unsafe code ---
- return rffi.cast(lltype.Signed, val)
+ return val
else:
raise NotImplementedError("size = %d" % size)
@@ -285,10 +298,10 @@
@specialize.argtype(2)
def bh_setarrayitem_gc_i(self, arraydescr, gcref, itemindex, newvalue):
- ofs, size = self.unpack_arraydescr_size(arraydescr)
+ ofs, size, sign = self.unpack_arraydescr_size(arraydescr)
# --- start of GC unsafe code (no GC operation!) ---
items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for TYPE, _, itemsize in unroll_basic_sizes:
if size == itemsize:
items = rffi.cast(rffi.CArrayPtr(TYPE), items)
items[itemindex] = rffi.cast(TYPE, newvalue)
@@ -339,14 +352,22 @@
@specialize.argtype(1)
def _base_do_getfield_i(self, struct, fielddescr):
- ofs, size = self.unpack_fielddescr_size(fielddescr)
+ ofs, size, sign = self.unpack_fielddescr_size(fielddescr)
# --- start of GC unsafe code (no GC operation!) ---
fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for STYPE, UTYPE, itemsize in unroll_basic_sizes:
if size == itemsize:
- val = rffi.cast(rffi.CArrayPtr(TYPE), fieldptr)[0]
+ # Note that in the common case where size==sizeof(Signed),
+ # both cases of what follows are doing the same thing.
+ # But gcc is clever enough to figure this out :-)
+ if sign:
+ val = rffi.cast(rffi.CArrayPtr(STYPE), fieldptr)[0]
+ val = rffi.cast(lltype.Signed, val)
+ else:
+ val = rffi.cast(rffi.CArrayPtr(UTYPE), fieldptr)[0]
+ val = rffi.cast(lltype.Signed, val)
# --- end of GC unsafe code ---
- return rffi.cast(lltype.Signed, val)
+ return val
else:
raise NotImplementedError("size = %d" % size)
@@ -378,10 +399,10 @@
@specialize.argtype(1)
def _base_do_setfield_i(self, struct, fielddescr, newvalue):
- ofs, size = self.unpack_fielddescr_size(fielddescr)
+ ofs, size, sign = self.unpack_fielddescr_size(fielddescr)
# --- start of GC unsafe code (no GC operation!) ---
fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for TYPE, _, itemsize in unroll_basic_sizes:
if size == itemsize:
fieldptr = rffi.cast(rffi.CArrayPtr(TYPE), fieldptr)
fieldptr[0] = rffi.cast(TYPE, newvalue)
Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/symbolic.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llsupport/symbolic.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/symbolic.py Fri Oct 22 23:09:43 2010
@@ -69,8 +69,9 @@
SIZEOF_INT = get_size(rffi.INT, False)
SIZEOF_FLOAT = get_size(lltype.Float, False)
-unroll_basic_sizes = unrolling_iterable([(lltype.Signed, WORD),
- (lltype.Char, SIZEOF_CHAR),
- (rffi.SHORT, SIZEOF_SHORT),
- (rffi.INT, SIZEOF_INT)])
+unroll_basic_sizes = unrolling_iterable([
+ (lltype.Signed, lltype.Unsigned, WORD),
+ (rffi.SIGNEDCHAR, lltype.Char, SIZEOF_CHAR),
+ (rffi.SHORT, rffi.USHORT, SIZEOF_SHORT),
+ (rffi.INT, rffi.UINT, SIZEOF_INT)])
# does not contain Float ^^^ which must be special-cased
Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_descr.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_descr.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_descr.py Fri Oct 22 23:09:43 2010
@@ -83,6 +83,18 @@
assert descr_f.is_float_field()
+def test_get_field_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ descr_x = get_field_descr(c2, S, 'x')
+ assert descr_x.is_field_signed() == signed
+
+
def test_get_array_descr():
U = lltype.Struct('U')
T = lltype.GcStruct('T')
@@ -164,6 +176,25 @@
assert descr.get_base_size(False) == 0
assert descr.get_ofs_length(False) == -1
+
+def test_get_array_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ A = lltype.GcArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ arraydescr = get_array_descr(c2, A)
+ assert arraydescr.is_item_signed() == signed
+ #
+ RA = rffi.CArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ arraydescr = get_array_descr(c2, RA)
+ assert arraydescr.is_item_signed() == signed
+
+
def test_get_call_descr_not_translated():
c0 = GcCache(False)
descr1 = get_call_descr(c0, [lltype.Char, lltype.Signed], lltype.Char)
@@ -219,6 +250,17 @@
extrainfo = descr3.get_extra_info()
assert extrainfo is None
+def test_get_call_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ A = lltype.GcArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ descr1 = get_call_descr(c2, [], RESTYPE)
+ assert descr1.is_result_signed() == signed
+
def test_repr_of_descr():
c0 = GcCache(False)
Modified: pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py Fri Oct 22 23:09:43 2010
@@ -9,12 +9,13 @@
ConstObj, BoxFloat, ConstFloat)
from pypy.jit.metainterp.resoperation import ResOperation, rop
from pypy.jit.metainterp.typesystem import deref
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi, rclass
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.llinterp import LLException
from pypy.jit.codewriter import heaptracker
+from pypy.rlib.rarithmetic import intmask
class Runner(object):
@@ -421,6 +422,7 @@
assert x == 3.5 - 42
def test_call(self):
+ from pypy.rlib.libffi import types
def func_int(a, b):
return a + b
@@ -428,23 +430,31 @@
return chr(ord(c) + ord(c1))
functions = [
- (func_int, lltype.Signed, 655360),
- (func_int, rffi.SHORT, 1213),
- (func_char, lltype.Char, 12)
+ (func_int, lltype.Signed, types.sint, 655360),
+ (func_int, rffi.SHORT, types.sint16, 1213),
+ (func_char, lltype.Char, types.uchar, 12)
]
- for func, TP, num in functions:
+ for func, TP, ffi_type, num in functions:
cpu = self.cpu
#
FPTR = self.Ptr(self.FuncType([TP, TP], TP))
func_ptr = llhelper(FPTR, func)
FUNC = deref(FPTR)
- calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
funcbox = self.get_funcbox(cpu, func_ptr)
+ # first, try it with the "normal" calldescr
+ calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
res = self.execute_operation(rop.CALL,
[funcbox, BoxInt(num), BoxInt(num)],
'int', descr=calldescr)
assert res.value == 2 * num
+ # then, try it with the dynamic calldescr
+ dyn_calldescr = cpu.calldescrof_dynamic([ffi_type, ffi_type], ffi_type)
+ res = self.execute_operation(rop.CALL,
+ [funcbox, BoxInt(num), BoxInt(num)],
+ 'int', descr=dyn_calldescr)
+ assert res.value == 2 * num
+
if cpu.supports_floats:
def func(f0, f1, f2, f3, f4, f5, f6, i0, i1, f7, f8, f9):
@@ -507,6 +517,24 @@
'int', descr=calldescr)
assert res.value == func_ints(*args)
+ def test_call_to_c_function(self):
+ from pypy.rlib.libffi import CDLL, types, ArgChain
+ from pypy.rpython.lltypesystem.ll2ctypes import libc_name
+ libc = CDLL(libc_name)
+ c_tolower = libc.getpointer('tolower', [types.uchar], types.sint)
+ argchain = ArgChain().arg(ord('A'))
+ assert c_tolower.call(argchain, rffi.INT) == ord('a')
+
+ func_adr = llmemory.cast_ptr_to_adr(c_tolower.funcsym)
+ funcbox = ConstInt(heaptracker.adr2int(func_adr))
+ calldescr = self.cpu.calldescrof_dynamic([types.uchar], types.sint)
+ res = self.execute_operation(rop.CALL,
+ [funcbox, BoxInt(ord('A'))],
+ 'int',
+ descr=calldescr)
+ assert res.value == ord('a')
+
+
def test_field_basic(self):
t_box, T_box = self.alloc_instance(self.T)
fielddescr = self.cpu.fielddescrof(self.S, 'value')
@@ -833,6 +861,23 @@
length_box], 'void')
assert self.look_string(r_box) == "!??cdef?!"
+ def test_copyunicodecontent(self):
+ s_box = self.alloc_unicode(u"abcdef")
+ for s_box in [s_box, s_box.constbox()]:
+ for srcstart_box in [BoxInt(2), ConstInt(2)]:
+ for dststart_box in [BoxInt(3), ConstInt(3)]:
+ for length_box in [BoxInt(4), ConstInt(4)]:
+ for r_box_is_const in [False, True]:
+ r_box = self.alloc_unicode(u"!???????!")
+ if r_box_is_const:
+ r_box = r_box.constbox()
+ self.execute_operation(rop.COPYUNICODECONTENT,
+ [s_box, r_box,
+ srcstart_box,
+ dststart_box,
+ length_box], 'void')
+ assert self.look_unicode(r_box) == u"!??cdef?!"
+
def test_do_unicode_basic(self):
u = self.cpu.bh_newunicode(5)
self.cpu.bh_unicodesetitem(u, 4, 123)
@@ -1227,6 +1272,10 @@
u_box = BoxPtr(lltype.cast_opaque_ptr(llmemory.GCREF, u))
return u_box
+ def look_unicode(self, unicode_box):
+ u = unicode_box.getref(lltype.Ptr(rstr.UNICODE))
+ return u''.join(u.chars)
+
def test_casts(self):
py.test.skip("xxx fix or kill")
@@ -1283,6 +1332,7 @@
descr=fd)
res = self.execute_operation(get_op, [s_box], 'int', descr=fd)
assert res.getint() == 32
+ lltype.free(s, flavor='raw')
def test_new_with_vtable(self):
cpu = self.cpu
@@ -1980,6 +2030,196 @@
assert self.cpu.get_latest_value_float(0) == 13.5
assert called
+ def test_short_result_of_getfield_direct(self):
+ # Test that a getfield that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ descrfld_x = cpu.fielddescrof(S, 'x')
+ s = lltype.malloc(S)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ s.x = rffi.cast(RESTYPE, value)
+ x = cpu.bh_getfield_gc_i(lltype.cast_opaque_ptr(llmemory.GCREF, s),
+ descrfld_x)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_getfield_compiled(self):
+ # Test that a getfield that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ descrfld_x = cpu.fielddescrof(S, 'x')
+ s = lltype.malloc(S)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ s.x = rffi.cast(RESTYPE, value)
+ s_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
+ res = self.execute_operation(rop.GETFIELD_GC, [BoxPtr(s_gcref)],
+ 'int', descr=descrfld_x)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
+ def test_short_result_of_getarrayitem_direct(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = lltype.GcArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ x = cpu.bh_getarrayitem_gc_i(
+ descrarray, lltype.cast_opaque_ptr(llmemory.GCREF, a), 3)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_getarrayitem_compiled(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = lltype.GcArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, a)
+ res = self.execute_operation(rop.GETARRAYITEM_GC,
+ [BoxPtr(a_gcref), BoxInt(3)],
+ 'int', descr=descrarray)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
+ def test_short_result_of_getarrayitem_raw_direct(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = rffi.CArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5, flavor='raw')
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_rawint = heaptracker.adr2int(llmemory.cast_ptr_to_adr(a))
+ x = cpu.bh_getarrayitem_raw_i(descrarray, a_rawint, 3)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+ lltype.free(a, flavor='raw')
+
+ def test_short_result_of_getarrayitem_raw_compiled(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = rffi.CArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5, flavor='raw')
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_rawint = heaptracker.adr2int(llmemory.cast_ptr_to_adr(a))
+ res = self.execute_operation(rop.GETARRAYITEM_RAW,
+ [BoxInt(a_rawint), BoxInt(3)],
+ 'int', descr=descrarray)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+ lltype.free(a, flavor='raw')
+
+ def test_short_result_of_call_direct(self):
+ # Test that calling a function that returns a CHAR, SHORT or INT,
+ # signed or unsigned, properly gets zero-extended or sign-extended.
+ from pypy.translator.tool.cbuild import ExternalCompilationInfo
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ # Tested with a function that intentionally does not cast the
+ # result to RESTYPE, but makes sure that we return the whole
+ # value in eax or rax.
+ eci = ExternalCompilationInfo(separate_module_sources=["""
+ long fn_test_result_of_call(long x)
+ {
+ return x + 1;
+ }
+ """])
+ f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed],
+ RESTYPE, compilation_info=eci, _nowrapper=True)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1))
+ assert intmask(f(value)) == expected
+ #
+ FUNC = self.FuncType([lltype.Signed], RESTYPE)
+ FPTR = self.Ptr(FUNC)
+ calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+ x = self.cpu.bh_call_i(self.get_funcbox(self.cpu, f).value,
+ calldescr, [value], None, None)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_call_compiled(self):
+ # Test that calling a function that returns a CHAR, SHORT or INT,
+ # signed or unsigned, properly gets zero-extended or sign-extended.
+ from pypy.translator.tool.cbuild import ExternalCompilationInfo
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ # Tested with a function that intentionally does not cast the
+ # result to RESTYPE, but makes sure that we return the whole
+ # value in eax or rax.
+ eci = ExternalCompilationInfo(separate_module_sources=["""
+ long fn_test_result_of_call(long x)
+ {
+ return x + 1;
+ }
+ """])
+ f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed],
+ RESTYPE, compilation_info=eci, _nowrapper=True)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1))
+ assert intmask(f(value)) == expected
+ #
+ FUNC = self.FuncType([lltype.Signed], RESTYPE)
+ FPTR = self.Ptr(FUNC)
+ calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+ funcbox = self.get_funcbox(self.cpu, f)
+ res = self.execute_operation(rop.CALL, [funcbox, BoxInt(value)],
+ 'int', descr=calldescr)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
class OOtypeBackendTest(BaseBackendTest):
Modified: pypy/branch/fast-forward/pypy/jit/backend/test/test_ll_random.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/test/test_ll_random.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/test/test_ll_random.py Fri Oct 22 23:09:43 2010
@@ -599,7 +599,7 @@
OPERATIONS.append(StrLenOperation(rop.STRLEN))
OPERATIONS.append(UnicodeLenOperation(rop.UNICODELEN))
OPERATIONS.append(CopyStrContentOperation(rop.COPYSTRCONTENT))
- #OPERATIONS.append(CopyUnicodeContentOperation(rop.COPYUNICODECONTENT))
+ OPERATIONS.append(CopyUnicodeContentOperation(rop.COPYUNICODECONTENT))
for i in range(2):
OPERATIONS.append(GuardClassOperation(rop.GUARD_CLASS))
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py Fri Oct 22 23:09:43 2010
@@ -8,7 +8,8 @@
from pypy.rpython.annlowlevel import llhelper
from pypy.tool.uid import fixid
from pypy.jit.backend.x86.regalloc import (RegAlloc, X86RegisterManager,
- X86XMMRegisterManager, get_ebp_ofs)
+ X86XMMRegisterManager, get_ebp_ofs,
+ _get_scale)
from pypy.jit.backend.x86.arch import (FRAME_FIXED_SIZE, FORCE_INDEX_OFS, WORD,
IS_X86_32, IS_X86_64)
@@ -22,7 +23,8 @@
X86_64_SCRATCH_REG,
X86_64_XMM_SCRATCH_REG,
RegLoc, StackLoc, ConstFloatLoc,
- ImmedLoc, AddressLoc, imm)
+ ImmedLoc, AddressLoc, imm,
+ imm0, imm1)
from pypy.rlib.objectmodel import we_are_translated, specialize
from pypy.jit.backend.x86 import rx86, regloc, codebuf
@@ -244,12 +246,13 @@
f = open_file_as_stream(output_log, "w")
for i in range(len(self.loop_run_counters)):
name, struct = self.loop_run_counters[i]
- f.write(str(struct.i) + " " * (8 - len(str(struct.i))) + name + "\n")
+ f.write(str(name) + ":" + str(struct.i) + "\n")
f.close()
def _build_float_constants(self):
# 44 bytes: 32 bytes for the data, and up to 12 bytes for alignment
- addr = lltype.malloc(rffi.CArray(lltype.Char), 44, flavor='raw')
+ addr = lltype.malloc(rffi.CArray(lltype.Char), 44, flavor='raw',
+ track_allocation=False)
if not we_are_translated():
self._keepalive_malloced_float_consts = addr
float_constants = rffi.cast(lltype.Signed, addr)
@@ -399,9 +402,10 @@
funcname = "" % len(self.loop_run_counters)
# invent the counter, so we don't get too confused
if self._debug:
- struct = lltype.malloc(DEBUG_COUNTER, flavor='raw')
+ struct = lltype.malloc(DEBUG_COUNTER, flavor='raw',
+ track_allocation=False) # known to leak
struct.i = 0
- self.loop_run_counters.append((funcname, struct))
+ self.loop_run_counters.append((len(self.loop_run_counters), struct))
return funcname
def patch_jump_for_descr(self, faildescr, adr_new_target):
@@ -440,7 +444,7 @@
# self.mc.PUSH(eax)
# adr = rffi.cast(lltype.Signed, self.loop_run_counters[-1][1])
# self.mc.MOV(eax, heap(adr))
- # self.mc.ADD(eax, imm(1))
+ # self.mc.ADD(eax, imm1)
# self.mc.MOV(heap(adr), eax)
# self.mc.POP(eax)
return operations
@@ -711,7 +715,7 @@
self.regalloc_perform_with_guard(None, guard_op, faillocs, arglocs,
resloc, current_depths)
- def load_effective_addr(self, sizereg, baseofs, scale, result, frm=imm(0)):
+ def load_effective_addr(self, sizereg, baseofs, scale, result, frm=imm0):
self.mc.LEA(result, addr_add(frm, sizereg, baseofs, scale))
def _unaryop(asmop):
@@ -973,28 +977,28 @@
def genop_guard_int_is_true(self, op, guard_op, guard_token, arglocs, resloc):
guard_opnum = guard_op.getopnum()
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
if guard_opnum == rop.GUARD_TRUE:
return self.implement_guard(guard_token, 'Z')
else:
return self.implement_guard(guard_token, 'NZ')
def genop_int_is_true(self, op, arglocs, resloc):
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
rl = resloc.lowest8bits()
self.mc.SET_ir(rx86.Conditions['NE'], rl.value)
self.mc.MOVZX8(resloc, rl)
def genop_guard_int_is_zero(self, op, guard_op, guard_token, arglocs, resloc):
guard_opnum = guard_op.getopnum()
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
if guard_opnum == rop.GUARD_TRUE:
return self.implement_guard(guard_token, 'NZ')
else:
return self.implement_guard(guard_token, 'Z')
def genop_int_is_zero(self, op, arglocs, resloc):
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
rl = resloc.lowest8bits()
self.mc.SET_ir(rx86.Conditions['E'], rl.value)
self.mc.MOVZX8(resloc, rl)
@@ -1050,50 +1054,66 @@
assert result_loc is eax
self.call(self.malloc_unicode_func_addr, arglocs, eax)
- def genop_getfield_gc(self, op, arglocs, resloc):
- base_loc, ofs_loc, size_loc = arglocs
- assert isinstance(size_loc, ImmedLoc)
+ # ----------
+
+ def load_from_mem(self, resloc, source_addr, size_loc, sign_loc):
assert isinstance(resloc, RegLoc)
size = size_loc.value
-
- source_addr = AddressLoc(base_loc, ofs_loc)
+ sign = sign_loc.value
if resloc.is_xmm:
self.mc.MOVSD(resloc, source_addr)
+ elif size == WORD:
+ self.mc.MOV(resloc, source_addr)
+ elif size == 1:
+ if sign:
+ self.mc.MOVSX8(resloc, source_addr)
+ else:
+ self.mc.MOVZX8(resloc, source_addr)
+ elif size == 2:
+ if sign:
+ self.mc.MOVSX16(resloc, source_addr)
+ else:
+ self.mc.MOVZX16(resloc, source_addr)
+ elif IS_X86_64 and size == 4:
+ if sign:
+ self.mc.MOVSX32(resloc, source_addr)
+ else:
+ self.mc.MOV32(resloc, source_addr) # zero-extending
+ else:
+ not_implemented("load_from_mem size = %d" % size)
+
+ def save_into_mem(self, dest_addr, value_loc, size_loc):
+ size = size_loc.value
+ if isinstance(value_loc, RegLoc) and value_loc.is_xmm:
+ self.mc.MOVSD(dest_addr, value_loc)
elif size == 1:
- self.mc.MOVZX8(resloc, source_addr)
+ self.mc.MOV8(dest_addr, value_loc.lowest8bits())
elif size == 2:
- self.mc.MOVZX16(resloc, source_addr)
+ self.mc.MOV16(dest_addr, value_loc)
elif size == 4:
- # MOV32 is zero-extending on 64-bit, so this is okay
- self.mc.MOV32(resloc, source_addr)
+ self.mc.MOV32(dest_addr, value_loc)
elif IS_X86_64 and size == 8:
- self.mc.MOV(resloc, source_addr)
+ self.mc.MOV(dest_addr, value_loc)
else:
- raise NotImplementedError("getfield size = %d" % size)
+ not_implemented("save_into_mem size = %d" % size)
+
+ def genop_getfield_gc(self, op, arglocs, resloc):
+ base_loc, ofs_loc, size_loc, sign_loc = arglocs
+ assert isinstance(size_loc, ImmedLoc)
+ source_addr = AddressLoc(base_loc, ofs_loc)
+ self.load_from_mem(resloc, source_addr, size_loc, sign_loc)
genop_getfield_raw = genop_getfield_gc
genop_getfield_raw_pure = genop_getfield_gc
genop_getfield_gc_pure = genop_getfield_gc
def genop_getarrayitem_gc(self, op, arglocs, resloc):
- base_loc, ofs_loc, scale, ofs = arglocs
+ base_loc, ofs_loc, size_loc, ofs, sign_loc = arglocs
assert isinstance(ofs, ImmedLoc)
- assert isinstance(scale, ImmedLoc)
- src_addr = addr_add(base_loc, ofs_loc, ofs.value, scale.value)
- if op.result.type == FLOAT:
- self.mc.MOVSD(resloc, src_addr)
- else:
- if scale.value == 0:
- self.mc.MOVZX8(resloc, src_addr)
- elif scale.value == 1:
- self.mc.MOVZX16(resloc, src_addr)
- elif scale.value == 2:
- self.mc.MOV32(resloc, src_addr)
- elif IS_X86_64 and scale.value == 3:
- self.mc.MOV(resloc, src_addr)
- else:
- print "[asmgen]getarrayitem unsupported size: %d" % scale.value
- raise NotImplementedError()
+ assert isinstance(size_loc, ImmedLoc)
+ scale = _get_scale(size_loc.value)
+ src_addr = addr_add(base_loc, ofs_loc, ofs.value, scale)
+ self.load_from_mem(resloc, src_addr, size_loc, sign_loc)
genop_getarrayitem_gc_pure = genop_getarrayitem_gc
genop_getarrayitem_raw = genop_getarrayitem_gc
@@ -1101,40 +1121,16 @@
def genop_discard_setfield_gc(self, op, arglocs):
base_loc, ofs_loc, size_loc, value_loc = arglocs
assert isinstance(size_loc, ImmedLoc)
- size = size_loc.value
dest_addr = AddressLoc(base_loc, ofs_loc)
- if isinstance(value_loc, RegLoc) and value_loc.is_xmm:
- self.mc.MOVSD(dest_addr, value_loc)
- elif IS_X86_64 and size == 8:
- self.mc.MOV(dest_addr, value_loc)
- elif size == 4:
- self.mc.MOV32(dest_addr, value_loc)
- elif size == 2:
- self.mc.MOV16(dest_addr, value_loc)
- elif size == 1:
- self.mc.MOV8(dest_addr, value_loc.lowest8bits())
- else:
- print "[asmgen]setfield addr size %d" % size
- raise NotImplementedError("Addr size %d" % size)
+ self.save_into_mem(dest_addr, value_loc, size_loc)
def genop_discard_setarrayitem_gc(self, op, arglocs):
- base_loc, ofs_loc, value_loc, scale_loc, baseofs = arglocs
+ base_loc, ofs_loc, value_loc, size_loc, baseofs = arglocs
assert isinstance(baseofs, ImmedLoc)
- assert isinstance(scale_loc, ImmedLoc)
- dest_addr = AddressLoc(base_loc, ofs_loc, scale_loc.value, baseofs.value)
- if op.getarg(2).type == FLOAT:
- self.mc.MOVSD(dest_addr, value_loc)
- else:
- if IS_X86_64 and scale_loc.value == 3:
- self.mc.MOV(dest_addr, value_loc)
- elif scale_loc.value == 2:
- self.mc.MOV32(dest_addr, value_loc)
- elif scale_loc.value == 1:
- self.mc.MOV16(dest_addr, value_loc)
- elif scale_loc.value == 0:
- self.mc.MOV8(dest_addr, value_loc.lowest8bits())
- else:
- raise NotImplementedError("scale = %d" % scale_loc.value)
+ assert isinstance(size_loc, ImmedLoc)
+ scale = _get_scale(size_loc.value)
+ dest_addr = AddressLoc(base_loc, ofs_loc, scale, baseofs.value)
+ self.save_into_mem(dest_addr, value_loc, size_loc)
def genop_discard_strsetitem(self, op, arglocs):
base_loc, ofs_loc, val_loc = arglocs
@@ -1201,7 +1197,7 @@
def genop_guard_guard_no_exception(self, ign_1, guard_op, guard_token,
locs, ign_2):
- self.mc.CMP(heap(self.cpu.pos_exception()), imm(0))
+ self.mc.CMP(heap(self.cpu.pos_exception()), imm0)
return self.implement_guard(guard_token, 'NZ')
def genop_guard_guard_exception(self, ign_1, guard_op, guard_token,
@@ -1213,8 +1209,8 @@
addr = self.implement_guard(guard_token, 'NE')
if resloc is not None:
self.mc.MOV(resloc, heap(self.cpu.pos_exc_value()))
- self.mc.MOV(heap(self.cpu.pos_exception()), imm(0))
- self.mc.MOV(heap(self.cpu.pos_exc_value()), imm(0))
+ self.mc.MOV(heap(self.cpu.pos_exception()), imm0)
+ self.mc.MOV(heap(self.cpu.pos_exc_value()), imm0)
return addr
def _gen_guard_overflow(self, guard_op, guard_token):
@@ -1224,8 +1220,8 @@
elif guard_opnum == rop.GUARD_OVERFLOW:
return self.implement_guard(guard_token, 'NO')
else:
- print "int_xxx_ovf followed by", guard_op.getopname()
- raise AssertionError
+ not_implemented("int_xxx_ovf followed by %s" %
+ guard_op.getopname())
def genop_guard_int_add_ovf(self, op, guard_op, guard_token, arglocs, result_loc):
self.genop_int_add(op, arglocs, result_loc)
@@ -1288,7 +1284,7 @@
def genop_guard_guard_nonnull_class(self, ign_1, guard_op,
guard_token, locs, ign_2):
self.mc.ensure_bytes_available(256)
- self.mc.CMP(locs[0], imm(1))
+ self.mc.CMP(locs[0], imm1)
# Patched below
self.mc.J_il8(rx86.Conditions['B'], 0)
jb_location = self.mc.get_relative_pos()
@@ -1637,25 +1633,34 @@
sizeloc = arglocs[0]
assert isinstance(sizeloc, ImmedLoc)
size = sizeloc.value
+ signloc = arglocs[1]
if isinstance(op.getarg(0), Const):
x = imm(op.getarg(0).getint())
else:
- x = arglocs[1]
+ x = arglocs[2]
if x is eax:
tmp = ecx
else:
tmp = eax
- self._emit_call(x, arglocs, 2, tmp=tmp)
+ self._emit_call(x, arglocs, 3, tmp=tmp)
+
+ if IS_X86_32 and isinstance(resloc, StackLoc) and resloc.width == 8:
+ self.mc.FSTP_b(resloc.value) # float return
+ elif size == WORD:
+ assert resloc is eax or resloc is xmm0 # a full word
+ elif size == 0:
+ pass # void return
+ else:
+ # use the code in load_from_mem to do the zero- or sign-extension
+ assert resloc is eax
+ if size == 1:
+ srcloc = eax.lowest8bits()
+ else:
+ srcloc = eax
+ self.load_from_mem(eax, srcloc, sizeloc, signloc)
- if isinstance(resloc, StackLoc) and resloc.width == 8 and IS_X86_32:
- self.mc.FSTP_b(resloc.value)
- elif size == 1:
- self.mc.AND_ri(eax.value, 0xff)
- elif size == 2:
- self.mc.AND_ri(eax.value, 0xffff)
-
def genop_guard_call_may_force(self, op, guard_op, guard_token,
arglocs, result_loc):
faildescr = guard_op.getdescr()
@@ -1808,20 +1813,16 @@
self.mc.LEA_rb(resloc.value, FORCE_INDEX_OFS)
def not_implemented_op_discard(self, op, arglocs):
- msg = "not implemented operation: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation: %s" % op.getopname())
def not_implemented_op(self, op, arglocs, resloc):
- msg = "not implemented operation with res: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation with res: %s" %
+ op.getopname())
def not_implemented_op_guard(self, op, guard_op,
failaddr, arglocs, resloc):
- msg = "not implemented operation (guard): %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation (guard): %s" %
+ op.getopname())
def mark_gc_roots(self):
gcrootmap = self.cpu.gc_ll_descr.gcrootmap
@@ -1905,3 +1906,7 @@
def heap(addr):
return AddressLoc(ImmedLoc(addr), ImmedLoc(0), 0, 0)
+
+def not_implemented(msg):
+ os.write(2, '[x86/asm] %s\n' % msg)
+ raise NotImplementedError(msg)
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py Fri Oct 22 23:09:43 2010
@@ -2,6 +2,7 @@
""" Register allocation scheme.
"""
+import os
from pypy.jit.metainterp.history import (Box, Const, ConstInt, ConstPtr,
ResOperation, BoxPtr,
LoopToken, INT, REF, FLOAT)
@@ -40,12 +41,10 @@
return imm(c.value)
elif isinstance(c, ConstPtr):
if we_are_translated() and c.value and rgc.can_move(c.value):
- print "convert_to_imm: ConstPtr needs special care"
- raise AssertionError
+ not_implemented("convert_to_imm: ConstPtr needs special care")
return imm(rffi.cast(lltype.Signed, c.value))
else:
- print "convert_to_imm: got a %s" % c
- raise AssertionError
+ not_implemented("convert_to_imm: got a %s" % c)
class X86_64_RegisterManager(X86RegisterManager):
# r11 omitted because it's used as scratch
@@ -70,8 +69,9 @@
def _get_new_array(self):
n = self.BASE_CONSTANT_SIZE
+ # known to leak
self.cur_array = lltype.malloc(rffi.CArray(lltype.Float), n,
- flavor='raw')
+ flavor='raw', track_allocation=False)
self.cur_array_free = n
_get_new_array._dont_inline_ = True
@@ -359,8 +359,8 @@
if op.is_ovf():
if (operations[i + 1].getopnum() != rop.GUARD_NO_OVERFLOW and
operations[i + 1].getopnum() != rop.GUARD_OVERFLOW):
- print "int_xxx_ovf not followed by guard_(no)_overflow"
- raise AssertionError
+ not_implemented("int_xxx_ovf not followed by "
+ "guard_(no)_overflow")
return True
return False
if (operations[i + 1].getopnum() != rop.GUARD_TRUE and
@@ -412,8 +412,8 @@
arg = op.getarg(j)
if isinstance(arg, Box):
if arg not in start_live:
- print "Bogus arg in operation %d at %d" % (op.getopnum(), i)
- raise AssertionError
+ not_implemented("Bogus arg in operation %d at %d" %
+ (op.getopnum(), i))
longevity[arg] = (start_live[arg], i)
if op.is_guard():
for arg in op.getfailargs():
@@ -421,8 +421,8 @@
continue
assert isinstance(arg, Box)
if arg not in start_live:
- print "Bogus arg in guard %d at %d" % (op.getopnum(), i)
- raise AssertionError
+ not_implemented("Bogus arg in guard %d at %d" %
+ (op.getopnum(), i))
longevity[arg] = (start_live[arg], i)
for arg in inputargs:
if arg not in longevity:
@@ -667,7 +667,13 @@
assert isinstance(calldescr, BaseCallDescr)
assert len(calldescr.arg_classes) == op.numargs() - 1
size = calldescr.get_result_size(self.translate_support_code)
- self._call(op, [imm(size)] + [self.loc(op.getarg(i)) for i in range(op.numargs())],
+ sign = calldescr.is_result_signed()
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self._call(op, [imm(size), sign_loc] +
+ [self.loc(op.getarg(i)) for i in range(op.numargs())],
guard_not_forced_op=guard_not_forced_op)
def consider_call(self, op):
@@ -688,7 +694,7 @@
self.rm._sync_var(op.getarg(vable_index))
vable = self.fm.loc(op.getarg(vable_index))
else:
- vable = imm(0)
+ vable = imm0
self._call(op, [imm(size), vable] +
[self.loc(op.getarg(i)) for i in range(op.numargs())],
guard_not_forced_op=guard_op)
@@ -778,15 +784,11 @@
loc = self.loc(op.getarg(0))
return self._call(op, [loc])
# boehm GC (XXX kill the following code at some point)
- ofs_items, itemsize, ofs = symbolic.get_array_token(rstr.UNICODE, self.translate_support_code)
- if itemsize == 4:
- return self._malloc_varsize(ofs_items, ofs, 2, op.getarg(0),
- op.result)
- elif itemsize == 2:
- return self._malloc_varsize(ofs_items, ofs, 1, op.getarg(0),
- op.result)
- else:
- assert False, itemsize
+ ofs_items, _, ofs = symbolic.get_array_token(rstr.UNICODE,
+ self.translate_support_code)
+ scale = self._get_unicode_item_scale()
+ return self._malloc_varsize(ofs_items, ofs, scale, op.getarg(0),
+ op.result)
def _malloc_varsize(self, ofs_items, ofs_length, scale, v, res_v):
# XXX kill this function at some point
@@ -818,8 +820,9 @@
arglocs.append(self.loc(op.getarg(0)))
return self._call(op, arglocs)
# boehm GC (XXX kill the following code at some point)
- scale_of_field, basesize, ofs_length, _ = (
+ itemsize, basesize, ofs_length, _, _ = (
self._unpack_arraydescr(op.getdescr()))
+ scale_of_field = _get_scale(itemsize)
return self._malloc_varsize(basesize, ofs_length, scale_of_field,
op.getarg(0), op.result)
@@ -829,21 +832,19 @@
ofs = arraydescr.get_base_size(self.translate_support_code)
size = arraydescr.get_item_size(self.translate_support_code)
ptr = arraydescr.is_array_of_pointers()
- scale = 0
- while (1 << scale) < size:
- scale += 1
- assert (1 << scale) == size
- return scale, ofs, ofs_length, ptr
+ sign = arraydescr.is_item_signed()
+ return size, ofs, ofs_length, ptr, sign
def _unpack_fielddescr(self, fielddescr):
assert isinstance(fielddescr, BaseFieldDescr)
ofs = fielddescr.offset
size = fielddescr.get_field_size(self.translate_support_code)
ptr = fielddescr.is_pointer_field()
- return imm(ofs), imm(size), ptr
+ sign = fielddescr.is_field_signed()
+ return imm(ofs), imm(size), ptr, sign
def consider_setfield_gc(self, op):
- ofs_loc, size_loc, ptr = self._unpack_fielddescr(op.getdescr())
+ ofs_loc, size_loc, _, _ = self._unpack_fielddescr(op.getdescr())
assert isinstance(size_loc, ImmedLoc)
if size_loc.value == 1:
need_lower_byte = True
@@ -870,10 +871,10 @@
consider_unicodesetitem = consider_strsetitem
def consider_setarrayitem_gc(self, op):
- scale, ofs, _, ptr = self._unpack_arraydescr(op.getdescr())
+ itemsize, ofs, _, _, _ = self._unpack_arraydescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
- if scale == 0:
+ if itemsize == 1:
need_lower_byte = True
else:
need_lower_byte = False
@@ -882,30 +883,39 @@
ofs_loc = self.rm.make_sure_var_in_reg(op.getarg(1), args)
self.possibly_free_vars(args)
self.PerformDiscard(op, [base_loc, ofs_loc, value_loc,
- imm(scale), imm(ofs)])
+ imm(itemsize), imm(ofs)])
consider_setarrayitem_raw = consider_setarrayitem_gc
def consider_getfield_gc(self, op):
- ofs_loc, size_loc, _ = self._unpack_fielddescr(op.getdescr())
+ ofs_loc, size_loc, _, sign = self._unpack_fielddescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
self.rm.possibly_free_vars(args)
result_loc = self.force_allocate_reg(op.result)
- self.Perform(op, [base_loc, ofs_loc, size_loc], result_loc)
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self.Perform(op, [base_loc, ofs_loc, size_loc, sign_loc], result_loc)
consider_getfield_raw = consider_getfield_gc
consider_getfield_raw_pure = consider_getfield_gc
consider_getfield_gc_pure = consider_getfield_gc
def consider_getarrayitem_gc(self, op):
- scale, ofs, _, _ = self._unpack_arraydescr(op.getdescr())
+ itemsize, ofs, _, _, sign = self._unpack_arraydescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
ofs_loc = self.rm.make_sure_var_in_reg(op.getarg(1), args)
self.rm.possibly_free_vars_for_op(op)
result_loc = self.force_allocate_reg(op.result)
- self.Perform(op, [base_loc, ofs_loc, imm(scale), imm(ofs)], result_loc)
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self.Perform(op, [base_loc, ofs_loc, imm(itemsize), imm(ofs),
+ sign_loc], result_loc)
consider_getarrayitem_raw = consider_getarrayitem_gc
consider_getarrayitem_gc_pure = consider_getarrayitem_gc
@@ -959,6 +969,12 @@
consider_unicodegetitem = consider_strgetitem
def consider_copystrcontent(self, op):
+ self._consider_copystrcontent(op, is_unicode=False)
+
+ def consider_copyunicodecontent(self, op):
+ self._consider_copystrcontent(op, is_unicode=True)
+
+ def _consider_copystrcontent(self, op, is_unicode):
# compute the source address
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(args[0], args)
@@ -970,7 +986,8 @@
srcaddr_box = TempBox()
forbidden_vars = [args[1], args[3], args[4], srcaddr_box]
srcaddr_loc = self.rm.force_allocate_reg(srcaddr_box, forbidden_vars)
- self._gen_address_inside_string(base_loc, ofs_loc, srcaddr_loc)
+ self._gen_address_inside_string(base_loc, ofs_loc, srcaddr_loc,
+ is_unicode=is_unicode)
# compute the destination address
base_loc = self.rm.make_sure_var_in_reg(args[1], forbidden_vars)
ofs_loc = self.rm.make_sure_var_in_reg(args[3], forbidden_vars)
@@ -980,25 +997,57 @@
forbidden_vars = [args[4], srcaddr_box]
dstaddr_box = TempBox()
dstaddr_loc = self.rm.force_allocate_reg(dstaddr_box, forbidden_vars)
- self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc)
+ self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc,
+ is_unicode=is_unicode)
+ # compute the length in bytes
+ length_box = args[4]
+ length_loc = self.loc(length_box)
+ if is_unicode:
+ self.rm.possibly_free_var(length_box)
+ forbidden_vars = [srcaddr_box, dstaddr_box]
+ bytes_box = TempBox()
+ bytes_loc = self.rm.force_allocate_reg(bytes_box, forbidden_vars)
+ scale = self._get_unicode_item_scale()
+ if not (isinstance(length_loc, ImmedLoc) or
+ isinstance(length_loc, RegLoc)):
+ self.assembler.mov(length_loc, bytes_loc)
+ length_loc = bytes_loc
+ self.assembler.load_effective_addr(length_loc, 0, scale, bytes_loc)
+ length_box = bytes_box
+ length_loc = bytes_loc
# call memcpy()
- length_loc = self.loc(args[4])
self.rm.before_call()
self.xrm.before_call()
self.assembler._emit_call(imm(self.assembler.memcpy_addr),
[dstaddr_loc, srcaddr_loc, length_loc])
- self.rm.possibly_free_var(args[4])
+ self.rm.possibly_free_var(length_box)
self.rm.possibly_free_var(dstaddr_box)
self.rm.possibly_free_var(srcaddr_box)
- def _gen_address_inside_string(self, baseloc, ofsloc, resloc):
+ def _gen_address_inside_string(self, baseloc, ofsloc, resloc, is_unicode):
cpu = self.assembler.cpu
- ofs_items, itemsize, _ = symbolic.get_array_token(rstr.STR,
+ if is_unicode:
+ ofs_items, _, _ = symbolic.get_array_token(rstr.UNICODE,
self.translate_support_code)
- assert itemsize == 1
- self.assembler.load_effective_addr(ofsloc, ofs_items, 0,
+ scale = self._get_unicode_item_scale()
+ else:
+ ofs_items, itemsize, _ = symbolic.get_array_token(rstr.STR,
+ self.translate_support_code)
+ assert itemsize == 1
+ scale = 0
+ self.assembler.load_effective_addr(ofsloc, ofs_items, scale,
resloc, baseloc)
+ def _get_unicode_item_scale(self):
+ _, itemsize, _ = symbolic.get_array_token(rstr.UNICODE,
+ self.translate_support_code)
+ if itemsize == 4:
+ return 2
+ elif itemsize == 2:
+ return 1
+ else:
+ raise AssertionError("bad unicode item size")
+
def consider_jump(self, op):
assembler = self.assembler
assert self.jump_target_descr is None
@@ -1033,6 +1082,9 @@
def consider_debug_merge_point(self, op):
pass
+ def consider_jit_debug(self, op):
+ pass
+
def get_mark_gc_roots(self, gcrootmap):
shape = gcrootmap.get_basic_shape(IS_X86_64)
for v, val in self.fm.frame_bindings.items():
@@ -1052,15 +1104,11 @@
self.Perform(op, [], loc)
def not_implemented_op(self, op):
- msg = "[regalloc] Not implemented operation: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation: %s" % op.getopname())
def not_implemented_op_with_guard(self, op, guard_op):
- msg = "[regalloc] Not implemented operation with guard: %s" % (
- op.getopname(),)
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation with guard: %s" % (
+ op.getopname(),))
oplist = [RegAlloc.not_implemented_op] * rop._LAST
oplist_with_guard = [RegAlloc.not_implemented_op_with_guard] * rop._LAST
@@ -1094,3 +1142,14 @@
# Returns (ebp-20), (ebp-24), (ebp-28)...
# i.e. the n'th word beyond the fixed frame size.
return -WORD * (FRAME_FIXED_SIZE + position)
+
+def _get_scale(size):
+ assert size == 1 or size == 2 or size == 4 or size == 8
+ if size < 4:
+ return size - 1 # 1, 2 => 0, 1
+ else:
+ return (size >> 2) + 1 # 4, 8 => 2, 3
+
+def not_implemented(msg):
+ os.write(2, '[x86/regalloc] %s\n' % msg)
+ raise NotImplementedError(msg)
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/regloc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/regloc.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/regloc.py Fri Oct 22 23:09:43 2010
@@ -442,8 +442,11 @@
MOV8 = _binaryop('MOV8')
MOV16 = _16_bit_binaryop('MOV')
MOVZX8 = _binaryop('MOVZX8')
+ MOVSX8 = _binaryop('MOVSX8')
MOVZX16 = _binaryop('MOVZX16')
+ MOVSX16 = _binaryop('MOVSX16')
MOV32 = _binaryop('MOV32')
+ MOVSX32 = _binaryop('MOVSX32')
XCHG = _binaryop('XCHG')
PUSH = _unaryop('PUSH')
@@ -473,6 +476,9 @@
else:
return ImmedLoc(x)
+imm0 = imm(0)
+imm1 = imm(1)
+
all_extra_instructions = [name for name in LocationCodeBuilder.__dict__
if name[0].isupper()]
all_extra_instructions.sort()
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py Fri Oct 22 23:09:43 2010
@@ -113,7 +113,8 @@
return CPU386.cast_adr_to_int(adr)
all_null_registers = lltype.malloc(rffi.LONGP.TO, 24,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
def force(self, addr_of_force_index):
TP = rffi.CArrayPtr(lltype.Signed)
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/rx86.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/rx86.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/rx86.py Fri Oct 22 23:09:43 2010
@@ -642,7 +642,10 @@
define_modrm_modes('MOV8_*i', [rex_w, '\xC6', orbyte(0<<3)], [immediate(2, 'b')], regtype='BYTE')
define_modrm_modes('MOVZX8_r*', [rex_w, '\x0F\xB6', register(1, 8)], regtype='BYTE')
+define_modrm_modes('MOVSX8_r*', [rex_w, '\x0F\xBE', register(1, 8)], regtype='BYTE')
define_modrm_modes('MOVZX16_r*', [rex_w, '\x0F\xB7', register(1, 8)])
+define_modrm_modes('MOVSX16_r*', [rex_w, '\x0F\xBF', register(1, 8)])
+define_modrm_modes('MOVSX32_r*', [rex_w, '\x63', register(1, 8)])
define_modrm_modes('MOVSD_x*', ['\xF2', rex_nw, '\x0F\x10', register(1,8)], regtype='XMM')
define_modrm_modes('MOVSD_*x', ['\xF2', rex_nw, '\x0F\x11', register(2,8)], regtype='XMM')
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_assembler.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_assembler.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_assembler.py Fri Oct 22 23:09:43 2010
@@ -81,7 +81,8 @@
# also test rebuild_faillocs_from_descr(), which should not
# reproduce the holes at all
- bytecode = lltype.malloc(rffi.UCHARP.TO, len(mc.content), flavor='raw')
+ bytecode = lltype.malloc(rffi.UCHARP.TO, len(mc.content), flavor='raw',
+ immortal=True)
for i in range(len(mc.content)):
assert 0 <= mc.content[i] <= 255
bytecode[i] = rffi.cast(rffi.UCHAR, mc.content[i])
@@ -115,7 +116,8 @@
assert withfloats
value = random.random() - 0.5
# make sure it fits into 64 bits
- tmp = lltype.malloc(rffi.LONGP.TO, 2, flavor='raw')
+ tmp = lltype.malloc(rffi.LONGP.TO, 2, flavor='raw',
+ track_allocation=False)
rffi.cast(rffi.DOUBLEP, tmp)[0] = value
return rffi.cast(rffi.DOUBLEP, tmp)[0], tmp[0], tmp[1]
@@ -152,10 +154,12 @@
# prepare the expected target arrays, the descr_bytecode,
# the 'registers' and the 'stack' arrays according to 'content'
- xmmregisters = lltype.malloc(rffi.LONGP.TO, 16+ACTUAL_CPU.NUM_REGS+1, flavor='raw')
+ xmmregisters = lltype.malloc(rffi.LONGP.TO, 16+ACTUAL_CPU.NUM_REGS+1,
+ flavor='raw', immortal=True)
registers = rffi.ptradd(xmmregisters, 16)
stacklen = baseloc + 10
- stack = lltype.malloc(rffi.LONGP.TO, stacklen, flavor='raw')
+ stack = lltype.malloc(rffi.LONGP.TO, stacklen, flavor='raw',
+ immortal=True)
expected_ints = [0] * len(content)
expected_ptrs = [lltype.nullptr(llmemory.GCREF.TO)] * len(content)
expected_floats = [0.0] * len(content)
@@ -221,7 +225,7 @@
descr_bytecode.append(0x00)
descr_bytecode.append(0xCC) # end marker
descr_bytes = lltype.malloc(rffi.UCHARP.TO, len(descr_bytecode),
- flavor='raw')
+ flavor='raw', immortal=True)
for i in range(len(descr_bytecode)):
assert 0 <= descr_bytecode[i] <= 255
descr_bytes[i] = rffi.cast(rffi.UCHAR, descr_bytecode[i])
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_gc_integration.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_gc_integration.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_gc_integration.py Fri Oct 22 23:09:43 2010
@@ -12,7 +12,7 @@
from pypy.jit.backend.detect_cpu import getcpuclass
from pypy.jit.backend.x86.regalloc import RegAlloc
from pypy.jit.backend.x86.arch import WORD, FRAME_FIXED_SIZE
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.rpython.lltypesystem import lltype, llmemory, rffi
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.lltypesystem import rclass, rstr
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_regalloc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_regalloc.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_regalloc.py Fri Oct 22 23:09:43 2010
@@ -11,7 +11,7 @@
from pypy.jit.backend.x86.regalloc import RegAlloc, X86RegisterManager,\
FloatConstants, is_comparison_or_ovf_op
from pypy.jit.backend.x86.arch import IS_X86_32, IS_X86_64
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.rpython.lltypesystem import lltype, llmemory, rffi
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.lltypesystem import rclass, rstr
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py Fri Oct 22 23:09:43 2010
@@ -10,7 +10,7 @@
from pypy.jit.metainterp.resoperation import rop
from pypy.jit.metainterp.executor import execute
from pypy.jit.backend.test.runner_test import LLtypeBackendTest
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.tool.udir import udir
import ctypes
import sys
@@ -506,8 +506,8 @@
self.cpu.execute_token(ops.token)
# check debugging info
name, struct = self.cpu.assembler.loop_run_counters[0]
- assert name == 'xyz'
+ assert name == 0 # 'xyz'
assert struct.i == 10
self.cpu.finish_once()
lines = py.path.local(self.logfile + ".count").readlines()
- assert lines[0] == '10 xyz\n'
+ assert lines[0] == '0:10\n' # '10 xyz\n'
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_string.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_string.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_string.py Fri Oct 22 23:09:43 2010
@@ -2,8 +2,12 @@
from pypy.jit.metainterp.test import test_string
from pypy.jit.backend.x86.test.test_basic import Jit386Mixin
-class TestString(Jit386Mixin, test_string.StringTests):
+class TestString(Jit386Mixin, test_string.TestLLtype):
# for the individual tests see
# ====> ../../../metainterp/test/test_string.py
- CALL = 'call'
- CALL_PURE = 'call_pure'
+ pass
+
+class TestUnicode(Jit386Mixin, test_string.TestLLtypeUnicode):
+ # for the individual tests see
+ # ====> ../../../metainterp/test/test_string.py
+ pass
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_zrpy_gc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_zrpy_gc.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_zrpy_gc.py Fri Oct 22 23:09:43 2010
@@ -191,6 +191,33 @@
def run_orig(self, name, n, x):
self.main_allfuncs(name, n, x)
+ def define_libffi_workaround(cls):
+ # XXX: this is a workaround for a bug in database.py. It seems that
+ # the problem is triggered by optimizeopt/fficall.py, and in
+ # particular by the ``cast_base_ptr_to_instance(Func, llfunc)``: in
+ # these tests, that line is the only place where libffi.Func is
+ # referenced.
+ #
+ # The problem occurs because the gctransformer tries to annotate a
+ # low-level helper to call the __del__ of libffi.Func when it's too
+ # late.
+ #
+ # This workaround works by forcing the annotator (and all the rest of
+ # the toolchain) to see libffi.Func in a "proper" context, not just as
+ # the target of cast_base_ptr_to_instance. Note that the function
+ # below is *never* called by any actual test, it's just annotated.
+ #
+ from pypy.rlib.libffi import get_libc_name, CDLL, types, ArgChain
+ libc_name = get_libc_name()
+ def f(n, x, *args):
+ libc = CDLL(libc_name)
+ ptr = libc.getpointer('labs', [types.slong], types.slong)
+ chain = ArgChain()
+ chain.arg(n)
+ n = ptr.call(chain, lltype.Signed)
+ return (n, x) + args
+ return None, f, None
+
def define_compile_framework_1(cls):
# a moving GC. Supports malloc_varsize_nonmovable. Simple test, works
# without write_barriers and root stack enumeration.
Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py (original)
+++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py Fri Oct 22 23:09:43 2010
@@ -1,4 +1,4 @@
-import py, os
+import py, os, sys
from pypy.tool.udir import udir
from pypy.rlib.jit import JitDriver, OPTIMIZER_FULL, unroll_parameters
from pypy.rlib.jit import PARAMETERS, dont_look_inside
@@ -63,8 +63,32 @@
if k - abs(j): raise ValueError
if k - abs(-j): raise ValueError
return total * 10
+ #
+ from pypy.rpython.lltypesystem import lltype, rffi
+ from pypy.rlib.libffi import types, CDLL, ArgChain
+ from pypy.rlib.test.test_libffi import get_libm_name
+ libm_name = get_libm_name(sys.platform)
+ jitdriver2 = JitDriver(greens=[], reds = ['i', 'func', 'res', 'x'])
+ def libffi_stuff(i, j):
+ lib = CDLL(libm_name)
+ func = lib.getpointer('fabs', [types.double], types.double)
+ res = 0.0
+ x = float(j)
+ while i > 0:
+ jitdriver2.jit_merge_point(i=i, res=res, func=func, x=x)
+ jitdriver2.can_enter_jit(i=i, res=res, func=func, x=x)
+ func = hint(func, promote=True)
+ argchain = ArgChain()
+ argchain.arg(x)
+ res = func.call(argchain, rffi.DOUBLE)
+ i -= 1
+ return res
+ #
+ def main(i, j):
+ return f(i, j) + libffi_stuff(i, j)
+ expected = f(40, -49)
res = self.meta_interp(f, [40, -49])
- assert res == f(40, -49)
+ assert res == expected
def test_direct_assembler_call_translates(self):
class Thing(object):
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/call.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/call.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/call.py Fri Oct 22 23:09:43 2010
@@ -277,3 +277,11 @@
return seen.pop()
else:
return None
+
+ def could_be_green_field(self, GTYPE, fieldname):
+ GTYPE_fieldname = (GTYPE, fieldname)
+ for jd in self.jitdrivers_sd:
+ if jd.greenfield_info is not None:
+ if GTYPE_fieldname in jd.greenfield_info.green_fields:
+ return True
+ return False
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/codewriter.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/codewriter.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/codewriter.py Fri Oct 22 23:09:43 2010
@@ -95,18 +95,18 @@
print '%s:' % (ssarepr.name,)
print format_assembler(ssarepr)
else:
- dir = udir.ensure("jitcodes", dir=1)
- if portal_jitdriver:
- name = "%02d_portal_runner" % (portal_jitdriver.index,)
- elif ssarepr.name and ssarepr.name != '?':
- name = ssarepr.name
- else:
- name = 'unnamed' % id(ssarepr)
- i = 1
- extra = ''
- while name+extra in self._seen_files:
- i += 1
- extra = '.%d' % i
- self._seen_files.add(name+extra)
- dir.join(name+extra).write(format_assembler(ssarepr))
log.dot()
+ dir = udir.ensure("jitcodes", dir=1)
+ if portal_jitdriver:
+ name = "%02d_portal_runner" % (portal_jitdriver.index,)
+ elif ssarepr.name and ssarepr.name != '?':
+ name = ssarepr.name
+ else:
+ name = 'unnamed' % id(ssarepr)
+ i = 1
+ extra = ''
+ while name+extra in self._seen_files:
+ i += 1
+ extra = '.%d' % i
+ self._seen_files.add(name+extra)
+ dir.join(name+extra).write(format_assembler(ssarepr))
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/effectinfo.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/effectinfo.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/effectinfo.py Fri Oct 22 23:09:43 2010
@@ -18,19 +18,34 @@
# the 'oopspecindex' field is one of the following values:
OS_NONE = 0 # normal case, no oopspec
OS_ARRAYCOPY = 1 # "list.ll_arraycopy"
- OS_STR_CONCAT = 2 # "stroruni.concat"
- OS_UNI_CONCAT = 3 # "stroruni.concat"
- OS_STR_SLICE = 4 # "stroruni.slice"
- OS_UNI_SLICE = 5 # "stroruni.slice"
- OS_STR_EQUAL = 6 # "stroruni.equal"
- OS_UNI_EQUAL = 7 # "stroruni.equal"
- OS_STREQ_SLICE_CHECKNULL = 8 # s2!=NULL and s1[x:x+length]==s2
- OS_STREQ_SLICE_NONNULL = 9 # s1[x:x+length]==s2 (assert s2!=NULL)
- OS_STREQ_SLICE_CHAR = 10 # s1[x:x+length]==char
- OS_STREQ_NONNULL = 11 # s1 == s2 (assert s1!=NULL,s2!=NULL)
- OS_STREQ_NONNULL_CHAR = 12 # s1 == char (assert s1!=NULL)
- OS_STREQ_CHECKNULL_CHAR = 13 # s1!=NULL and s1==char
- OS_STREQ_LENGTHOK = 14 # s1 == s2 (assert len(s1)==len(s2))
+ OS_STR2UNICODE = 2 # "str.str2unicode"
+ #
+ OS_STR_CONCAT = 22 # "stroruni.concat"
+ OS_STR_SLICE = 23 # "stroruni.slice"
+ OS_STR_EQUAL = 24 # "stroruni.equal"
+ OS_STREQ_SLICE_CHECKNULL = 25 # s2!=NULL and s1[x:x+length]==s2
+ OS_STREQ_SLICE_NONNULL = 26 # s1[x:x+length]==s2 (assert s2!=NULL)
+ OS_STREQ_SLICE_CHAR = 27 # s1[x:x+length]==char
+ OS_STREQ_NONNULL = 28 # s1 == s2 (assert s1!=NULL,s2!=NULL)
+ OS_STREQ_NONNULL_CHAR = 29 # s1 == char (assert s1!=NULL)
+ OS_STREQ_CHECKNULL_CHAR = 30 # s1!=NULL and s1==char
+ OS_STREQ_LENGTHOK = 31 # s1 == s2 (assert len(s1)==len(s2))
+ #
+ OS_UNI_CONCAT = 42 #
+ OS_UNI_SLICE = 43 #
+ OS_UNI_EQUAL = 44 #
+ OS_UNIEQ_SLICE_CHECKNULL = 45 #
+ OS_UNIEQ_SLICE_NONNULL = 46 #
+ OS_UNIEQ_SLICE_CHAR = 47 #
+ OS_UNIEQ_NONNULL = 48 # the same for unicode
+ OS_UNIEQ_NONNULL_CHAR = 49 # (must be the same amount as for
+ OS_UNIEQ_CHECKNULL_CHAR = 50 # STR, in the same order)
+ OS_UNIEQ_LENGTHOK = 51 #
+ _OS_offset_uni = OS_UNI_CONCAT - OS_STR_CONCAT
+ #
+ OS_LIBFFI_PREPARE = 60
+ OS_LIBFFI_PUSH_ARG = 61
+ OS_LIBFFI_CALL = 62
def __new__(cls, readonly_descrs_fields,
write_descrs_fields, write_descrs_arrays,
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py Fri Oct 22 23:09:43 2010
@@ -316,8 +316,14 @@
prepare = self._handle_list_call
elif oopspec_name.startswith('stroruni.'):
prepare = self._handle_stroruni_call
+ elif oopspec_name == 'str.str2unicode':
+ prepare = self._handle_str2unicode_call
elif oopspec_name.startswith('virtual_ref'):
prepare = self._handle_virtual_ref_call
+ elif oopspec_name.startswith('jit.'):
+ prepare = self._handle_jit_call
+ elif oopspec_name.startswith('libffi_'):
+ prepare = self._handle_libffi_call
else:
prepare = self.prepare_builtin_call
try:
@@ -427,7 +433,8 @@
op.result)
def rewrite_op_free(self, op):
- assert op.args[1].value == 'raw'
+ flags = op.args[1].value
+ assert flags['flavor'] == 'raw'
ARRAY = op.args[0].concretetype.TO
return self._do_builtin_call(op, 'raw_free', [op.args[0]],
extra = (ARRAY,), extrakey = ARRAY)
@@ -519,7 +526,12 @@
# check for deepfrozen structures that force constant-folding
immut = v_inst.concretetype.TO._immutable_field(c_fieldname.value)
if immut:
- pure = '_pure'
+ if (self.callcontrol is not None and
+ self.callcontrol.could_be_green_field(v_inst.concretetype.TO,
+ c_fieldname.value)):
+ pure = '_greenfield'
+ else:
+ pure = '_pure'
if immut == "[*]":
self.immutable_arrays[op.result] = True
else:
@@ -819,6 +831,8 @@
def rewrite_op_jit_marker(self, op):
key = op.args[0].value
jitdriver = op.args[1].value
+ if not jitdriver.active:
+ return []
return getattr(self, 'handle_jit_marker__%s' % key)(op, jitdriver)
def handle_jit_marker__jit_merge_point(self, op, jitdriver):
@@ -852,6 +866,15 @@
(self.graph,))
return []
+ def _handle_jit_call(self, op, oopspec_name, args):
+ if oopspec_name == 'jit.debug':
+ return SpaceOperation('jit_debug', args, None)
+ elif oopspec_name == 'jit.assert_green':
+ kind = getkind(args[0].concretetype)
+ return SpaceOperation('%s_assert_green' % kind, args, None)
+ else:
+ raise AssertionError("missing support for %r" % oopspec_name)
+
# ----------
# Lists.
@@ -1028,8 +1051,10 @@
# ----------
# Strings and Unicodes.
- def _handle_oopspec_call(self, op, args, oopspecindex):
+ def _handle_oopspec_call(self, op, args, oopspecindex, extraeffect=None):
calldescr = self.callcontrol.getcalldescr(op, oopspecindex)
+ if extraeffect:
+ calldescr.get_extra_info().extraeffect = extraeffect
if isinstance(op.args[0].value, str):
pass # for tests only
else:
@@ -1055,28 +1080,32 @@
[c_func] + [varoftype(T) for T in argtypes],
varoftype(resulttype))
calldescr = self.callcontrol.getcalldescr(op, oopspecindex)
- func = heaptracker.adr2int(
- llmemory.cast_ptr_to_adr(c_func.value))
+ if isinstance(c_func.value, str): # in tests only
+ func = c_func.value
+ else:
+ func = heaptracker.adr2int(
+ llmemory.cast_ptr_to_adr(c_func.value))
_callinfo_for_oopspec[oopspecindex] = calldescr, func
def _handle_stroruni_call(self, op, oopspec_name, args):
- if args[0].concretetype.TO == rstr.STR:
+ SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE)
+ if SoU.TO == rstr.STR:
dict = {"stroruni.concat": EffectInfo.OS_STR_CONCAT,
"stroruni.slice": EffectInfo.OS_STR_SLICE,
"stroruni.equal": EffectInfo.OS_STR_EQUAL,
}
- elif args[0].concretetype.TO == rstr.UNICODE:
+ CHR = lltype.Char
+ elif SoU.TO == rstr.UNICODE:
dict = {"stroruni.concat": EffectInfo.OS_UNI_CONCAT,
"stroruni.slice": EffectInfo.OS_UNI_SLICE,
"stroruni.equal": EffectInfo.OS_UNI_EQUAL,
}
+ CHR = lltype.UniChar
else:
assert 0, "args[0].concretetype must be STR or UNICODE"
#
if oopspec_name == "stroruni.equal":
- SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE)
for otherindex, othername, argtypes, resulttype in [
-
(EffectInfo.OS_STREQ_SLICE_CHECKNULL,
"str.eq_slice_checknull",
[SoU, lltype.Signed, lltype.Signed, SoU],
@@ -1087,7 +1116,7 @@
lltype.Signed),
(EffectInfo.OS_STREQ_SLICE_CHAR,
"str.eq_slice_char",
- [SoU, lltype.Signed, lltype.Signed, lltype.Char],
+ [SoU, lltype.Signed, lltype.Signed, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_NONNULL,
"str.eq_nonnull",
@@ -1095,22 +1124,27 @@
lltype.Signed),
(EffectInfo.OS_STREQ_NONNULL_CHAR,
"str.eq_nonnull_char",
- [SoU, lltype.Char],
+ [SoU, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_CHECKNULL_CHAR,
"str.eq_checknull_char",
- [SoU, lltype.Char],
+ [SoU, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_LENGTHOK,
"str.eq_lengthok",
[SoU, SoU],
lltype.Signed),
]:
+ if args[0].concretetype.TO == rstr.UNICODE:
+ otherindex += EffectInfo._OS_offset_uni
self._register_extra_helper(otherindex, othername,
argtypes, resulttype)
#
return self._handle_oopspec_call(op, args, dict[oopspec_name])
+ def _handle_str2unicode_call(self, op, oopspec_name, args):
+ return self._handle_oopspec_call(op, args, EffectInfo.OS_STR2UNICODE)
+
# ----------
# VirtualRefs.
@@ -1121,6 +1155,23 @@
vrefinfo.JIT_VIRTUAL_REF)
return SpaceOperation(oopspec_name, list(args), op.result)
+ # -----------
+ # rlib.libffi
+
+ def _handle_libffi_call(self, op, oopspec_name, args):
+ if oopspec_name == 'libffi_prepare_call':
+ oopspecindex = EffectInfo.OS_LIBFFI_PREPARE
+ extraeffect = EffectInfo.EF_CANNOT_RAISE
+ elif oopspec_name.startswith('libffi_push_'):
+ oopspecindex = EffectInfo.OS_LIBFFI_PUSH_ARG
+ extraeffect = EffectInfo.EF_CANNOT_RAISE
+ elif oopspec_name.startswith('libffi_call_'):
+ oopspecindex = EffectInfo.OS_LIBFFI_CALL
+ extraeffect = EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE
+ else:
+ assert False, 'unsupported oopspec: %s' % oopspec_name
+ return self._handle_oopspec_call(op, args, oopspecindex, extraeffect)
+
def rewrite_op_jit_force_virtual(self, op):
return self._do_builtin_call(op)
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/support.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/support.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/support.py Fri Oct 22 23:09:43 2010
@@ -1,5 +1,5 @@
import sys
-from pypy.rpython.lltypesystem import lltype, rclass
+from pypy.rpython.lltypesystem import lltype, rclass, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython import rlist
from pypy.rpython.lltypesystem import rstr as ll_rstr, rdict as ll_rdict
@@ -8,6 +8,7 @@
from pypy.rpython.ootypesystem import rdict as oo_rdict
from pypy.rpython.llinterp import LLInterpreter
from pypy.rpython.extregistry import ExtRegistryEntry
+from pypy.rpython.annlowlevel import cast_base_ptr_to_instance
from pypy.translator.simplify import get_funcobj
from pypy.translator.unsimplify import split_block
from pypy.objspace.flow.model import Constant
@@ -60,7 +61,7 @@
return rtyper.annotator.translator.graphs[0]
def split_before_jit_merge_point(graph, portalblock, portalopindex):
- """Find the block with 'jit_merge_point' and split just before,
+ """Split the block just before the 'jit_merge_point',
making sure the input args are in the canonical order.
"""
# split the block just before the jit_merge_point()
@@ -217,6 +218,33 @@
else:
return x
+
+# libffi support
+# --------------
+
+def func(llfunc):
+ from pypy.rlib.libffi import Func
+ return cast_base_ptr_to_instance(Func, llfunc)
+
+def _ll_1_libffi_prepare_call(llfunc):
+ return func(llfunc)._prepare()
+
+def _ll_4_libffi_push_int(llfunc, value, ll_args, i):
+ return func(llfunc)._push_int(value, ll_args, i)
+
+def _ll_4_libffi_push_float(llfunc, value, ll_args, i):
+ return func(llfunc)._push_float(value, ll_args, i)
+
+def _ll_3_libffi_call_int(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, rffi.LONG)
+
+def _ll_3_libffi_call_float(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, rffi.DOUBLE)
+
+def _ll_3_libffi_call_void(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, lltype.Void)
+
+
# in the following calls to builtins, the JIT is allowed to look inside:
inline_calls_to = [
('int_floordiv_ovf_zer', [lltype.Signed, lltype.Signed], lltype.Signed),
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/test/test_codewriter.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/test/test_codewriter.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/test/test_codewriter.py Fri Oct 22 23:09:43 2010
@@ -45,6 +45,7 @@
self.portal_graph = portal_graph
self.portal_runner_ptr = "???"
self.virtualizable_info = None
+ self.greenfield_info = None
def test_loop():
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/test/test_effectinfo.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/test/test_effectinfo.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/test/test_effectinfo.py Fri Oct 22 23:09:43 2010
@@ -1,7 +1,8 @@
from pypy.rpython.lltypesystem.rclass import OBJECT
from pypy.rpython.lltypesystem import lltype
from pypy.rpython.ootypesystem import ootype
-from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze
+from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze,\
+ EffectInfo
class FakeCPU:
def fielddescrof(self, T, fieldname):
@@ -9,6 +10,14 @@
def arraydescrof(self, A):
return ('arraydescr', A)
+def test_no_oopspec_duplicate():
+ # check that all the various EffectInfo.OS_* have unique values
+ oopspecs = set()
+ for name, value in EffectInfo.__dict__.iteritems():
+ if name.startswith('OS_'):
+ assert value not in oopspecs
+ oopspecs.add(value)
+
def test_include_read_field():
S = lltype.GcStruct("S", ("a", lltype.Signed))
effects = frozenset([("readstruct", lltype.Ptr(S), "a")])
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py Fri Oct 22 23:09:43 2010
@@ -77,7 +77,32 @@
class FakeBuiltinCallControl:
def guess_call_kind(self, op):
return 'builtin'
- def getcalldescr(self, op, oopspecindex):
+ def getcalldescr(self, op, oopspecindex=None):
+ assert oopspecindex is not None # in this test
+ EI = effectinfo.EffectInfo
+ if oopspecindex != EI.OS_ARRAYCOPY:
+ PSTR = lltype.Ptr(rstr.STR)
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ INT = lltype.Signed
+ UNICHAR = lltype.UniChar
+ argtypes = {
+ EI.OS_STR2UNICODE:([PSTR], PUNICODE),
+ EI.OS_STR_CONCAT: ([PSTR, PSTR], PSTR),
+ EI.OS_STR_SLICE: ([PSTR, INT, INT], PSTR),
+ EI.OS_UNI_CONCAT: ([PUNICODE, PUNICODE], PUNICODE),
+ EI.OS_UNI_SLICE: ([PUNICODE, INT, INT], PUNICODE),
+ EI.OS_UNI_EQUAL: ([PUNICODE, PUNICODE], lltype.Bool),
+ EI.OS_UNIEQ_SLICE_CHECKNULL:([PUNICODE, INT, INT, PUNICODE], INT),
+ EI.OS_UNIEQ_SLICE_NONNULL: ([PUNICODE, INT, INT, PUNICODE], INT),
+ EI.OS_UNIEQ_SLICE_CHAR: ([PUNICODE, INT, INT, UNICHAR], INT),
+ EI.OS_UNIEQ_NONNULL: ([PUNICODE, PUNICODE], INT),
+ EI.OS_UNIEQ_NONNULL_CHAR: ([PUNICODE, UNICHAR], INT),
+ EI.OS_UNIEQ_CHECKNULL_CHAR: ([PUNICODE, UNICHAR], INT),
+ EI.OS_UNIEQ_LENGTHOK: ([PUNICODE, PUNICODE], INT),
+ }
+ argtypes = argtypes[oopspecindex]
+ assert argtypes[0] == [v.concretetype for v in op.args[1:]]
+ assert argtypes[1] == op.result.concretetype
return 'calldescr-%d' % oopspecindex
def calldescr_canraise(self, calldescr):
return False
@@ -662,6 +687,79 @@
assert block.operations[1].result is None
assert block.exits[0].args == [v1]
+def test_jit_merge_point_1():
+ class FakeJitDriverSD:
+ index = 42
+ class jitdriver:
+ greens = ['green1', 'green2', 'voidgreen3']
+ reds = ['red1', 'red2', 'voidred3']
+ jd = FakeJitDriverSD()
+ v1 = varoftype(lltype.Signed)
+ v2 = varoftype(lltype.Signed)
+ vvoid1 = varoftype(lltype.Void)
+ v3 = varoftype(lltype.Signed)
+ v4 = varoftype(lltype.Signed)
+ vvoid2 = varoftype(lltype.Void)
+ v5 = varoftype(lltype.Void)
+ op = SpaceOperation('jit_marker',
+ [Constant('jit_merge_point', lltype.Void),
+ Constant(jd.jitdriver, lltype.Void),
+ v1, v2, vvoid1, v3, v4, vvoid2], v5)
+ tr = Transformer()
+ tr.portal_jd = jd
+ oplist = tr.rewrite_operation(op)
+ assert len(oplist) == 6
+ assert oplist[0].opname == '-live-'
+ assert oplist[1].opname == 'int_guard_value'
+ assert oplist[1].args == [v1]
+ assert oplist[2].opname == '-live-'
+ assert oplist[3].opname == 'int_guard_value'
+ assert oplist[3].args == [v2]
+ assert oplist[4].opname == 'jit_merge_point'
+ assert oplist[4].args[0].value == 42
+ assert list(oplist[4].args[1]) == [v1, v2]
+ assert list(oplist[4].args[4]) == [v3, v4]
+ assert oplist[5].opname == '-live-'
+
+def test_getfield_gc():
+ S = lltype.GcStruct('S', ('x', lltype.Char))
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
+def test_getfield_gc_pure():
+ S = lltype.GcStruct('S', ('x', lltype.Char),
+ hints={'immutable': True})
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i_pure'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
+def test_getfield_gc_greenfield():
+ class FakeCC:
+ def get_vinfo(self, v):
+ return None
+ def could_be_green_field(self, S1, name1):
+ assert S1 is S
+ assert name1 == 'x'
+ return True
+ S = lltype.GcStruct('S', ('x', lltype.Char),
+ hints={'immutable': True})
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU(), FakeCC()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i_greenfield'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
def test_int_abs():
v1 = varoftype(lltype.Signed)
v2 = varoftype(lltype.Signed)
@@ -766,6 +864,46 @@
assert op1.args[3] == ListOfKind('ref', [v1])
assert op1.result == v4
+def test_str2unicode():
+ # test that the oopspec is present and correctly transformed
+ PSTR = lltype.Ptr(rstr.STR)
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ FUNC = lltype.FuncType([PSTR], PUNICODE)
+ func = lltype.functionptr(FUNC, 'll_str2unicode',
+ _callable=rstr.LLHelpers.ll_str2unicode)
+ v1 = varoftype(PSTR)
+ v2 = varoftype(PUNICODE)
+ op = SpaceOperation('direct_call', [const(func), v1], v2)
+ tr = Transformer(FakeCPU(), FakeBuiltinCallControl())
+ op1 = tr.rewrite_operation(op)
+ assert op1.opname == 'residual_call_r_r'
+ assert op1.args[0].value == func
+ assert op1.args[1] == 'calldescr-%d' % effectinfo.EffectInfo.OS_STR2UNICODE
+ assert op1.args[2] == ListOfKind('ref', [v1])
+ assert op1.result == v2
+
+def test_unicode_eq_checknull_char():
+ # test that the oopspec is present and correctly transformed
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ FUNC = lltype.FuncType([PUNICODE, PUNICODE], lltype.Bool)
+ func = lltype.functionptr(FUNC, 'll_streq',
+ _callable=rstr.LLHelpers.ll_streq)
+ v1 = varoftype(PUNICODE)
+ v2 = varoftype(PUNICODE)
+ v3 = varoftype(lltype.Bool)
+ op = SpaceOperation('direct_call', [const(func), v1, v2], v3)
+ tr = Transformer(FakeCPU(), FakeBuiltinCallControl())
+ op1 = tr.rewrite_operation(op)
+ assert op1.opname == 'residual_call_r_i'
+ assert op1.args[0].value == func
+ assert op1.args[1] == 'calldescr-%d' % effectinfo.EffectInfo.OS_UNI_EQUAL
+ assert op1.args[2] == ListOfKind('ref', [v1, v2])
+ assert op1.result == v3
+ # test that the OS_UNIEQ_* functions are registered
+ cifo = effectinfo._callinfo_for_oopspec
+ assert effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL in cifo
+ assert effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR in cifo
+
def test_list_ll_arraycopy():
from pypy.rlib.rgc import ll_arraycopy
LIST = lltype.GcArray(lltype.Signed)
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/blackhole.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/blackhole.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/blackhole.py Fri Oct 22 23:09:43 2010
@@ -760,6 +760,20 @@
def bhimpl_debug_fatalerror(msg):
llop.debug_fatalerror(lltype.Void, msg)
+ @arguments("r", "i", "i", "i", "i")
+ def bhimpl_jit_debug(string, arg1=0, arg2=0, arg3=0, arg4=0):
+ pass
+
+ @arguments("i")
+ def bhimpl_int_assert_green(x):
+ pass
+ @arguments("r")
+ def bhimpl_ref_assert_green(x):
+ pass
+ @arguments("f")
+ def bhimpl_float_assert_green(x):
+ pass
+
# ----------
# the main hints and recursive calls
@@ -1073,6 +1087,10 @@
bhimpl_getfield_vable_r = bhimpl_getfield_gc_r
bhimpl_getfield_vable_f = bhimpl_getfield_gc_f
+ bhimpl_getfield_gc_i_greenfield = bhimpl_getfield_gc_i
+ bhimpl_getfield_gc_r_greenfield = bhimpl_getfield_gc_r
+ bhimpl_getfield_gc_f_greenfield = bhimpl_getfield_gc_f
+
@arguments("cpu", "i", "d", returns="i")
def bhimpl_getfield_raw_i(cpu, struct, fielddescr):
return cpu.bh_getfield_raw_i(struct, fielddescr)
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/compile.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/compile.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/compile.py Fri Oct 22 23:09:43 2010
@@ -370,7 +370,8 @@
from pypy.jit.metainterp.resume import force_from_resumedata
metainterp_sd = self.metainterp_sd
vinfo = self.jitdriver_sd.virtualizable_info
- all_virtuals = force_from_resumedata(metainterp_sd, self, vinfo)
+ ginfo = self.jitdriver_sd.greenfield_info
+ all_virtuals = force_from_resumedata(metainterp_sd, self, vinfo, ginfo)
# The virtualizable data was stored on the real virtualizable above.
# Handle all_virtuals: keep them for later blackholing from the
# future failure of the GUARD_NOT_FORCED
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/executor.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/executor.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/executor.py Fri Oct 22 23:09:43 2010
@@ -80,6 +80,9 @@
do_call_loopinvariant = do_call
do_call_may_force = do_call
+def do_call_c(cpu, metainterp, argboxes, descr):
+ raise NotImplementedError("Should never be called directly")
+
def do_getarrayitem_gc(cpu, _, arraybox, indexbox, arraydescr):
array = arraybox.getref_base()
index = indexbox.getint()
@@ -304,6 +307,7 @@
rop.CALL_ASSEMBLER,
rop.COND_CALL_GC_WB,
rop.DEBUG_MERGE_POINT,
+ rop.JIT_DEBUG,
rop.SETARRAYITEM_RAW,
): # list of opcodes never executed by pyjitpl
continue
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/graphpage.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/graphpage.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/graphpage.py Fri Oct 22 23:09:43 2010
@@ -153,7 +153,7 @@
opindex = opstartindex
while True:
op = operations[opindex]
- lines.append(repr(op))
+ lines.append(op.repr(graytext=True))
if is_interesting_guard(op):
tgt = op.getdescr()._debug_suboperations[0]
tgt_g, tgt_i = self.all_operations[tgt]
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/history.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/history.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/history.py Fri Oct 22 23:09:43 2010
@@ -698,6 +698,21 @@
return result
_const_ptr_for_string = {}
+def get_const_ptr_for_unicode(s):
+ from pypy.rpython.annlowlevel import llunicode
+ if not we_are_translated():
+ try:
+ return _const_ptr_for_unicode[s]
+ except KeyError:
+ pass
+ if isinstance(s, str):
+ s = unicode(s)
+ result = ConstPtr(lltype.cast_opaque_ptr(llmemory.GCREF, llunicode(s)))
+ if not we_are_translated():
+ _const_ptr_for_unicode[s] = result
+ return result
+_const_ptr_for_unicode = {}
+
# ____________________________________________________________
# The TreeLoop class contains a loop or a generalized loop, i.e. a tree
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/jitdriver.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/jitdriver.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/jitdriver.py Fri Oct 22 23:09:43 2010
@@ -13,8 +13,10 @@
# self.num_red_args ... pypy.jit.metainterp.warmspot
# self.result_type ... pypy.jit.metainterp.warmspot
# self.virtualizable_info... pypy.jit.metainterp.warmspot
+ # self.greenfield_info ... pypy.jit.metainterp.warmspot
# self.warmstate ... pypy.jit.metainterp.warmspot
# self.handle_jitexc_from_bh pypy.jit.metainterp.warmspot
+ # self.no_loop_header ... pypy.jit.metainterp.warmspot
# self.portal_finishtoken... pypy.jit.metainterp.pyjitpl
# self.index ... pypy.jit.codewriter.call
# self.mainjitcode ... pypy.jit.codewriter.call
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimize_nopspec.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/optimize_nopspec.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/optimize_nopspec.py Fri Oct 22 23:09:43 2010
@@ -14,6 +14,9 @@
def _optimize_loop(metainterp_sd, old_loop_tokens, loop):
cpu = metainterp_sd.cpu
metainterp_sd.logger_noopt.log_loop(loop.inputargs, loop.operations)
+ # XXX the following lines are probably still needed, to discard invalid
+ # loops. bit silly to run a full perfect specialization and throw the
+ # result away.
finder = PerfectSpecializationFinder(cpu)
finder.find_nodes_loop(loop, False)
if old_loop_tokens:
@@ -31,6 +34,7 @@
def _optimize_bridge(metainterp_sd, old_loop_tokens, bridge):
cpu = metainterp_sd.cpu
metainterp_sd.logger_noopt.log_loop(bridge.inputargs, bridge.operations)
+ # XXX same comment as above applies
finder = BridgeSpecializationFinder(cpu)
finder.find_nodes_bridge(bridge)
if old_loop_tokens:
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/__init__.py Fri Oct 22 23:09:43 2010
@@ -3,6 +3,7 @@
from pypy.jit.metainterp.optimizeopt.intbounds import OptIntBounds
from pypy.jit.metainterp.optimizeopt.virtualize import OptVirtualize
from pypy.jit.metainterp.optimizeopt.heap import OptHeap
+from pypy.jit.metainterp.optimizeopt.fficall import OptFfiCall
from pypy.jit.metainterp.optimizeopt.string import OptString
def optimize_loop_1(metainterp_sd, loop, virtuals=True):
@@ -16,6 +17,7 @@
OptVirtualize(),
OptString(),
OptHeap(),
+ OptFfiCall(),
]
optimizer = Optimizer(metainterp_sd, loop, optimizations, virtuals)
optimizer.propagate_all_forward()
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/intbounds.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/intbounds.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/intbounds.py Fri Oct 22 23:09:43 2010
@@ -191,6 +191,7 @@
v1.intbound.make_ge(IntLowerBound(0))
optimize_STRLEN = optimize_ARRAYLEN_GC
+ optimize_UNICODELEN = optimize_ARRAYLEN_GC
def make_int_lt(self, box1, box2):
v1 = self.getvalue(box1)
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/rewrite.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/rewrite.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/rewrite.py Fri Oct 22 23:09:43 2010
@@ -3,6 +3,7 @@
from pypy.jit.metainterp.history import ConstInt
from pypy.jit.metainterp.optimizeutil import _findall
from pypy.jit.metainterp.resoperation import rop, ResOperation
+from pypy.jit.codewriter.effectinfo import EffectInfo
class OptRewrite(Optimization):
"""Rewrite operations into equivalent, cheaper operations.
@@ -245,6 +246,9 @@
def optimize_CALL_LOOPINVARIANT(self, op):
funcvalue = self.getvalue(op.getarg(0))
if not funcvalue.is_constant():
+ # XXX this code path is never executed in tests nor in production.
+ # in fact, it can't even happen since residual_call in codewriter
+ # expects a compile-time constant
self.emit_operation(op)
return
key = make_hashable_int(op.getarg(0).getint())
@@ -323,8 +327,37 @@
## return
## self.emit_operation(op)
-optimize_ops = _findall(OptRewrite, 'optimize_')
-
+ def optimize_CALL(self, op):
+ # dispatch based on 'oopspecindex' to a method that handles
+ # specifically the given oopspec call. For non-oopspec calls,
+ # oopspecindex is just zero.
+ effectinfo = op.getdescr().get_extra_info()
+ if effectinfo is not None:
+ oopspecindex = effectinfo.oopspecindex
+ if oopspecindex == EffectInfo.OS_ARRAYCOPY:
+ if self._optimize_CALL_ARRAYCOPY(op):
+ return
+ self.emit_operation(op)
+ def _optimize_CALL_ARRAYCOPY(self, op):
+ source_value = self.getvalue(op.getarg(1))
+ dest_value = self.getvalue(op.getarg(2))
+ source_start_box = self.get_constant_box(op.getarg(3))
+ dest_start_box = self.get_constant_box(op.getarg(4))
+ length = self.get_constant_box(op.getarg(5))
+ if (source_value.is_virtual() and source_start_box and dest_start_box
+ and length and dest_value.is_virtual()):
+ # XXX optimize the case where dest value is not virtual,
+ # but we still can avoid a mess
+ source_start = source_start_box.getint()
+ dest_start = dest_start_box.getint()
+ for index in range(length.getint()):
+ val = source_value.getitem(index + source_start)
+ dest_value.setitem(index + dest_start, val)
+ return True
+ if length and length.getint() == 0:
+ return True # 0-length arraycopy
+ return False
+
+optimize_ops = _findall(OptRewrite, 'optimize_')
-
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/string.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/string.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/string.py Fri Oct 22 23:09:43 2010
@@ -3,6 +3,7 @@
from pypy.jit.metainterp.history import Box, BoxInt, BoxPtr
from pypy.jit.metainterp.history import Const, ConstInt, ConstPtr
from pypy.jit.metainterp.history import get_const_ptr_for_string
+from pypy.jit.metainterp.history import get_const_ptr_for_unicode
from pypy.jit.metainterp.resoperation import rop, ResOperation
from pypy.jit.metainterp.optimizeopt import optimizer, virtualize
from pypy.jit.metainterp.optimizeopt.optimizer import CONST_0, CONST_1
@@ -11,56 +12,106 @@
from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec
from pypy.jit.codewriter import heaptracker
from pypy.rlib.unroll import unrolling_iterable
+from pypy.rlib.objectmodel import specialize, we_are_translated
+
+
+class StrOrUnicode(object):
+ def __init__(self, LLTYPE, hlstr, emptystr, chr,
+ NEWSTR, STRLEN, STRGETITEM, STRSETITEM, COPYSTRCONTENT,
+ OS_offset):
+ self.LLTYPE = LLTYPE
+ self.hlstr = hlstr
+ self.emptystr = emptystr
+ self.chr = chr
+ self.NEWSTR = NEWSTR
+ self.STRLEN = STRLEN
+ self.STRGETITEM = STRGETITEM
+ self.STRSETITEM = STRSETITEM
+ self.COPYSTRCONTENT = COPYSTRCONTENT
+ self.OS_offset = OS_offset
+
+ def _freeze_(self):
+ return True
+
+mode_string = StrOrUnicode(rstr.STR, annlowlevel.hlstr, '', chr,
+ rop.NEWSTR, rop.STRLEN, rop.STRGETITEM,
+ rop.STRSETITEM, rop.COPYSTRCONTENT, 0)
+mode_unicode = StrOrUnicode(rstr.UNICODE, annlowlevel.hlunicode, u'', unichr,
+ rop.NEWUNICODE, rop.UNICODELEN, rop.UNICODEGETITEM,
+ rop.UNICODESETITEM, rop.COPYUNICODECONTENT,
+ EffectInfo._OS_offset_uni)
+
+# ____________________________________________________________
class __extend__(optimizer.OptValue):
"""New methods added to the base class OptValue for this file."""
- def getstrlen(self, newoperations):
- s = self.get_constant_string()
- if s is not None:
- return ConstInt(len(s))
+ def getstrlen(self, newoperations, mode):
+ if mode is mode_string:
+ s = self.get_constant_string_spec(mode_string)
+ if s is not None:
+ return ConstInt(len(s))
else:
- if newoperations is None:
- return None
- self.ensure_nonnull()
- box = self.force_box()
- lengthbox = BoxInt()
- newoperations.append(ResOperation(rop.STRLEN, [box], lengthbox))
- return lengthbox
+ s = self.get_constant_string_spec(mode_unicode)
+ if s is not None:
+ return ConstInt(len(s))
+ if newoperations is None:
+ return None
+ self.ensure_nonnull()
+ box = self.force_box()
+ lengthbox = BoxInt()
+ newoperations.append(ResOperation(mode.STRLEN, [box], lengthbox))
+ return lengthbox
- def get_constant_string(self):
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
if self.is_constant():
- s = self.box.getref(lltype.Ptr(rstr.STR))
- return annlowlevel.hlstr(s)
+ s = self.box.getref(lltype.Ptr(mode.LLTYPE))
+ return mode.hlstr(s)
else:
return None
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
# Copies the pointer-to-string 'self' into the target string
# given by 'targetbox', at the specified offset. Returns the offset
# at the end of the copy.
- lengthbox = self.getstrlen(newoperations)
+ lengthbox = self.getstrlen(newoperations, mode)
srcbox = self.force_box()
return copy_str_content(newoperations, srcbox, targetbox,
- CONST_0, offsetbox, lengthbox)
+ CONST_0, offsetbox, lengthbox, mode)
class VAbstractStringValue(virtualize.AbstractVirtualValue):
- _attrs_ = ()
+ _attrs_ = ('mode',)
+
+ def __init__(self, optimizer, keybox, source_op, mode):
+ virtualize.AbstractVirtualValue.__init__(self, optimizer, keybox,
+ source_op)
+ self.mode = mode
def _really_force(self):
- s = self.get_constant_string()
- if s is not None:
- c_s = get_const_ptr_for_string(s)
- self.make_constant(c_s)
- return
+ if self.mode is mode_string:
+ s = self.get_constant_string_spec(mode_string)
+ if s is not None:
+ c_s = get_const_ptr_for_string(s)
+ self.make_constant(c_s)
+ return
+ else:
+ s = self.get_constant_string_spec(mode_unicode)
+ if s is not None:
+ c_s = get_const_ptr_for_unicode(s)
+ self.make_constant(c_s)
+ return
assert self.source_op is not None
self.box = box = self.source_op.result
newoperations = self.optimizer.newoperations
- lengthbox = self.getstrlen(newoperations)
- newoperations.append(ResOperation(rop.NEWSTR, [lengthbox], box))
- self.string_copy_parts(newoperations, box, CONST_0)
+ lengthbox = self.getstrlen(newoperations, self.mode)
+ op = ResOperation(self.mode.NEWSTR, [lengthbox], box)
+ if not we_are_translated():
+ op.name = 'FORCE'
+ newoperations.append(op)
+ self.string_copy_parts(newoperations, box, CONST_0, self.mode)
class VStringPlainValue(VAbstractStringValue):
@@ -74,7 +125,7 @@
assert 0 <= start <= stop <= len(longerlist)
self._chars = longerlist[start:stop]
- def getstrlen(self, _):
+ def getstrlen(self, _, mode):
if self._lengthbox is None:
self._lengthbox = ConstInt(len(self._chars))
return self._lengthbox
@@ -86,18 +137,21 @@
assert isinstance(charvalue, optimizer.OptValue)
self._chars[index] = charvalue
- def get_constant_string(self):
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
for c in self._chars:
if c is optimizer.CVAL_UNINITIALIZED_ZERO or not c.is_constant():
return None
- return ''.join([chr(c.box.getint()) for c in self._chars])
+ return mode.emptystr.join([mode.chr(c.box.getint())
+ for c in self._chars])
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
for i in range(len(self._chars)):
charbox = self._chars[i].force_box()
- newoperations.append(ResOperation(rop.STRSETITEM, [targetbox,
- offsetbox,
- charbox], None))
+ newoperations.append(ResOperation(mode.STRSETITEM, [targetbox,
+ offsetbox,
+ charbox],
+ None))
offsetbox = _int_add(newoperations, offsetbox, CONST_1)
return offsetbox
@@ -109,7 +163,7 @@
value.get_args_for_fail(modifier)
def _make_virtual(self, modifier):
- return modifier.make_vstrplain()
+ return modifier.make_vstrplain(self.mode is mode_unicode)
class VStringConcatValue(VAbstractStringValue):
@@ -120,23 +174,24 @@
self.right = right
self.lengthbox = lengthbox
- def getstrlen(self, _):
+ def getstrlen(self, _, mode):
return self.lengthbox
- def get_constant_string(self):
- s1 = self.left.get_constant_string()
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
+ s1 = self.left.get_constant_string_spec(mode)
if s1 is None:
return None
- s2 = self.right.get_constant_string()
+ s2 = self.right.get_constant_string_spec(mode)
if s2 is None:
return None
return s1 + s2
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
offsetbox = self.left.string_copy_parts(newoperations, targetbox,
- offsetbox)
+ offsetbox, mode)
offsetbox = self.right.string_copy_parts(newoperations, targetbox,
- offsetbox)
+ offsetbox, mode)
return offsetbox
def get_args_for_fail(self, modifier):
@@ -150,7 +205,7 @@
self.right.get_args_for_fail(modifier)
def _make_virtual(self, modifier):
- return modifier.make_vstrconcat()
+ return modifier.make_vstrconcat(self.mode is mode_unicode)
class VStringSliceValue(VAbstractStringValue):
@@ -162,12 +217,13 @@
self.vstart = vstart
self.vlength = vlength
- def getstrlen(self, _):
+ def getstrlen(self, _, mode):
return self.vlength.force_box()
- def get_constant_string(self):
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
if self.vstart.is_constant() and self.vlength.is_constant():
- s1 = self.vstr.get_constant_string()
+ s1 = self.vstr.get_constant_string_spec(mode)
if s1 is None:
return None
start = self.vstart.box.getint()
@@ -177,12 +233,12 @@
return s1[start : start + length]
return None
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
- lengthbox = self.getstrlen(newoperations)
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
+ lengthbox = self.getstrlen(newoperations, mode)
return copy_str_content(newoperations,
self.vstr.force_box(), targetbox,
self.vstart.force_box(), offsetbox,
- lengthbox)
+ lengthbox, mode)
def get_args_for_fail(self, modifier):
if self.box is None and not modifier.already_seen_virtual(self.keybox):
@@ -195,11 +251,11 @@
self.vlength.get_args_for_fail(modifier)
def _make_virtual(self, modifier):
- return modifier.make_vstrslice()
+ return modifier.make_vstrslice(self.mode is mode_unicode)
def copy_str_content(newoperations, srcbox, targetbox,
- srcoffsetbox, offsetbox, lengthbox):
+ srcoffsetbox, offsetbox, lengthbox, mode):
if isinstance(srcbox, ConstPtr) and isinstance(srcoffsetbox, Const):
M = 5
else:
@@ -208,17 +264,18 @@
# up to M characters are done "inline", i.e. with STRGETITEM/STRSETITEM
# instead of just a COPYSTRCONTENT.
for i in range(lengthbox.value):
- charbox = _strgetitem(newoperations, srcbox, srcoffsetbox)
+ charbox = _strgetitem(newoperations, srcbox, srcoffsetbox, mode)
srcoffsetbox = _int_add(newoperations, srcoffsetbox, CONST_1)
- newoperations.append(ResOperation(rop.STRSETITEM, [targetbox,
- offsetbox,
- charbox], None))
+ newoperations.append(ResOperation(mode.STRSETITEM, [targetbox,
+ offsetbox,
+ charbox],
+ None))
offsetbox = _int_add(newoperations, offsetbox, CONST_1)
else:
nextoffsetbox = _int_add(newoperations, offsetbox, lengthbox)
- op = ResOperation(rop.COPYSTRCONTENT, [srcbox, targetbox,
- srcoffsetbox, offsetbox,
- lengthbox], None)
+ op = ResOperation(mode.COPYSTRCONTENT, [srcbox, targetbox,
+ srcoffsetbox, offsetbox,
+ lengthbox], None)
newoperations.append(op)
offsetbox = nextoffsetbox
return offsetbox
@@ -245,12 +302,16 @@
newoperations.append(ResOperation(rop.INT_SUB, [box1, box2], resbox))
return resbox
-def _strgetitem(newoperations, strbox, indexbox):
+def _strgetitem(newoperations, strbox, indexbox, mode):
if isinstance(strbox, ConstPtr) and isinstance(indexbox, ConstInt):
- s = strbox.getref(lltype.Ptr(rstr.STR))
- return ConstInt(ord(s.chars[indexbox.getint()]))
+ if mode is mode_string:
+ s = strbox.getref(lltype.Ptr(rstr.STR))
+ return ConstInt(ord(s.chars[indexbox.getint()]))
+ else:
+ s = strbox.getref(lltype.Ptr(rstr.UNICODE))
+ return ConstInt(ord(s.chars[indexbox.getint()]))
resbox = BoxInt()
- newoperations.append(ResOperation(rop.STRGETITEM, [strbox, indexbox],
+ newoperations.append(ResOperation(mode.STRGETITEM, [strbox, indexbox],
resbox))
return resbox
@@ -258,62 +319,34 @@
class OptString(optimizer.Optimization):
"Handling of strings and unicodes."
- def make_vstring_plain(self, box, source_op=None):
- vvalue = VStringPlainValue(self.optimizer, box, source_op)
+ def make_vstring_plain(self, box, source_op, mode):
+ vvalue = VStringPlainValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
return vvalue
- def make_vstring_concat(self, box, source_op=None):
- vvalue = VStringConcatValue(self.optimizer, box, source_op)
+ def make_vstring_concat(self, box, source_op, mode):
+ vvalue = VStringConcatValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
return vvalue
- def make_vstring_slice(self, box, source_op=None):
- vvalue = VStringSliceValue(self.optimizer, box, source_op)
+ def make_vstring_slice(self, box, source_op, mode):
+ vvalue = VStringSliceValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
return vvalue
- def optimize_CALL(self, op):
- # dispatch based on 'oopspecindex' to a method that handles
- # specifically the given oopspec call. For non-oopspec calls,
- # oopspecindex is just zero.
- effectinfo = op.getdescr().get_extra_info()
- if effectinfo is not None:
- oopspecindex = effectinfo.oopspecindex
- for value, meth in opt_call_oopspec_ops:
- if oopspecindex == value:
- if meth(self, op):
- return
- self.emit_operation(op)
-
- def opt_call_oopspec_ARRAYCOPY(self, op):
- source_value = self.getvalue(op.getarg(1))
- dest_value = self.getvalue(op.getarg(2))
- source_start_box = self.get_constant_box(op.getarg(3))
- dest_start_box = self.get_constant_box(op.getarg(4))
- length = self.get_constant_box(op.getarg(5))
- if (source_value.is_virtual() and source_start_box and dest_start_box
- and length and dest_value.is_virtual()):
- # XXX optimize the case where dest value is not virtual,
- # but we still can avoid a mess
- source_start = source_start_box.getint()
- dest_start = dest_start_box.getint()
- for index in range(length.getint()):
- val = source_value.getitem(index + source_start)
- dest_value.setitem(index + dest_start, val)
- return True
- if length and length.getint() == 0:
- return True # 0-length arraycopy
- return False
-
def optimize_NEWSTR(self, op):
+ self._optimize_NEWSTR(op, mode_string)
+ def optimize_NEWUNICODE(self, op):
+ self._optimize_NEWSTR(op, mode_unicode)
+
+ def _optimize_NEWSTR(self, op, mode):
length_box = self.get_constant_box(op.getarg(0))
if length_box:
# if the original 'op' did not have a ConstInt as argument,
# build a new one with the ConstInt argument
if not isinstance(op.getarg(0), ConstInt):
- op = ResOperation(rop.NEWSTR, [length_box], op.result)
- vvalue = self.make_vstring_plain(op.result, op)
+ op = ResOperation(mode.NEWSTR, [length_box], op.result)
+ vvalue = self.make_vstring_plain(op.result, op, mode)
vvalue.setup(length_box.getint())
else:
self.getvalue(op.result).ensure_nonnull()
@@ -329,13 +362,20 @@
value.ensure_nonnull()
self.emit_operation(op)
+ optimize_UNICODESETITEM = optimize_STRSETITEM
+
def optimize_STRGETITEM(self, op):
+ self._optimize_STRGETITEM(op, mode_string)
+ def optimize_UNICODEGETITEM(self, op):
+ self._optimize_STRGETITEM(op, mode_unicode)
+
+ def _optimize_STRGETITEM(self, op, mode):
value = self.getvalue(op.getarg(0))
vindex = self.getvalue(op.getarg(1))
- vresult = self.strgetitem(value, vindex)
+ vresult = self.strgetitem(value, vindex, mode)
self.make_equal_to(op.result, vresult)
- def strgetitem(self, value, vindex):
+ def strgetitem(self, value, vindex, mode):
value.ensure_nonnull()
#
if value.is_virtual() and isinstance(value, VStringSliceValue):
@@ -350,28 +390,71 @@
return value.getitem(vindex.box.getint())
#
resbox = _strgetitem(self.optimizer.newoperations,
- value.force_box(),vindex.force_box())
+ value.force_box(),vindex.force_box(), mode)
return self.getvalue(resbox)
def optimize_STRLEN(self, op):
+ self._optimize_STRLEN(op, mode_string)
+ def optimize_UNICODELEN(self, op):
+ self._optimize_STRLEN(op, mode_unicode)
+
+ def _optimize_STRLEN(self, op, mode):
value = self.getvalue(op.getarg(0))
- lengthbox = value.getstrlen(self.optimizer.newoperations)
+ lengthbox = value.getstrlen(self.optimizer.newoperations, mode)
self.make_equal_to(op.result, self.getvalue(lengthbox))
- def opt_call_oopspec_STR_CONCAT(self, op):
+ def optimize_CALL(self, op):
+ # dispatch based on 'oopspecindex' to a method that handles
+ # specifically the given oopspec call. For non-oopspec calls,
+ # oopspecindex is just zero.
+ effectinfo = op.getdescr().get_extra_info()
+ if effectinfo is not None:
+ oopspecindex = effectinfo.oopspecindex
+ for value, meth in opt_call_oopspec_ops:
+ if oopspecindex == value: # a match with the OS_STR_xxx
+ if meth(self, op, mode_string):
+ return
+ break
+ if oopspecindex == value + EffectInfo._OS_offset_uni:
+ # a match with the OS_UNI_xxx
+ if meth(self, op, mode_unicode):
+ return
+ break
+ if oopspecindex == EffectInfo.OS_STR2UNICODE:
+ if self.opt_call_str_STR2UNICODE(op):
+ return
+ self.emit_operation(op)
+
+ def opt_call_str_STR2UNICODE(self, op):
+ # Constant-fold unicode("constant string").
+ # More generally, supporting non-constant but virtual cases is
+ # not obvious, because of the exception UnicodeDecodeError that
+ # can be raised by ll_str2unicode()
+ varg = self.getvalue(op.getarg(1))
+ s = varg.get_constant_string_spec(mode_string)
+ if s is None:
+ return False
+ try:
+ u = unicode(s)
+ except UnicodeDecodeError:
+ return False
+ self.make_constant(op.result, get_const_ptr_for_unicode(u))
+ return True
+
+ def opt_call_stroruni_STR_CONCAT(self, op, mode):
vleft = self.getvalue(op.getarg(1))
vright = self.getvalue(op.getarg(2))
vleft.ensure_nonnull()
vright.ensure_nonnull()
newoperations = self.optimizer.newoperations
- len1box = vleft.getstrlen(newoperations)
- len2box = vright.getstrlen(newoperations)
+ len1box = vleft.getstrlen(newoperations, mode)
+ len2box = vright.getstrlen(newoperations, mode)
lengthbox = _int_add(newoperations, len1box, len2box)
- value = self.make_vstring_concat(op.result, op)
+ value = self.make_vstring_concat(op.result, op, mode)
value.setup(vleft, vright, lengthbox)
return True
- def opt_call_oopspec_STR_SLICE(self, op):
+ def opt_call_stroruni_STR_SLICE(self, op, mode):
newoperations = self.optimizer.newoperations
vstr = self.getvalue(op.getarg(1))
vstart = self.getvalue(op.getarg(2))
@@ -380,7 +463,7 @@
if (isinstance(vstr, VStringPlainValue) and vstart.is_constant()
and vstop.is_constant()):
# slicing with constant bounds of a VStringPlainValue
- value = self.make_vstring_plain(op.result, op)
+ value = self.make_vstring_plain(op.result, op, mode)
value.setup_slice(vstr._chars, vstart.box.getint(),
vstop.box.getint())
return True
@@ -398,16 +481,16 @@
vstart.force_box())
vstart = self.getvalue(startbox)
#
- value = self.make_vstring_slice(op.result, op)
+ value = self.make_vstring_slice(op.result, op, mode)
value.setup(vstr, vstart, self.getvalue(lengthbox))
return True
- def opt_call_oopspec_STR_EQUAL(self, op):
+ def opt_call_stroruni_STR_EQUAL(self, op, mode):
v1 = self.getvalue(op.getarg(1))
v2 = self.getvalue(op.getarg(2))
#
- l1box = v1.getstrlen(None)
- l2box = v2.getstrlen(None)
+ l1box = v1.getstrlen(None, mode)
+ l2box = v2.getstrlen(None, mode)
if (l1box is not None and l2box is not None and
isinstance(l1box, ConstInt) and
isinstance(l2box, ConstInt) and
@@ -416,13 +499,13 @@
self.make_constant(op.result, CONST_0)
return True
#
- if self.handle_str_equal_level1(v1, v2, op.result):
+ if self.handle_str_equal_level1(v1, v2, op.result, mode):
return True
- if self.handle_str_equal_level1(v2, v1, op.result):
+ if self.handle_str_equal_level1(v2, v1, op.result, mode):
return True
- if self.handle_str_equal_level2(v1, v2, op.result):
+ if self.handle_str_equal_level2(v1, v2, op.result, mode):
return True
- if self.handle_str_equal_level2(v2, v1, op.result):
+ if self.handle_str_equal_level2(v2, v1, op.result, mode):
return True
#
if v1.is_nonnull() and v2.is_nonnull():
@@ -434,37 +517,37 @@
else:
do = EffectInfo.OS_STREQ_NONNULL
self.generate_modified_call(do, [v1.force_box(),
- v2.force_box()], op.result)
+ v2.force_box()], op.result, mode)
return True
return False
- def handle_str_equal_level1(self, v1, v2, resultbox):
- l2box = v2.getstrlen(None)
+ def handle_str_equal_level1(self, v1, v2, resultbox, mode):
+ l2box = v2.getstrlen(None, mode)
if isinstance(l2box, ConstInt):
if l2box.value == 0:
- lengthbox = v1.getstrlen(self.optimizer.newoperations)
+ lengthbox = v1.getstrlen(self.optimizer.newoperations, mode)
seo = self.optimizer.send_extra_operation
seo(ResOperation(rop.INT_EQ, [lengthbox, CONST_0], resultbox))
return True
if l2box.value == 1:
- l1box = v1.getstrlen(None)
+ l1box = v1.getstrlen(None, mode)
if isinstance(l1box, ConstInt) and l1box.value == 1:
# comparing two single chars
- vchar1 = self.strgetitem(v1, optimizer.CVAL_ZERO)
- vchar2 = self.strgetitem(v2, optimizer.CVAL_ZERO)
+ vchar1 = self.strgetitem(v1, optimizer.CVAL_ZERO, mode)
+ vchar2 = self.strgetitem(v2, optimizer.CVAL_ZERO, mode)
seo = self.optimizer.send_extra_operation
seo(ResOperation(rop.INT_EQ, [vchar1.force_box(),
vchar2.force_box()],
resultbox))
return True
if isinstance(v1, VStringSliceValue):
- vchar = self.strgetitem(v2, optimizer.CVAL_ZERO)
+ vchar = self.strgetitem(v2, optimizer.CVAL_ZERO, mode)
do = EffectInfo.OS_STREQ_SLICE_CHAR
self.generate_modified_call(do, [v1.vstr.force_box(),
v1.vstart.force_box(),
v1.vlength.force_box(),
vchar.force_box()],
- resultbox)
+ resultbox, mode)
return True
#
if v2.is_null():
@@ -482,17 +565,18 @@
#
return False
- def handle_str_equal_level2(self, v1, v2, resultbox):
- l2box = v2.getstrlen(None)
+ def handle_str_equal_level2(self, v1, v2, resultbox, mode):
+ l2box = v2.getstrlen(None, mode)
if isinstance(l2box, ConstInt):
if l2box.value == 1:
- vchar = self.strgetitem(v2, optimizer.CVAL_ZERO)
+ vchar = self.strgetitem(v2, optimizer.CVAL_ZERO, mode)
if v1.is_nonnull():
do = EffectInfo.OS_STREQ_NONNULL_CHAR
else:
do = EffectInfo.OS_STREQ_CHECKNULL_CHAR
self.generate_modified_call(do, [v1.force_box(),
- vchar.force_box()], resultbox)
+ vchar.force_box()], resultbox,
+ mode)
return True
#
if v1.is_virtual() and isinstance(v1, VStringSliceValue):
@@ -503,11 +587,12 @@
self.generate_modified_call(do, [v1.vstr.force_box(),
v1.vstart.force_box(),
v1.vlength.force_box(),
- v2.force_box()], resultbox)
+ v2.force_box()], resultbox, mode)
return True
return False
- def generate_modified_call(self, oopspecindex, args, result):
+ def generate_modified_call(self, oopspecindex, args, result, mode):
+ oopspecindex += mode.OS_offset
calldescr, func = callinfo_for_oopspec(oopspecindex)
op = ResOperation(rop.CALL, [ConstInt(func)] + args, result,
descr=calldescr)
@@ -525,7 +610,7 @@
optimize_ops = _findall(OptString, 'optimize_')
def _findall_call_oopspec():
- prefix = 'opt_call_oopspec_'
+ prefix = 'opt_call_stroruni_'
result = []
for name in dir(OptString):
if name.startswith(prefix):
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/virtualize.py Fri Oct 22 23:09:43 2010
@@ -74,6 +74,8 @@
assert self.source_op is not None
# ^^^ This case should not occur any more (see test_bug_3).
#
+ if not we_are_translated():
+ self.source_op.name = 'FORCE ' + self.source_op.name
newoperations = self.optimizer.newoperations
newoperations.append(self.source_op)
self.box = box = self.source_op.result
@@ -134,6 +136,11 @@
fielddescrs = self._get_field_descr_list()
return modifier.make_virtual(self.known_class, fielddescrs)
+ def __repr__(self):
+ cls_name = self.known_class.value.adr.ptr._obj._TYPE._name
+ field_names = [field.name for field in self._fields]
+ return "" % (cls_name, field_names)
+
class VStructValue(AbstractVirtualStructValue):
def __init__(self, optimizer, structdescr, keybox, source_op=None):
@@ -165,6 +172,8 @@
def _really_force(self):
assert self.source_op is not None
+ if not we_are_translated():
+ self.source_op.name = 'FORCE ' + self.source_op.name
newoperations = self.optimizer.newoperations
newoperations.append(self.source_op)
self.box = box = self.source_op.result
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py Fri Oct 22 23:09:43 2010
@@ -1,4 +1,4 @@
-import py, os
+import py, os, sys
from pypy.rpython.lltypesystem import lltype, llmemory, rclass
from pypy.rlib.objectmodel import we_are_translated
from pypy.rlib.unroll import unrolling_iterable
@@ -498,6 +498,22 @@
opimpl_getfield_gc_r_pure = _opimpl_getfield_gc_pure_any
opimpl_getfield_gc_f_pure = _opimpl_getfield_gc_pure_any
+ @arguments("orgpc", "box", "descr")
+ def _opimpl_getfield_gc_greenfield_any(self, pc, box, fielddescr):
+ ginfo = self.metainterp.jitdriver_sd.greenfield_info
+ if (ginfo is not None and fielddescr in ginfo.green_field_descrs
+ and not self._nonstandard_virtualizable(pc, box)):
+ # fetch the result, but consider it as a Const box and don't
+ # record any operation
+ resbox = executor.execute(self.metainterp.cpu, self.metainterp,
+ rop.GETFIELD_GC_PURE, fielddescr, box)
+ return resbox.constbox()
+ # fall-back
+ return self.execute_with_descr(rop.GETFIELD_GC_PURE, fielddescr, box)
+ opimpl_getfield_gc_i_greenfield = _opimpl_getfield_gc_greenfield_any
+ opimpl_getfield_gc_r_greenfield = _opimpl_getfield_gc_greenfield_any
+ opimpl_getfield_gc_f_greenfield = _opimpl_getfield_gc_greenfield_any
+
@arguments("box", "descr", "box")
def _opimpl_setfield_gc_any(self, box, fielddescr, valuebox):
self.execute_with_descr(rop.SETFIELD_GC, fielddescr, box, valuebox)
@@ -529,7 +545,8 @@
def _nonstandard_virtualizable(self, pc, box):
# returns True if 'box' is actually not the "standard" virtualizable
# that is stored in metainterp.virtualizable_boxes[-1]
- if self.metainterp.jitdriver_sd.virtualizable_info is None:
+ if (self.metainterp.jitdriver_sd.virtualizable_info is None and
+ self.metainterp.jitdriver_sd.greenfield_info is None):
return True # can occur in case of multiple JITs
standard_box = self.metainterp.virtualizable_boxes[-1]
if standard_box is box:
@@ -799,12 +816,16 @@
@arguments("orgpc", "int", "boxes3", "boxes3")
def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes, redboxes):
+ any_operation = len(self.metainterp.history.operations) > 0
jitdriver_sd = self.metainterp.staticdata.jitdrivers_sd[jdindex]
self.verify_green_args(jitdriver_sd, greenboxes)
# xxx we may disable the following line in some context later
self.debug_merge_point(jitdriver_sd, greenboxes)
if self.metainterp.seen_loop_header_for_jdindex < 0:
- return
+ if not jitdriver_sd.no_loop_header or not any_operation:
+ return
+ # automatically add a loop_header if there is none
+ self.metainterp.seen_loop_header_for_jdindex = jdindex
#
assert self.metainterp.seen_loop_header_for_jdindex == jdindex, (
"found a loop_header for a JitDriver that does not match "
@@ -893,6 +914,40 @@
msg = box.getref(lltype.Ptr(rstr.STR))
lloperation.llop.debug_fatalerror(msg)
+ @arguments("box", "box", "box", "box", "box")
+ def opimpl_jit_debug(self, stringbox, arg1box, arg2box, arg3box, arg4box):
+ from pypy.rpython.lltypesystem import rstr
+ from pypy.rpython.annlowlevel import hlstr
+ msg = stringbox.getref(lltype.Ptr(rstr.STR))
+ debug_print('jit_debug:', hlstr(msg),
+ arg1box.getint(), arg2box.getint(),
+ arg3box.getint(), arg4box.getint())
+ args = [stringbox, arg1box, arg2box, arg3box, arg4box]
+ i = 4
+ while i > 0 and args[i].getint() == -sys.maxint-1:
+ i -= 1
+ assert i >= 0
+ op = self.metainterp.history.record(rop.JIT_DEBUG, args[:i+1], None)
+ self.metainterp.attach_debug_info(op)
+
+ @arguments("box")
+ def _opimpl_assert_green(self, box):
+ if not isinstance(box, Const):
+ msg = "assert_green failed at %s:%d" % (
+ self.jitcode.name,
+ self.pc)
+ if we_are_translated():
+ from pypy.rpython.annlowlevel import llstr
+ from pypy.rpython.lltypesystem import lloperation
+ lloperation.llop.debug_fatalerror(lltype.Void, llstr(msg))
+ else:
+ from pypy.rlib.jit import AssertGreenFailed
+ raise AssertGreenFailed(msg)
+
+ opimpl_int_assert_green = _opimpl_assert_green
+ opimpl_ref_assert_green = _opimpl_assert_green
+ opimpl_float_assert_green = _opimpl_assert_green
+
@arguments("box")
def opimpl_virtual_ref(self, box):
# Details on the content of metainterp.virtualref_boxes:
@@ -998,7 +1053,8 @@
guard_op = metainterp.history.record(opnum, moreargs, None,
descr=resumedescr)
virtualizable_boxes = None
- if metainterp.jitdriver_sd.virtualizable_info is not None:
+ if (metainterp.jitdriver_sd.virtualizable_info is not None or
+ metainterp.jitdriver_sd.greenfield_info is not None):
virtualizable_boxes = metainterp.virtualizable_boxes
saved_pc = self.pc
if resumepc >= 0:
@@ -1646,6 +1702,7 @@
duplicates)
live_arg_boxes += self.virtualizable_boxes
live_arg_boxes.pop()
+ #
assert len(self.virtualref_boxes) == 0, "missing virtual_ref_finish()?"
# Called whenever we reach the 'loop_header' hint.
# First, attempt to make a bridge:
@@ -1832,6 +1889,7 @@
f.setup_call(original_boxes)
assert self.in_recursion == 0
self.virtualref_boxes = []
+ self.initialize_withgreenfields(original_boxes)
self.initialize_virtualizable(original_boxes)
def initialize_state_from_guard_failure(self, resumedescr):
@@ -1856,6 +1914,14 @@
self.virtualizable_boxes.append(virtualizable_box)
self.initialize_virtualizable_enter()
+ def initialize_withgreenfields(self, original_boxes):
+ ginfo = self.jitdriver_sd.greenfield_info
+ if ginfo is not None:
+ assert self.jitdriver_sd.virtualizable_info is None
+ index = (self.jitdriver_sd.num_green_args +
+ ginfo.red_index)
+ self.virtualizable_boxes = [original_boxes[index]]
+
def initialize_virtualizable_enter(self):
vinfo = self.jitdriver_sd.virtualizable_info
virtualizable_box = self.virtualizable_boxes[-1]
@@ -1949,8 +2015,10 @@
def rebuild_state_after_failure(self, resumedescr):
vinfo = self.jitdriver_sd.virtualizable_info
+ ginfo = self.jitdriver_sd.greenfield_info
self.framestack = []
- boxlists = resume.rebuild_from_resumedata(self, resumedescr, vinfo)
+ boxlists = resume.rebuild_from_resumedata(self, resumedescr, vinfo,
+ ginfo)
inputargs_and_holes, virtualizable_boxes, virtualref_boxes = boxlists
#
# virtual refs: make the vrefs point to the freshly allocated virtuals
@@ -1975,6 +2043,12 @@
assert not virtualizable.vable_token
# fill the virtualizable with the local boxes
self.synchronize_virtualizable()
+ #
+ elif self.jitdriver_sd.greenfield_info:
+ self.virtualizable_boxes = virtualizable_boxes
+ else:
+ assert not virtualizable_boxes
+ #
return inputargs_and_holes
def check_synchronized_virtualizable(self):
@@ -2048,7 +2122,8 @@
for i in range(len(boxes)):
if boxes[i] is oldbox:
boxes[i] = newbox
- if self.jitdriver_sd.virtualizable_info is not None:
+ if (self.jitdriver_sd.virtualizable_info is not None or
+ self.jitdriver_sd.greenfield_info is not None):
boxes = self.virtualizable_boxes
for i in range(len(boxes)):
if boxes[i] is oldbox:
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py Fri Oct 22 23:09:43 2010
@@ -93,7 +93,7 @@
def __repr__(self):
return self.repr()
- def repr(self):
+ def repr(self, graytext=False):
# RPython-friendly version
if self.result is not None:
sres = '%s = ' % (self.result,)
@@ -101,6 +101,8 @@
sres = ''
if self.name:
prefix = "%s:%s " % (self.name, self.pc)
+ if graytext:
+ prefix = "\f%s\f" % prefix
else:
prefix = ""
args = self.getarglist()
@@ -457,13 +459,14 @@
#'RUNTIMENEW/1', # ootype operation
'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier)
'DEBUG_MERGE_POINT/1', # debugging only
+ 'JIT_DEBUG/*', # debugging only
'VIRTUAL_REF_FINISH/2', # removed before it's passed to the backend
'COPYSTRCONTENT/5', # src, dst, srcstart, dststart, length
'COPYUNICODECONTENT/5',
'_CANRAISE_FIRST', # ----- start of can_raise operations -----
'CALL/*d',
- 'CALL_ASSEMBLER/*d',
+ 'CALL_ASSEMBLER/*d', # call already compiled assembler
'CALL_MAY_FORCE/*d',
'CALL_LOOPINVARIANT/*d',
#'OOSEND', # ootype operation
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/resume.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/resume.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/resume.py Fri Oct 22 23:09:43 2010
@@ -255,13 +255,19 @@
def make_varray(self, arraydescr):
return VArrayInfo(arraydescr)
- def make_vstrplain(self):
+ def make_vstrplain(self, is_unicode=False):
+ if is_unicode:
+ return VUniPlainInfo()
return VStrPlainInfo()
- def make_vstrconcat(self):
+ def make_vstrconcat(self, is_unicode=False):
+ if is_unicode:
+ return VUniConcatInfo()
return VStrConcatInfo()
- def make_vstrslice(self):
+ def make_vstrslice(self, is_unicode=False):
+ if is_unicode:
+ return VUniSliceInfo()
return VStrSliceInfo()
def register_virtual_fields(self, virtualbox, fieldboxes):
@@ -550,6 +556,60 @@
for i in self.fieldnums:
debug_print("\t\t", str(untag(i)))
+
+class VUniPlainInfo(AbstractVirtualInfo):
+ """Stands for the unicode string made out of the characters of all
+ fieldnums."""
+
+ @specialize.argtype(1)
+ def allocate(self, decoder, index):
+ length = len(self.fieldnums)
+ string = decoder.allocate_unicode(length)
+ decoder.virtuals_cache[index] = string
+ for i in range(length):
+ decoder.unicode_setitem(string, i, self.fieldnums[i])
+ return string
+
+ def debug_prints(self):
+ debug_print("\tvuniplaininfo length", len(self.fieldnums))
+
+
+class VUniConcatInfo(AbstractVirtualInfo):
+ """Stands for the unicode string made out of the concatenation of two
+ other unicode strings."""
+
+ @specialize.argtype(1)
+ def allocate(self, decoder, index):
+ # xxx for blackhole resuming, this will build all intermediate
+ # strings and throw them away immediately, which is a bit sub-
+ # efficient. Not sure we care.
+ left, right = self.fieldnums
+ string = decoder.concat_unicodes(left, right)
+ decoder.virtuals_cache[index] = string
+ return string
+
+ def debug_prints(self):
+ debug_print("\tvuniconcatinfo")
+ for i in self.fieldnums:
+ debug_print("\t\t", str(untag(i)))
+
+
+class VUniSliceInfo(AbstractVirtualInfo):
+ """Stands for the unicode string made out of slicing another
+ unicode string."""
+
+ @specialize.argtype(1)
+ def allocate(self, decoder, index):
+ largerstr, start, length = self.fieldnums
+ string = decoder.slice_unicode(largerstr, start, length)
+ decoder.virtuals_cache[index] = string
+ return string
+
+ def debug_prints(self):
+ debug_print("\tvunisliceinfo")
+ for i in self.fieldnums:
+ debug_print("\t\t", str(untag(i)))
+
# ____________________________________________________________
class AbstractResumeDataReader(object):
@@ -629,9 +689,11 @@
# ---------- when resuming for pyjitpl.py, make boxes ----------
-def rebuild_from_resumedata(metainterp, storage, virtualizable_info):
+def rebuild_from_resumedata(metainterp, storage, virtualizable_info,
+ greenfield_info):
resumereader = ResumeDataBoxReader(storage, metainterp)
- boxes = resumereader.consume_vref_and_vable_boxes(virtualizable_info)
+ boxes = resumereader.consume_vref_and_vable_boxes(virtualizable_info,
+ greenfield_info)
virtualizable_boxes, virtualref_boxes = boxes
frameinfo = storage.rd_frame_info_list
while True:
@@ -676,15 +738,18 @@
assert (end & 1) == 0
return [self.decode_ref(nums[i]) for i in range(end)]
- def consume_vref_and_vable_boxes(self, vinfo):
+ def consume_vref_and_vable_boxes(self, vinfo, ginfo):
nums = self.cur_numb.nums
self.cur_numb = self.cur_numb.prev
- if vinfo is None:
- virtualizable_boxes = None
- end = len(nums)
- else:
+ if vinfo is not None:
virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, nums)
end = len(nums) - len(virtualizable_boxes)
+ elif ginfo is not None:
+ virtualizable_boxes = [self.decode_ref(nums[-1])]
+ end = len(nums) - 1
+ else:
+ virtualizable_boxes = None
+ end = len(nums)
virtualref_boxes = self.consume_virtualref_boxes(nums, end)
return virtualizable_boxes, virtualref_boxes
@@ -725,6 +790,32 @@
return self.metainterp.execute_and_record_varargs(
rop.CALL, [ConstInt(func), strbox, startbox, stopbox], calldescr)
+ def allocate_unicode(self, length):
+ return self.metainterp.execute_and_record(rop.NEWUNICODE,
+ None, ConstInt(length))
+
+ def unicode_setitem(self, strbox, index, charnum):
+ charbox = self.decode_box(charnum, INT)
+ self.metainterp.execute_and_record(rop.UNICODESETITEM, None,
+ strbox, ConstInt(index), charbox)
+
+ def concat_unicodes(self, str1num, str2num):
+ calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT)
+ str1box = self.decode_box(str1num, REF)
+ str2box = self.decode_box(str2num, REF)
+ return self.metainterp.execute_and_record_varargs(
+ rop.CALL, [ConstInt(func), str1box, str2box], calldescr)
+
+ def slice_unicode(self, strnum, startnum, lengthnum):
+ calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE)
+ strbox = self.decode_box(strnum, REF)
+ startbox = self.decode_box(startnum, INT)
+ lengthbox = self.decode_box(lengthnum, INT)
+ stopbox = self.metainterp.execute_and_record(rop.INT_ADD, None,
+ startbox, lengthbox)
+ return self.metainterp.execute_and_record_varargs(
+ rop.CALL, [ConstInt(func), strbox, startbox, stopbox], calldescr)
+
def setfield(self, descr, structbox, fieldnum):
if descr.is_pointer_field():
kind = REF
@@ -815,8 +906,9 @@
resumereader = ResumeDataDirectReader(blackholeinterpbuilder.cpu, storage,
all_virtuals)
vinfo = jitdriver_sd.virtualizable_info
+ ginfo = jitdriver_sd.greenfield_info
vrefinfo = blackholeinterpbuilder.metainterp_sd.virtualref_info
- resumereader.consume_vref_and_vable(vrefinfo, vinfo)
+ resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo)
#
# First get a chain of blackhole interpreters whose length is given
# by the depth of rd_frame_info_list. The first one we get must be
@@ -846,11 +938,11 @@
resumereader.done()
return firstbh
-def force_from_resumedata(metainterp_sd, storage, vinfo=None):
+def force_from_resumedata(metainterp_sd, storage, vinfo, ginfo):
resumereader = ResumeDataDirectReader(metainterp_sd.cpu, storage)
resumereader.handling_async_forcing()
vrefinfo = metainterp_sd.virtualref_info
- resumereader.consume_vref_and_vable(vrefinfo, vinfo)
+ resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo)
return resumereader.force_all_virtuals()
class ResumeDataDirectReader(AbstractResumeDataReader):
@@ -925,11 +1017,12 @@
return specialize_value(TYPE, x)
load_value_of_type._annspecialcase_ = 'specialize:arg(1)'
- def consume_vref_and_vable(self, vrefinfo, vinfo):
+ def consume_vref_and_vable(self, vrefinfo, vinfo, ginfo):
nums = self.cur_numb.nums
self.cur_numb = self.cur_numb.prev
if self.resume_after_guard_not_forced != 2:
end_vref = self.consume_vable_info(vinfo, nums)
+ if ginfo is not None: end_vref -= 1
self.consume_virtualref_info(vrefinfo, nums, end_vref)
def allocate_with_vtable(self, known_class):
@@ -967,6 +1060,31 @@
result = funcptr(str, start, start + length)
return lltype.cast_opaque_ptr(llmemory.GCREF, result)
+ def allocate_unicode(self, length):
+ return self.cpu.bh_newunicode(length)
+
+ def unicode_setitem(self, str, index, charnum):
+ char = self.decode_int(charnum)
+ self.cpu.bh_unicodesetitem(str, index, char)
+
+ def concat_unicodes(self, str1num, str2num):
+ str1 = self.decode_ref(str1num)
+ str2 = self.decode_ref(str2num)
+ str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str1)
+ str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str2)
+ funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT)
+ result = funcptr(str1, str2)
+ return lltype.cast_opaque_ptr(llmemory.GCREF, result)
+
+ def slice_unicode(self, strnum, startnum, lengthnum):
+ str = self.decode_ref(strnum)
+ start = self.decode_int(startnum)
+ length = self.decode_int(lengthnum)
+ str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str)
+ funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE)
+ result = funcptr(str, start, start + length)
+ return lltype.cast_opaque_ptr(llmemory.GCREF, result)
+
def setfield(self, descr, struct, fieldnum):
if descr.is_pointer_field():
newvalue = self.decode_ref(fieldnum)
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/simple_optimize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/simple_optimize.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/simple_optimize.py Fri Oct 22 23:09:43 2010
@@ -9,11 +9,13 @@
def transform(op):
from pypy.jit.metainterp.history import AbstractDescr
- # Rename CALL_PURE to CALL.
+ # Rename CALL_PURE and CALL_LOOPINVARIANT to CALL.
# Simplify the VIRTUAL_REF_* so that they don't show up in the backend.
if op.getopnum() == rop.CALL_PURE:
op = ResOperation(rop.CALL, op.getarglist()[1:], op.result,
op.getdescr())
+ elif op.getopnum() == rop.CALL_LOOPINVARIANT:
+ op = op.copy_and_change(rop.CALL)
elif op.getopnum() == rop.VIRTUAL_REF:
op = ResOperation(rop.SAME_AS, [op.getarg(0)], op.result)
elif op.getopnum() == rop.VIRTUAL_REF_FINISH:
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py Fri Oct 22 23:09:43 2010
@@ -2,6 +2,7 @@
import sys
from pypy.rlib.jit import JitDriver, we_are_jitted, hint, dont_look_inside
from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_SIMPLE, loop_invariant
+from pypy.rlib.jit import jit_debug, assert_green, AssertGreenFailed
from pypy.jit.metainterp.warmspot import ll_meta_interp, get_stats
from pypy.jit.backend.llgraph import runner
from pypy.jit.metainterp import pyjitpl, history
@@ -44,6 +45,7 @@
num_green_args = 0
portal_graph = graphs[0]
virtualizable_info = None
+ greenfield_info = None
result_type = result_kind
portal_runner_ptr = "???"
@@ -1644,6 +1646,33 @@
res = self.interp_operations(f, [10, 3.5])
assert res == 3.5
+ def test_jit_debug(self):
+ myjitdriver = JitDriver(greens = [], reds = ['x'])
+ class A:
+ pass
+ def f(x):
+ while x > 0:
+ myjitdriver.can_enter_jit(x=x)
+ myjitdriver.jit_merge_point(x=x)
+ jit_debug("hi there:", x)
+ jit_debug("foobar")
+ x -= 1
+ return x
+ res = self.meta_interp(f, [8])
+ assert res == 0
+ self.check_loops(jit_debug=2)
+
+ def test_assert_green(self):
+ def f(x, promote):
+ if promote:
+ x = hint(x, promote=True)
+ assert_green(x)
+ return x
+ res = self.interp_operations(f, [8, 1])
+ assert res == 8
+ py.test.raises(AssertGreenFailed, self.interp_operations, f, [8, 0])
+
+
class TestOOtype(BasicTests, OOJitMixin):
def test_oohash(self):
@@ -1751,7 +1780,7 @@
c = bool(p1)
d = not bool(p2)
return 1000*a + 100*b + 10*c + d
- prebuilt = [lltype.malloc(TP, flavor='raw')] * 2
+ prebuilt = [lltype.malloc(TP, flavor='raw', immortal=True)] * 2
expected = f(0, 1)
assert self.interp_operations(f, [0, 1]) == expected
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_compile.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_compile.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_compile.py Fri Oct 22 23:09:43 2010
@@ -6,8 +6,8 @@
from pypy.jit.metainterp.compile import ResumeGuardCountersInt
from pypy.jit.metainterp.compile import compile_tmp_callback
from pypy.jit.metainterp import optimize, jitprof, typesystem, compile
-from pypy.jit.metainterp.test.oparser import parse
from pypy.jit.metainterp.test.test_optimizefindnode import LLtypeMixin
+from pypy.jit.tool.oparser import parse
def test_insert_loop_token():
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_jitdriver.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_jitdriver.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_jitdriver.py Fri Oct 22 23:09:43 2010
@@ -72,6 +72,33 @@
# we expect no loop at all for 'loop1': it should always be inlined
self.check_tree_loop_count(2) # 1 x loop, 1 x enter bridge
+ def test_inactive_jitdriver(self):
+ myjitdriver1 = JitDriver(greens=[], reds=['n', 'm'],
+ get_printable_location = getloc1)
+ myjitdriver2 = JitDriver(greens=['g'], reds=['r'],
+ get_printable_location = getloc2)
+ #
+ myjitdriver1.active = False # <===
+ #
+ def loop1(n, m):
+ while n > 0:
+ myjitdriver1.can_enter_jit(n=n, m=m)
+ myjitdriver1.jit_merge_point(n=n, m=m)
+ n -= m
+ return n
+ #
+ def loop2(g, r):
+ while r > 0:
+ myjitdriver2.can_enter_jit(g=g, r=r)
+ myjitdriver2.jit_merge_point(g=g, r=r)
+ r += loop1(r, g) + (-1)
+ return r
+ #
+ res = self.meta_interp(loop2, [4, 40], repeat=7, inline=True)
+ assert res == loop2(4, 40)
+ # we expect no int_sub, but a residual call
+ self.check_loops(int_sub=0, call=1)
+
class TestLLtype(MultipleJitDriversTests, LLJitMixin):
pass
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py Fri Oct 22 23:09:43 2010
@@ -1,6 +1,6 @@
import sys
from pypy.rlib import debug
-from pypy.jit.metainterp.test.oparser import pure_parse
+from pypy.jit.tool.oparser import pure_parse
from pypy.jit.metainterp import logger
from pypy.jit.metainterp.typesystem import llhelper
from StringIO import StringIO
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_oparser.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_oparser.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_oparser.py Fri Oct 22 23:09:43 2010
@@ -1,7 +1,7 @@
from pypy.rpython.lltypesystem import lltype, llmemory
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.jit.metainterp.resoperation import rop
from pypy.jit.metainterp.history import AbstractDescr, BoxInt, LoopToken,\
BoxFloat
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizefindnode.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizefindnode.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizefindnode.py Fri Oct 22 23:09:43 2010
@@ -18,7 +18,7 @@
from pypy.jit.metainterp.specnode import ConstantSpecNode
from pypy.jit.codewriter.effectinfo import EffectInfo
from pypy.jit.codewriter.heaptracker import register_known_gctype, adr2int
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
def test_sort_descrs():
class PseudoDescr(AbstractDescr):
@@ -117,33 +117,32 @@
EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE))
arraycopydescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
EffectInfo([], [], [], oopspecindex=EffectInfo.OS_ARRAYCOPY))
- strconcatdescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_CONCAT))
- slicedescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_SLICE))
- strequaldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_EQUAL))
- streq_slice_checknull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_CHECKNULL))
- streq_slice_nonnull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_NONNULL))
- streq_slice_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_CHAR))
- streq_nonnull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_NONNULL))
- streq_nonnull_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_NONNULL_CHAR))
- streq_checknull_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_CHECKNULL_CHAR))
- streq_lengthok_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_LENGTHOK))
+
+ for _name, _os in [
+ ('strconcatdescr', 'OS_STR_CONCAT'),
+ ('strslicedescr', 'OS_STR_SLICE'),
+ ('strequaldescr', 'OS_STR_EQUAL'),
+ ('streq_slice_checknull_descr', 'OS_STREQ_SLICE_CHECKNULL'),
+ ('streq_slice_nonnull_descr', 'OS_STREQ_SLICE_NONNULL'),
+ ('streq_slice_char_descr', 'OS_STREQ_SLICE_CHAR'),
+ ('streq_nonnull_descr', 'OS_STREQ_NONNULL'),
+ ('streq_nonnull_char_descr', 'OS_STREQ_NONNULL_CHAR'),
+ ('streq_checknull_char_descr', 'OS_STREQ_CHECKNULL_CHAR'),
+ ('streq_lengthok_descr', 'OS_STREQ_LENGTHOK'),
+ ]:
+ _oopspecindex = getattr(EffectInfo, _os)
+ locals()[_name] = \
+ cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=_oopspecindex))
+ #
+ _oopspecindex = getattr(EffectInfo, _os.replace('STR', 'UNI'))
+ locals()[_name.replace('str', 'unicode')] = \
+ cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=_oopspecindex))
+
+ s2u_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR2UNICODE))
+ #
class LoopToken(AbstractDescr):
pass
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py Fri Oct 22 23:09:43 2010
@@ -12,7 +12,7 @@
from pypy.jit.metainterp.jitprof import EmptyProfiler
from pypy.jit.metainterp import executor, compile, resume, history
from pypy.jit.metainterp.resoperation import rop, opname, ResOperation
-from pypy.jit.metainterp.test.oparser import pure_parse
+from pypy.jit.tool.oparser import pure_parse
##class FakeFrame(object):
## parent_resumedata_snapshot = None
@@ -132,14 +132,21 @@
# ____________________________________________________________
def equaloplists(oplist1, oplist2, strict_fail_args=True, remap={}):
- print '-'*20, 'Comparing lists', '-'*20
+ # try to use the full width of the terminal to display the list
+ # unfortunately, does not work with the default capture method of py.test
+ # (which is fd), you you need to use either -s or --capture=sys, else you
+ # get the standard 80 columns width
+ totwidth = py.io.get_terminal_width()
+ width = totwidth / 2 - 1
+ print ' Comparing lists '.center(totwidth, '-')
+ print '%s| %s' % ('optimized'.center(width), 'expected'.center(width))
for op1, op2 in zip(oplist1, oplist2):
txt1 = str(op1)
txt2 = str(op2)
while txt1 or txt2:
- print '%-39s| %s' % (txt1[:39], txt2[:39])
- txt1 = txt1[39:]
- txt2 = txt2[39:]
+ print '%s| %s' % (txt1[:width].ljust(width), txt2[:width])
+ txt1 = txt1[width:]
+ txt2 = txt2[width:]
assert op1.getopnum() == op2.getopnum()
assert op1.numargs() == op2.numargs()
for i in range(op1.numargs()):
@@ -262,6 +269,10 @@
expected = self.parse(optops)
print '\n'.join([str(o) for o in loop.operations])
self.assert_equal(loop, expected)
+ return loop
+
+
+class OptimizeOptTest(BaseTestOptimizeOpt):
def test_simple(self):
ops = """
@@ -2643,7 +2654,7 @@
''', rop.GUARD_TRUE)
-class TestLLtype(BaseTestOptimizeOpt, LLtypeMixin):
+class TestLLtype(OptimizeOptTest, LLtypeMixin):
def test_residual_call_does_not_invalidate_caches(self):
ops = """
@@ -3487,7 +3498,7 @@
i0 = strlen(p0)
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not', expected)
def test_addsub_const(self):
ops = """
@@ -3893,6 +3904,15 @@
"""
self.optimize_loop(ops, 'Not, Not', expected)
+ # ----------
+ def optimize_strunicode_loop(self, ops, spectext, optops):
+ # check with the arguments passed in
+ self.optimize_loop(ops, spectext, optops)
+ # check with replacing 'str' with 'unicode' everywhere
+ self.optimize_loop(ops.replace('str','unicode').replace('s"', 'u"'),
+ spectext,
+ optops.replace('str','unicode').replace('s"', 'u"'))
+
def test_newstr_1(self):
ops = """
[i0]
@@ -3905,7 +3925,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not', expected)
def test_newstr_2(self):
ops = """
@@ -3921,7 +3941,7 @@
[i0, i1]
jump(i1, i0)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_concat_1(self):
ops = """
@@ -3942,7 +3962,7 @@
copystrcontent(p2, p3, 0, i4, i5)
jump(p2, p3)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_concat_vstr2_str(self):
ops = """
@@ -3965,7 +3985,7 @@
copystrcontent(p2, p3, 0, 2, i4)
jump(i1, i0, p3)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_concat_str_vstr2(self):
ops = """
@@ -3989,7 +4009,7 @@
i6 = int_add(i5, 1) # will be killed by the backend
jump(i1, i0, p3)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_concat_str_str_str(self):
ops = """
@@ -4016,12 +4036,12 @@
copystrcontent(p3, p5, 0, i12b, i3b)
jump(p2, p3, p5)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_concat_str_cstr1(self):
ops = """
[p2]
- p3 = call(0, p2, "x", descr=strconcatdescr)
+ p3 = call(0, p2, s"x", descr=strconcatdescr)
jump(p3)
"""
expected = """
@@ -4035,28 +4055,28 @@
i5 = int_add(i4, 1) # will be killed by the backend
jump(p3)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not', expected)
def test_str_concat_consts(self):
ops = """
[]
- p1 = same_as("ab")
- p2 = same_as("cde")
+ p1 = same_as(s"ab")
+ p2 = same_as(s"cde")
p3 = call(0, p1, p2, descr=strconcatdescr)
escape(p3)
jump()
"""
expected = """
[]
- escape("abcde")
+ escape(s"abcde")
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_strunicode_loop(ops, '', expected)
def test_str_slice_1(self):
ops = """
[p1, i1, i2]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
jump(p2, i1, i2)
"""
expected = """
@@ -4066,12 +4086,12 @@
copystrcontent(p1, p2, i1, 0, i3)
jump(p2, i1, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_slice_2(self):
ops = """
[p1, i2]
- p2 = call(0, p1, 0, i2, descr=slicedescr)
+ p2 = call(0, p1, 0, i2, descr=strslicedescr)
jump(p2, i2)
"""
expected = """
@@ -4080,13 +4100,13 @@
copystrcontent(p1, p2, 0, 0, i2)
jump(p2, i2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_slice_3(self):
ops = """
[p1, i1, i2, i3, i4]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
- p3 = call(0, p2, i3, i4, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
+ p3 = call(0, p2, i3, i4, descr=strslicedescr)
jump(p3, i1, i2, i3, i4)
"""
expected = """
@@ -4098,12 +4118,12 @@
copystrcontent(p1, p3, i6, 0, i5)
jump(p3, i1, i2, i3, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not, Not', expected)
def test_str_slice_getitem1(self):
ops = """
[p1, i1, i2, i3]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
i4 = strgetitem(p2, i3)
escape(i4)
jump(p1, i1, i2, i3)
@@ -4116,7 +4136,7 @@
escape(i4)
jump(p1, i1, i2, i3)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not', expected)
def test_str_slice_plain(self):
ops = """
@@ -4124,7 +4144,7 @@
p1 = newstr(2)
strsetitem(p1, 0, i3)
strsetitem(p1, 1, i4)
- p2 = call(0, p1, 1, 2, descr=slicedescr)
+ p2 = call(0, p1, 1, 2, descr=strslicedescr)
i5 = strgetitem(p2, 0)
escape(i5)
jump(i3, i4)
@@ -4134,12 +4154,12 @@
escape(i4)
jump(i3, i4)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_slice_concat(self):
ops = """
[p1, i1, i2, p2]
- p3 = call(0, p1, i1, i2, descr=slicedescr)
+ p3 = call(0, p1, i1, i2, descr=strslicedescr)
p4 = call(0, p3, p2, descr=strconcatdescr)
jump(p4, i1, i2, p2)
"""
@@ -4155,10 +4175,10 @@
copystrcontent(p2, p4, 0, i3, i4b)
jump(p4, i1, i2, p2)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not', expected)
# ----------
- def optimize_loop_extradescrs(self, ops, spectext, optops):
+ def optimize_strunicode_loop_extradescrs(self, ops, spectext, optops):
from pypy.jit.metainterp.optimizeopt import string
def my_callinfo_for_oopspec(oopspecindex):
calldescrtype = type(LLtypeMixin.strequaldescr)
@@ -4173,7 +4193,7 @@
saved = string.callinfo_for_oopspec
try:
string.callinfo_for_oopspec = my_callinfo_for_oopspec
- self.optimize_loop(ops, spectext, optops)
+ self.optimize_strunicode_loop(ops, spectext, optops)
finally:
string.callinfo_for_oopspec = saved
@@ -4184,7 +4204,7 @@
escape(i0)
jump(p1, p2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', ops)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', ops)
def test_str_equal_noop2(self):
ops = """
@@ -4209,12 +4229,13 @@
escape(i0)
jump(p1, p2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not',
+ expected)
def test_str_equal_slice1(self):
ops = """
[p1, i1, i2, p3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p4, p3, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4226,12 +4247,13 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_slice2(self):
ops = """
[p1, i1, i2, p3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p3, p4, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4243,13 +4265,14 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_slice3(self):
ops = """
[p1, i1, i2, p3]
guard_nonnull(p3) []
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p3, p4, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4262,13 +4285,14 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_slice4(self):
ops = """
[p1, i1, i2]
- p3 = call(0, p1, i1, i2, descr=slicedescr)
- i0 = call(0, p3, "x", descr=strequaldescr)
+ p3 = call(0, p1, i1, i2, descr=strslicedescr)
+ i0 = call(0, p3, s"x", descr=strequaldescr)
escape(i0)
jump(p1, i1, i2)
"""
@@ -4279,12 +4303,13 @@
escape(i0)
jump(p1, i1, i2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not',
+ expected)
def test_str_equal_slice5(self):
ops = """
[p1, i1, i2, i3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
p5 = newstr(1)
strsetitem(p5, 0, i3)
i0 = call(0, p5, p4, descr=strequaldescr)
@@ -4298,7 +4323,8 @@
escape(i0)
jump(p1, i1, i2, i3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_none1(self):
ops = """
@@ -4313,7 +4339,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_none2(self):
ops = """
@@ -4328,30 +4354,30 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull1(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "hello world", descr=strequaldescr)
+ i0 = call(0, p1, s"hello world", descr=strequaldescr)
escape(i0)
jump(p1)
"""
expected = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "hello world", descr=streq_nonnull_descr)
+ i0 = call(0, p1, s"hello world", descr=streq_nonnull_descr)
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull2(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "", descr=strequaldescr)
+ i0 = call(0, p1, s"", descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4363,13 +4389,13 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull3(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "x", descr=strequaldescr)
+ i0 = call(0, p1, s"x", descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4380,13 +4406,13 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull4(self):
ops = """
[p1, p2]
p4 = call(0, p1, p2, descr=strconcatdescr)
- i0 = call(0, "hello world", p4, descr=strequaldescr)
+ i0 = call(0, s"hello world", p4, descr=strequaldescr)
escape(i0)
jump(p1, p2)
"""
@@ -4401,17 +4427,17 @@
i5 = strlen(p2)
i6 = int_add(i4, i5) # will be killed by the backend
copystrcontent(p2, p4, 0, i4, i5)
- i0 = call(0, "hello world", p4, descr=streq_nonnull_descr)
+ i0 = call(0, s"hello world", p4, descr=streq_nonnull_descr)
escape(i0)
jump(p1, p2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', expected)
def test_str_equal_chars0(self):
ops = """
[i1]
p1 = newstr(0)
- i0 = call(0, p1, "", descr=strequaldescr)
+ i0 = call(0, p1, s"", descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4420,14 +4446,14 @@
escape(1)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_chars1(self):
ops = """
[i1]
p1 = newstr(1)
strsetitem(p1, 0, i1)
- i0 = call(0, p1, "x", descr=strequaldescr)
+ i0 = call(0, p1, s"x", descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4437,7 +4463,7 @@
escape(i0)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_chars2(self):
ops = """
@@ -4445,7 +4471,7 @@
p1 = newstr(2)
strsetitem(p1, 0, i1)
strsetitem(p1, 1, i2)
- i0 = call(0, p1, "xy", descr=strequaldescr)
+ i0 = call(0, p1, s"xy", descr=strequaldescr)
escape(i0)
jump(i1, i2)
"""
@@ -4454,16 +4480,16 @@
p1 = newstr(2)
strsetitem(p1, 0, i1)
strsetitem(p1, 1, i2)
- i0 = call(0, p1, "xy", descr=streq_lengthok_descr)
+ i0 = call(0, p1, s"xy", descr=streq_lengthok_descr)
escape(i0)
jump(i1, i2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', expected)
def test_str_equal_chars3(self):
ops = """
[p1]
- i0 = call(0, "x", p1, descr=strequaldescr)
+ i0 = call(0, s"x", p1, descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4473,14 +4499,14 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_lengthmismatch1(self):
ops = """
[i1]
p1 = newstr(1)
strsetitem(p1, 0, i1)
- i0 = call(0, "xy", p1, descr=strequaldescr)
+ i0 = call(0, s"xy", p1, descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4489,13 +4515,36 @@
escape(0)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
- # XXX unicode operations
- # XXX str2unicode
+ def test_str2unicode_constant(self):
+ ops = """
+ []
+ p0 = call(0, "xy", descr=s2u_descr) # string -> unicode
+ escape(p0)
+ jump()
+ """
+ expected = """
+ []
+ escape(u"xy")
+ jump()
+ """
+ self.optimize_strunicode_loop_extradescrs(ops, '', expected)
+
+ def test_str2unicode_nonconstant(self):
+ ops = """
+ [p0]
+ p1 = call(0, p0, descr=s2u_descr) # string -> unicode
+ escape(p1)
+ jump(p1)
+ """
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', ops)
+ # more generally, supporting non-constant but virtual cases is
+ # not obvious, because of the exception UnicodeDecodeError that
+ # can be raised by ll_str2unicode()
-##class TestOOtype(BaseTestOptimizeOpt, OOtypeMixin):
+##class TestOOtype(OptimizeOptTest, OOtypeMixin):
## def test_instanceof(self):
## ops = """
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py Fri Oct 22 23:09:43 2010
@@ -240,6 +240,17 @@
return FakeBuiltObject(strconcat=[left, right])
def slice_string(self, str, start, length):
return FakeBuiltObject(strslice=[str, start, length])
+ def allocate_unicode(self, length):
+ return FakeBuiltObject(unistring=[None]*length)
+ def unicode_setitem(self, unistring, i, fieldnum):
+ value, tag = untag(fieldnum)
+ assert tag == TAGINT
+ assert 0 <= i < len(unistring.unistring)
+ unistring.unistring[i] = value
+ def concat_unicodes(self, left, right):
+ return FakeBuiltObject(uniconcat=[left, right])
+ def slice_unicode(self, str, start, length):
+ return FakeBuiltObject(unislice=[str, start, length])
class FakeBuiltObject(object):
def __init__(self, **kwds):
@@ -304,6 +315,30 @@
assert reader.force_all_virtuals() == [
FakeBuiltObject(strslice=info.fieldnums)]
+def test_vuniplaininfo():
+ info = VUniPlainInfo()
+ info.fieldnums = [tag(60, TAGINT)]
+ reader = FakeResumeDataReader()
+ reader._prepare_virtuals([info])
+ assert reader.force_all_virtuals() == [
+ FakeBuiltObject(unistring=[60])]
+
+def test_vuniconcatinfo():
+ info = VUniConcatInfo()
+ info.fieldnums = [tag(10, TAGBOX), tag(20, TAGBOX)]
+ reader = FakeResumeDataReader()
+ reader._prepare_virtuals([info])
+ assert reader.force_all_virtuals() == [
+ FakeBuiltObject(uniconcat=info.fieldnums)]
+
+def test_vunisliceinfo():
+ info = VUniSliceInfo()
+ info.fieldnums = [tag(10, TAGBOX), tag(20, TAGBOX), tag(30, TAGBOX)]
+ reader = FakeResumeDataReader()
+ reader._prepare_virtuals([info])
+ assert reader.force_all_virtuals() == [
+ FakeBuiltObject(unislice=info.fieldnums)]
+
# ____________________________________________________________
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_string.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_string.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_string.py Fri Oct 22 23:09:43 2010
@@ -6,14 +6,17 @@
class StringTests:
+ _str, _chr = str, chr
+
def test_eq_residual(self):
+ _str = self._str
jitdriver = JitDriver(greens = [], reds = ['n', 'i', 's'])
- global_s = "hello"
+ global_s = _str("hello")
def f(n, b, s):
if b:
- s += "ello"
+ s += _str("ello")
else:
- s += "allo"
+ s += _str("allo")
i = 0
while n > 0:
jitdriver.can_enter_jit(s=s, n=n, i=i)
@@ -21,18 +24,19 @@
n -= 1 + (s == global_s)
i += 1
return i
- res = self.meta_interp(f, [10, True, 'h'], listops=True)
+ res = self.meta_interp(f, [10, True, _str('h')], listops=True)
assert res == 5
self.check_loops(**{self.CALL: 1, self.CALL_PURE: 0})
def test_eq_folded(self):
+ _str = self._str
jitdriver = JitDriver(greens = ['s'], reds = ['n', 'i'])
- global_s = "hello"
+ global_s = _str("hello")
def f(n, b, s):
if b:
- s += "ello"
+ s += _str("ello")
else:
- s += "allo"
+ s += _str("allo")
i = 0
while n > 0:
jitdriver.can_enter_jit(s=s, n=n, i=i)
@@ -40,31 +44,18 @@
n -= 1 + (s == global_s)
i += 1
return i
- res = self.meta_interp(f, [10, True, 'h'], listops=True)
+ res = self.meta_interp(f, [10, True, _str('h')], listops=True)
assert res == 5
self.check_loops(**{self.CALL: 0, self.CALL_PURE: 0})
def test_newstr(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['n', 'm'])
def f(n, m):
while True:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- bytecode = 'adlfkj' + chr(n)
- res = bytecode[n]
- m -= 1
- if m < 0:
- return ord(res)
- res = self.meta_interp(f, [6, 10])
- assert res == 6
-
- def test_newunicode(self):
- jitdriver = JitDriver(greens = [], reds = ['n', 'm'])
- def f(n, m):
- while True:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- bytecode = u'adlfkj' + unichr(n)
+ bytecode = _str('adlfkj') + _chr(n)
res = bytecode[n]
m -= 1
if m < 0:
@@ -73,95 +64,96 @@
assert res == 6
def test_char2string_pure(self):
- for dochr in [chr, ]: #unichr]:
- jitdriver = JitDriver(greens = [], reds = ['n'])
- @dont_look_inside
- def escape(x):
- pass
- def f(n):
- while n > 0:
- jitdriver.can_enter_jit(n=n)
- jitdriver.jit_merge_point(n=n)
- s = dochr(n)
- if not we_are_jitted():
- s += s # forces to be a string
- if n > 100:
- escape(s)
- n -= 1
- return 42
- self.meta_interp(f, [6])
- self.check_loops(newstr=0, strsetitem=0, strlen=0,
- newunicode=0, unicodesetitem=0, unicodelen=0)
+ _str, _chr = self._str, self._chr
+ jitdriver = JitDriver(greens = [], reds = ['n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ def f(n):
+ while n > 0:
+ jitdriver.can_enter_jit(n=n)
+ jitdriver.jit_merge_point(n=n)
+ s = _chr(n)
+ if not we_are_jitted():
+ s += s # forces to be a string
+ if n > 100:
+ escape(s)
+ n -= 1
+ return 42
+ self.meta_interp(f, [6])
+ self.check_loops(newstr=0, strsetitem=0, strlen=0,
+ newunicode=0, unicodesetitem=0, unicodelen=0)
def test_char2string_escape(self):
- for dochr in [chr, ]: #unichr]:
- jitdriver = JitDriver(greens = [], reds = ['n', 'total'])
- @dont_look_inside
- def escape(x):
- return ord(x[0])
- def f(n):
- total = 0
- while n > 0:
- jitdriver.can_enter_jit(n=n, total=total)
- jitdriver.jit_merge_point(n=n, total=total)
- s = dochr(n)
- if not we_are_jitted():
- s += s # forces to be a string
- total += escape(s)
- n -= 1
- return total
- res = self.meta_interp(f, [6])
- assert res == 21
+ _str, _chr = self._str, self._chr
+ jitdriver = JitDriver(greens = [], reds = ['n', 'total'])
+ @dont_look_inside
+ def escape(x):
+ return ord(x[0])
+ def f(n):
+ total = 0
+ while n > 0:
+ jitdriver.can_enter_jit(n=n, total=total)
+ jitdriver.jit_merge_point(n=n, total=total)
+ s = _chr(n)
+ if not we_are_jitted():
+ s += s # forces to be a string
+ total += escape(s)
+ n -= 1
+ return total
+ res = self.meta_interp(f, [6])
+ assert res == 21
def test_char2string2char(self):
- for dochr in [chr, ]: #unichr]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'total'])
- def f(m):
- total = 0
- while m > 0:
- jitdriver.can_enter_jit(m=m, total=total)
- jitdriver.jit_merge_point(m=m, total=total)
- string = dochr(m)
- if m > 100:
- string += string # forces to be a string
- # read back the character
- c = string[0]
- total += ord(c)
- m -= 1
- return total
- res = self.meta_interp(f, [6])
- assert res == 21
- self.check_loops(newstr=0, strgetitem=0, strsetitem=0, strlen=0,
- newunicode=0, unicodegetitem=0, unicodesetitem=0,
- unicodelen=0)
+ _str, _chr = self._str, self._chr
+ jitdriver = JitDriver(greens = [], reds = ['m', 'total'])
+ def f(m):
+ total = 0
+ while m > 0:
+ jitdriver.can_enter_jit(m=m, total=total)
+ jitdriver.jit_merge_point(m=m, total=total)
+ string = _chr(m)
+ if m > 100:
+ string += string # forces to be a string
+ # read back the character
+ c = string[0]
+ total += ord(c)
+ m -= 1
+ return total
+ res = self.meta_interp(f, [6])
+ assert res == 21
+ self.check_loops(newstr=0, strgetitem=0, strsetitem=0, strlen=0,
+ newunicode=0, unicodegetitem=0, unicodesetitem=0,
+ unicodelen=0)
def test_strconcat_pure(self):
- for somestr in ["abc", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- mylist = [somestr+str(i) for i in range(10)]
- def f(n, m):
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + mylist[m]
- if m > 100:
- escape(s)
- m -= 1
- return 42
- self.meta_interp(f, [6, 7])
- self.check_loops(newstr=0, strsetitem=0,
- newunicode=0, unicodesetitem=0,
- call=0, call_pure=0)
+ _str = self._str
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ mylist = [_str("abc") + _str(i) for i in range(10)]
+ def f(n, m):
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = mylist[n] + mylist[m]
+ if m > 100:
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 7])
+ self.check_loops(newstr=0, strsetitem=0,
+ newunicode=0, unicodesetitem=0,
+ call=0, call_pure=0)
def test_strconcat_escape_str_str(self):
+ _str = self._str
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
@@ -171,46 +163,64 @@
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=0, copystrcontent=2,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=0, copystrcontent=2,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=0,
+ copyunicodecontent=2,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_str_char(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + chr(m)
+ s = mylist[n] + _chr(m)
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=1,
+ copyunicodecontent=1,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_char_str(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = chr(n) + mylist[m]
+ s = _chr(n) + mylist[m]
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=1,
+ copyunicodecontent=1,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_char_char(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
@@ -219,91 +229,132 @@
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = chr(n) + chr(m)
+ s = _chr(n) + _chr(m)
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=2, copystrcontent=0,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=2, copystrcontent=0,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=2,
+ copyunicodecontent=0,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_str_char_str(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + chr(n) + mylist[m]
+ s = mylist[n] + _chr(n) + mylist[m]
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=1, copystrcontent=2,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=1, copystrcontent=2,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=1,
+ copyunicodecontent=2,
+ call=1, call_pure=0) # escape
def test_strconcat_guard_fail(self):
- for somestr in ["abc", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- mylist = [somestr+str(i) for i in range(12)]
- def f(n, m):
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + mylist[m]
- if m & 1:
- escape(s)
- m -= 1
- return 42
- self.meta_interp(f, [6, 10])
+ _str = self._str
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ mylist = [_str("abc") + _str(i) for i in range(12)]
+ def f(n, m):
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = mylist[n] + mylist[m]
+ if m & 1:
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 10])
def test_strslice(self):
- for somestr in ["abc", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- def f(n, m):
- assert n >= 0
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = "foobarbazetc"[m:n]
- if m <= 5:
- escape(s)
- m -= 1
- return 42
- self.meta_interp(f, [10, 10])
+ _str = self._str
+ longstring = _str("foobarbazetc")
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ def f(n, m):
+ assert n >= 0
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = longstring[m:n]
+ if m <= 5:
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [10, 10])
def test_streq_char(self):
- for somestr in ["?abcdefg", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- def f(n, m):
- assert n >= 0
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = somestr[:m]
- escape(s == "?")
- m -= 1
- return 42
- self.meta_interp(f, [6, 7])
- self.check_loops(newstr=0, newunicode=0)
-
-
-class TestOOtype(StringTests, OOJitMixin):
- CALL = "oosend"
- CALL_PURE = "oosend_pure"
+ _str = self._str
+ longstring = _str("?abcdefg")
+ somechar = _str("?")
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ def f(n, m):
+ assert n >= 0
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = longstring[:m]
+ escape(s == somechar)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 7])
+ self.check_loops(newstr=0, newunicode=0)
+
+
+#class TestOOtype(StringTests, OOJitMixin):
+# CALL = "oosend"
+# CALL_PURE = "oosend_pure"
class TestLLtype(StringTests, LLJitMixin):
CALL = "call"
CALL_PURE = "call_pure"
+
+class TestLLtypeUnicode(TestLLtype):
+ _str, _chr = unicode, unichr
+
+ def test_str2unicode(self):
+ _str = self._str
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ class Foo:
+ pass
+ @dont_look_inside
+ def escape(x):
+ assert x == _str("6y")
+ def f(n, m):
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ foo = Foo()
+ foo.y = chr(m)
+ foo.y = "y"
+ s = _str(str(n)) + _str(foo.y)
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 7])
+ self.check_loops(call=3, # str(), _str(), escape()
+ newunicode=1, unicodegetitem=0,
+ unicodesetitem=1, copyunicodecontent=1)
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_virtualref.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_virtualref.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_virtualref.py Fri Oct 22 23:09:43 2010
@@ -93,7 +93,7 @@
lst = []
vrefinfo.continue_tracing = lambda vref, virtual: \
lst.append((vref, virtual))
- resumereader.consume_vref_and_vable(vrefinfo, None)
+ resumereader.consume_vref_and_vable(vrefinfo, None, None)
del vrefinfo.continue_tracing
assert len(lst) == 1
lltype.cast_opaque_ptr(lltype.Ptr(JIT_VIRTUAL_REF),
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmspot.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmspot.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmspot.py Fri Oct 22 23:09:43 2010
@@ -296,6 +296,69 @@
assert res == 1
self.check_loops(int_add=1) # I get 13 without the loop_header()
+ def test_omit_can_enter_jit(self):
+ # Simple test comparing the effects of always giving a can_enter_jit(),
+ # or not giving any. Mostly equivalent, except that if given, it is
+ # ignored the first time, and so it ends up taking one extra loop to
+ # start JITting.
+ mydriver = JitDriver(greens=[], reds=['m'])
+ #
+ for i2 in range(10):
+ def f2(m):
+ while m > 0:
+ mydriver.jit_merge_point(m=m)
+ m -= 1
+ self.meta_interp(f2, [i2])
+ try:
+ self.check_tree_loop_count(1)
+ break
+ except AssertionError:
+ print "f2: no loop generated for i2==%d" % i2
+ else:
+ raise # re-raise the AssertionError: check_loop_count never 1
+ #
+ for i1 in range(10):
+ def f1(m):
+ while m > 0:
+ mydriver.can_enter_jit(m=m)
+ mydriver.jit_merge_point(m=m)
+ m -= 1
+ self.meta_interp(f1, [i1])
+ try:
+ self.check_tree_loop_count(1)
+ break
+ except AssertionError:
+ print "f1: no loop generated for i1==%d" % i1
+ else:
+ raise # re-raise the AssertionError: check_loop_count never 1
+ #
+ assert i1 - 1 == i2
+
+ def test_no_loop_at_all(self):
+ mydriver = JitDriver(greens=[], reds=['m'])
+ def f2(m):
+ mydriver.jit_merge_point(m=m)
+ return m - 1
+ def f1(m):
+ while m > 0:
+ m = f2(m)
+ self.meta_interp(f1, [8])
+ # it should generate one "loop" only, which ends in a FINISH
+ # corresponding to the return from f2.
+ self.check_tree_loop_count(1)
+ self.check_loop_count(0)
+
+ def test_simple_loop(self):
+ mydriver = JitDriver(greens=[], reds=['m'])
+ def f1(m):
+ while m > 0:
+ mydriver.jit_merge_point(m=m)
+ m = m - 1
+ self.meta_interp(f1, [8])
+ self.check_loop_count(1)
+ self.check_loops({'int_sub': 1, 'int_gt': 1, 'guard_true': 1,
+ 'jump': 1})
+
class TestLLWarmspot(WarmspotTests, LLJitMixin):
CPUClass = runner.LLtypeCPU
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/virtualref.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/virtualref.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/virtualref.py Fri Oct 22 23:09:43 2010
@@ -16,7 +16,8 @@
('virtualref_index', lltype.Signed),
('forced', rclass.OBJECTPTR))
self.jit_virtual_ref_vtable = lltype.malloc(rclass.OBJECT_VTABLE,
- zero=True, flavor='raw')
+ zero=True, flavor='raw',
+ immortal=True)
self.jit_virtual_ref_vtable.name = rclass.alloc_array_name(
'jit_virtual_ref')
# build some constants
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py Fri Oct 22 23:09:43 2010
@@ -110,15 +110,16 @@
for i in range(len(block.operations)):
op = block.operations[i]
if (op.opname == 'jit_marker' and
- op.args[0].value == marker_name):
+ op.args[0].value == marker_name and
+ op.args[1].value.active): # the jitdriver
results.append((graph, block, i))
return results
def find_can_enter_jit(graphs):
- results = _find_jit_marker(graphs, 'can_enter_jit')
- if not results:
- raise Exception("no can_enter_jit found!")
- return results
+ return _find_jit_marker(graphs, 'can_enter_jit')
+
+def find_loop_headers(graphs):
+ return _find_jit_marker(graphs, 'loop_header')
def find_jit_merge_points(graphs):
results = _find_jit_marker(graphs, 'jit_merge_point')
@@ -211,9 +212,9 @@
"there are multiple jit_merge_points with the same jitdriver"
def split_graph_and_record_jitdriver(self, graph, block, pos):
- jd = JitDriverStaticData()
- jd._jit_merge_point_pos = (graph, block, pos)
op = block.operations[pos]
+ jd = JitDriverStaticData()
+ jd._jit_merge_point_pos = (graph, op)
args = op.args[2:]
s_binding = self.translator.annotator.binding
jd._portal_args_s = [s_binding(v) for v in args]
@@ -286,10 +287,20 @@
def make_virtualizable_infos(self):
vinfos = {}
for jd in self.jitdrivers_sd:
+ #
+ jd.greenfield_info = None
+ for name in jd.jitdriver.greens:
+ if '.' in name:
+ from pypy.jit.metainterp.greenfield import GreenFieldInfo
+ jd.greenfield_info = GreenFieldInfo(self.cpu, jd)
+ break
+ #
if not jd.jitdriver.virtualizables:
jd.virtualizable_info = None
jd.index_of_virtualizable = -1
continue
+ else:
+ assert jd.greenfield_info is None, "XXX not supported yet"
#
jitdriver = jd.jitdriver
assert len(jitdriver.virtualizables) == 1 # for now
@@ -457,8 +468,7 @@
self.make_args_specification(jd)
def make_args_specification(self, jd):
- graph, block, index = jd._jit_merge_point_pos
- op = block.operations[index]
+ graph, op = jd._jit_merge_point_pos
greens_v, reds_v = support.decode_hp_hint_args(op)
ALLARGS = [v.concretetype for v in (greens_v + reds_v)]
jd._green_args_spec = [v.concretetype for v in greens_v]
@@ -474,26 +484,37 @@
[lltype.Signed, llmemory.GCREF], RESTYPE)
def rewrite_can_enter_jits(self):
- can_enter_jits = find_can_enter_jit(self.translator.graphs)
sublists = {}
for jd in self.jitdrivers_sd:
- sublists[jd.jitdriver] = []
+ sublists[jd.jitdriver] = jd, []
+ jd.no_loop_header = True
+ #
+ loop_headers = find_loop_headers(self.translator.graphs)
+ for graph, block, index in loop_headers:
+ op = block.operations[index]
+ jitdriver = op.args[1].value
+ assert jitdriver in sublists, \
+ "loop_header with no matching jit_merge_point"
+ jd, sublist = sublists[jitdriver]
+ jd.no_loop_header = False
+ #
+ can_enter_jits = find_can_enter_jit(self.translator.graphs)
for graph, block, index in can_enter_jits:
op = block.operations[index]
jitdriver = op.args[1].value
assert jitdriver in sublists, \
"can_enter_jit with no matching jit_merge_point"
+ jd, sublist = sublists[jitdriver]
origportalgraph = jd._jit_merge_point_pos[0]
if graph is not origportalgraph:
- sublists[jitdriver].append((graph, block, index))
+ sublist.append((graph, block, index))
+ jd.no_loop_header = False
else:
pass # a 'can_enter_jit' before the 'jit-merge_point', but
# originally in the same function: we ignore it here
# see e.g. test_jitdriver.test_simple
for jd in self.jitdrivers_sd:
- sublist = sublists[jd.jitdriver]
- assert len(sublist) > 0, \
- "found no can_enter_jit for %r" % (jd.jitdriver,)
+ _, sublist = sublists[jd.jitdriver]
self.rewrite_can_enter_jit(jd, sublist)
def rewrite_can_enter_jit(self, jd, can_enter_jits):
@@ -501,6 +522,19 @@
FUNCPTR = jd._PTR_JIT_ENTER_FUNCTYPE
jit_enter_fnptr = self.helper_func(FUNCPTR, jd._maybe_enter_jit_fn)
+ if len(can_enter_jits) == 0:
+ # see test_warmspot.test_no_loop_at_all
+ operations = jd.portal_graph.startblock.operations
+ op1 = operations[0]
+ assert (op1.opname == 'jit_marker' and
+ op1.args[0].value == 'jit_merge_point')
+ op0 = SpaceOperation(
+ 'jit_marker',
+ [Constant('can_enter_jit', lltype.Void)] + op1.args[1:],
+ None)
+ operations.insert(0, op0)
+ can_enter_jits = [(jd.portal_graph, jd.portal_graph.startblock, 0)]
+
for graph, block, index in can_enter_jits:
if graph is jd._jit_merge_point_pos[0]:
continue
@@ -709,8 +743,14 @@
# ____________________________________________________________
# Now mutate origportalgraph to end with a call to portal_runner_ptr
#
- _, origblock, origindex = jd._jit_merge_point_pos
- op = origblock.operations[origindex]
+ _, op = jd._jit_merge_point_pos
+ for origblock in origportalgraph.iterblocks():
+ if op in origblock.operations:
+ break
+ else:
+ assert False, "lost the operation %r in the graph %r" % (
+ op, origportalgraph)
+ origindex = origblock.operations.index(op)
assert op.opname == 'jit_marker'
assert op.args[0].value == 'jit_merge_point'
greens_v, reds_v = support.decode_hp_hint_args(op)
Modified: pypy/branch/fast-forward/pypy/jit/metainterp/warmstate.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/metainterp/warmstate.py (original)
+++ pypy/branch/fast-forward/pypy/jit/metainterp/warmstate.py Fri Oct 22 23:09:43 2010
@@ -1,5 +1,5 @@
import sys
-from pypy.rpython.lltypesystem import lltype, llmemory, rstr
+from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.annlowlevel import hlstr, llstr, cast_base_ptr_to_instance
from pypy.rpython.annlowlevel import cast_object_to_ptr
@@ -24,7 +24,11 @@
"""
INPUT = lltype.typeOf(x)
if INPUT is lltype.Signed:
- return lltype.cast_primitive(TYPE, x) # XXX missing: Ptr(non-gc)
+ if isinstance(TYPE, lltype.Ptr) and TYPE.TO._gckind == 'raw':
+ # non-gc pointer
+ return rffi.cast(TYPE, x)
+ else:
+ return lltype.cast_primitive(TYPE, x)
elif INPUT is lltype.Float:
assert TYPE is lltype.Float
return x
@@ -172,6 +176,9 @@
meth(default_value)
def set_param_threshold(self, threshold):
+ if threshold < 0:
+ self.increment_threshold = 0 # never reach the THRESHOLD_LIMIT
+ return
if threshold < 2:
threshold = 2
self.increment_threshold = (self.THRESHOLD_LIMIT // threshold) + 1
Modified: pypy/branch/fast-forward/pypy/jit/tl/pypyjit.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/tl/pypyjit.py (original)
+++ pypy/branch/fast-forward/pypy/jit/tl/pypyjit.py Fri Oct 22 23:09:43 2010
@@ -40,6 +40,9 @@
config.objspace.usemodules.array = True
config.objspace.usemodules._weakref = False
config.objspace.usemodules._sre = False
+#
+config.objspace.usemodules._ffi = True
+#
set_pypy_opt_level(config, level='jit')
config.objspace.std.withinlineddict = True
Modified: pypy/branch/fast-forward/pypy/jit/tl/pypyjit_child.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/tl/pypyjit_child.py (original)
+++ pypy/branch/fast-forward/pypy/jit/tl/pypyjit_child.py Fri Oct 22 23:09:43 2010
@@ -16,7 +16,7 @@
interp.heap.malloc_nonmovable = returns_null # XXX
from pypy.jit.backend.llgraph.runner import LLtypeCPU
- LLtypeCPU.supports_floats = False # for now
+ #LLtypeCPU.supports_floats = False # for now
apply_jit(interp, graph, LLtypeCPU)
Modified: pypy/branch/fast-forward/pypy/jit/tool/loopviewer.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/tool/loopviewer.py (original)
+++ pypy/branch/fast-forward/pypy/jit/tool/loopviewer.py Fri Oct 22 23:09:43 2010
@@ -9,7 +9,7 @@
import optparse
from pprint import pprint
from pypy.tool import logparser
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.jit.metainterp.history import ConstInt
from pypy.rpython.lltypesystem import llmemory, lltype
Modified: pypy/branch/fast-forward/pypy/jit/tool/showstats.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/tool/showstats.py (original)
+++ pypy/branch/fast-forward/pypy/jit/tool/showstats.py Fri Oct 22 23:09:43 2010
@@ -4,7 +4,7 @@
import autopath
import sys, py
from pypy.tool import logparser
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.jit.metainterp.resoperation import rop
from pypy.rpython.lltypesystem import lltype, llmemory
Modified: pypy/branch/fast-forward/pypy/jit/tool/traceviewer.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/tool/traceviewer.py (original)
+++ pypy/branch/fast-forward/pypy/jit/tool/traceviewer.py Fri Oct 22 23:09:43 2010
@@ -253,10 +253,10 @@
def main(loopfile, use_threshold, view=True):
countname = py.path.local(loopfile + '.count')
if countname.check():
- counts = [re.split('( 20 and use_threshold:
counts.threshold = l[-20]
Modified: pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py (original)
+++ pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py Fri Oct 22 23:09:43 2010
@@ -2,9 +2,11 @@
from pypy.interpreter.error import OperationError, operationerrfmt
from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped, applevel
from pypy.interpreter.gateway import interp2app, ObjSpace
-from pypy.interpreter.typedef import TypeDef, make_weakref_descr
+from pypy.interpreter.typedef import TypeDef
from pypy.interpreter.argument import Arguments
from pypy.interpreter.baseobjspace import Wrappable
+from pypy.interpreter.typedef import GetSetProperty, descr_get_dict
+from pypy.interpreter.typedef import descr_set_dict
from pypy.rlib.rarithmetic import r_uint, intmask
from pypy.rlib.objectmodel import compute_identity_hash
from pypy.rlib.debug import make_sure_not_resized
@@ -57,6 +59,14 @@
self.bases_w = bases
self.w_dict = w_dict
+ def instantiate(self, space):
+ cache = space.fromcache(Cache)
+ if self.lookup(space, '__del__') is not None:
+ w_inst = cache.cls_with_del(space, self)
+ else:
+ w_inst = cache.cls_without_del(space, self)
+ return w_inst
+
def getdict(self):
return self.w_dict
@@ -100,15 +110,15 @@
return False
@jit.unroll_safe
- def lookup(self, space, w_attr):
+ def lookup(self, space, attr):
# returns w_value or interplevel None
- w_result = space.finditem(self.w_dict, w_attr)
+ w_result = space.finditem_str(self.w_dict, attr)
if w_result is not None:
return w_result
for base in self.bases_w:
# XXX fix annotation of bases_w to be a list of W_ClassObjects
assert isinstance(base, W_ClassObject)
- w_result = base.lookup(space, w_attr)
+ w_result = base.lookup(space, attr)
if w_result is not None:
return w_result
return None
@@ -122,7 +132,7 @@
return space.wrap(self.name)
elif name == "__bases__":
return space.newtuple(self.bases_w)
- w_value = self.lookup(space, w_attr)
+ w_value = self.lookup(space, name)
if w_value is None:
raise operationerrfmt(
space.w_AttributeError,
@@ -147,7 +157,7 @@
self.setbases(space, w_value)
return
elif name == "__del__":
- if self.lookup(space, w_attr) is None:
+ if self.lookup(space, name) is None:
msg = ("a __del__ method added to an existing class "
"will not be called")
space.warn(msg, space.w_RuntimeWarning)
@@ -195,13 +205,20 @@
# NOT_RPYTHON
return '' % self.name
+class Cache:
+ def __init__(self, space):
+ from pypy.interpreter.typedef import _usersubclswithfeature
+ # evil
+ self.cls_without_del = _usersubclswithfeature(
+ space.config, W_InstanceObject, "dict", "weakref")
+ self.cls_with_del = _usersubclswithfeature(
+ space.config, self.cls_without_del, "del")
+
+
def class_descr_call(space, w_self, __args__):
self = space.interp_w(W_ClassObject, w_self)
- if self.lookup(space, space.wrap('__del__')) is not None:
- w_inst = W_InstanceObjectWithDel(space, self)
- else:
- w_inst = W_InstanceObject(space, self)
- w_init = w_inst.getattr_from_class(space, space.wrap('__init__'))
+ w_inst = self.instantiate(space)
+ w_init = w_inst.getattr_from_class(space, '__init__')
if w_init is not None:
w_result = space.call_args(w_init, __args__)
if not space.is_w(w_result, space.w_None):
@@ -235,7 +252,7 @@
def make_unary_instance_method(name):
def unaryop(self, space):
- w_meth = self.getattr(space, space.wrap(name), True)
+ w_meth = self.getattr(space, name, True)
return space.call_function(w_meth)
unaryop.func_name = name
return unaryop
@@ -243,7 +260,7 @@
def make_binary_returning_notimplemented_instance_method(name):
def binaryop(self, space, w_other):
try:
- w_meth = self.getattr(space, space.wrap(name), False)
+ w_meth = self.getattr(space, name, False)
except OperationError, e:
if e.match(space, space.w_AttributeError):
return space.w_NotImplemented
@@ -268,7 +285,7 @@
w_a = self
w_b = w_other
if w_a is self:
- w_meth = self.getattr(space, space.wrap(specialname), False)
+ w_meth = self.getattr(space, specialname, False)
if w_meth is None:
return space.w_NotImplemented
return space.call_function(w_meth, w_b)
@@ -279,7 +296,7 @@
def rbinaryop(self, space, w_other):
w_a, w_b = _coerce_helper(space, self, w_other)
if w_a is None or w_a is self:
- w_meth = self.getattr(space, space.wrap(rspecialname), False)
+ w_meth = self.getattr(space, rspecialname, False)
if w_meth is None:
return space.w_NotImplemented
return space.call_function(w_meth, w_other)
@@ -303,46 +320,34 @@
raise OperationError(
space.w_TypeError,
space.wrap("instance() first arg must be class"))
- if space.is_w(w_dict, space.w_None):
- w_dict = None
- elif not space.is_true(space.isinstance(w_dict, space.w_dict)):
- raise OperationError(
- space.w_TypeError,
- space.wrap("instance() second arg must be dictionary or None"))
- return W_InstanceObject(space, w_class, w_dict)
+ w_result = w_class.instantiate(space)
+ if not space.is_w(w_dict, space.w_None):
+ w_result.setdict(space, w_dict)
+ return w_result
class W_InstanceObject(Wrappable):
- def __init__(self, space, w_class, w_dict=None):
- if w_dict is None:
- w_dict = space.newdict(instance=True)
+ def __init__(self, space, w_class):
+ # note that user_setup is overridden by the typedef.py machinery
+ self.user_setup(space, space.gettypeobject(self.typedef))
assert isinstance(w_class, W_ClassObject)
self.w_class = w_class
- self.w_dict = w_dict
- self.space = space
-
- def getdict(self):
- return self.w_dict
- def setdict(self, space, w_dict):
- if (w_dict is None or
- not space.is_true(space.isinstance(w_dict, space.w_dict))):
- raise OperationError(
- space.w_TypeError,
- space.wrap("__dict__ must be a dictionary object"))
- self.w_dict = w_dict
+ def user_setup(self, space, w_subtype):
+ self.space = space
- def setclass(self, space, w_class):
+ def set_oldstyle_class(self, space, w_class):
if w_class is None or not isinstance(w_class, W_ClassObject):
raise OperationError(
space.w_TypeError,
space.wrap("__class__ must be set to a class"))
self.w_class = w_class
- def getattr_from_class(self, space, w_name):
+ def getattr_from_class(self, space, name):
# Look up w_name in the class dict, and call its __get__.
# This method ignores the instance dict and the __getattr__.
# Returns None if not found.
- w_value = self.w_class.lookup(space, w_name)
+ assert isinstance(name, str)
+ w_value = self.w_class.lookup(space, name)
if w_value is None:
return None
w_descr_get = space.lookup(w_value, '__get__')
@@ -350,19 +355,20 @@
return w_value
return space.call_function(w_descr_get, w_value, self, self.w_class)
- def getattr(self, space, w_name, exc=True):
+ def getattr(self, space, name, exc=True):
# Normal getattr rules: look up w_name in the instance dict,
# in the class dict, and then via a call to __getatttr__.
- w_result = space.finditem(self.w_dict, w_name)
+ assert isinstance(name, str)
+ w_result = self.getdictvalue(space, name)
if w_result is not None:
return w_result
- w_result = self.getattr_from_class(space, w_name)
+ w_result = self.getattr_from_class(space, name)
if w_result is not None:
return w_result
- w_meth = self.getattr_from_class(space, space.wrap('__getattr__'))
+ w_meth = self.getattr_from_class(space, '__getattr__')
if w_meth is not None:
try:
- return space.call_function(w_meth, w_name)
+ return space.call_function(w_meth, space.wrap(name))
except OperationError, e:
if not exc and e.match(space, space.w_AttributeError):
return None # eat the AttributeError
@@ -372,7 +378,7 @@
raise operationerrfmt(
space.w_AttributeError,
"%s instance has no attribute '%s'",
- self.w_class.name, space.str_w(w_name))
+ self.w_class.name, name)
else:
return None
@@ -380,44 +386,46 @@
name = space.str_w(w_attr)
if len(name) >= 8 and name[0] == '_':
if name == "__dict__":
- return self.w_dict
+ return self.getdict()
elif name == "__class__":
return self.w_class
- return self.getattr(space, w_attr)
+ return self.getattr(space, name)
def descr_setattr(self, space, w_name, w_value):
name = unwrap_attr(space, w_name)
- w_meth = self.getattr_from_class(space, space.wrap('__setattr__'))
+ w_meth = self.getattr_from_class(space, '__setattr__')
if name and name[0] == "_":
if name == '__dict__':
self.setdict(space, w_value)
return
if name == '__class__':
- self.setclass(space, w_value)
+ self.set_oldstyle_class(space, w_value)
return
if name == '__del__' and w_meth is None:
- if (not isinstance(self, W_InstanceObjectWithDel)
- and space.finditem(self.w_dict, w_name) is None):
+ cache = space.fromcache(Cache)
+ if (not isinstance(self, cache.cls_with_del)
+ and self.getdictvalue(space, '__del__') is None):
msg = ("a __del__ method added to an instance "
"with no __del__ in the class will not be called")
space.warn(msg, space.w_RuntimeWarning)
if w_meth is not None:
space.call_function(w_meth, w_name, w_value)
else:
- self.setdictvalue(space, name, w_value)
+ # bit obscure: appease normalization
+ self.setdictvalue(space, name, w_value, True)
def descr_delattr(self, space, w_name):
name = unwrap_attr(space, w_name)
if name and name[0] == "_":
if name == '__dict__':
# use setdict to raise the error
- self.setdict(space, None)
+ self.setdict(space, space.w_None)
return
elif name == '__class__':
- # use setclass to raise the error
- self.setclass(space, None)
+ # use set_oldstyle_class to raise the error
+ self.set_oldstyle_class(space, None)
return
- w_meth = self.getattr_from_class(space, space.wrap('__delattr__'))
+ w_meth = self.getattr_from_class(space, '__delattr__')
if w_meth is not None:
space.call_function(w_meth, w_name)
else:
@@ -428,7 +436,7 @@
self.w_class.name, name)
def descr_repr(self, space):
- w_meth = self.getattr(space, space.wrap('__repr__'), False)
+ w_meth = self.getattr(space, '__repr__', False)
if w_meth is None:
w_class = self.w_class
mod = w_class.get_module_string(space)
@@ -436,19 +444,19 @@
return space.call_function(w_meth)
def descr_str(self, space):
- w_meth = self.getattr(space, space.wrap('__str__'), False)
+ w_meth = self.getattr(space, '__str__', False)
if w_meth is None:
return self.descr_repr(space)
return space.call_function(w_meth)
def descr_unicode(self, space):
- w_meth = self.getattr(space, space.wrap('__unicode__'), False)
+ w_meth = self.getattr(space, '__unicode__', False)
if w_meth is None:
return self.descr_str(space)
return space.call_function(w_meth)
def descr_len(self, space):
- w_meth = self.getattr(space, space.wrap('__len__'))
+ w_meth = self.getattr(space, '__len__')
w_result = space.call_function(w_meth)
if space.is_true(space.isinstance(w_result, space.w_int)):
if space.is_true(space.lt(w_result, space.wrap(0))):
@@ -461,22 +469,22 @@
space.wrap("__len__() should return an int"))
def descr_getitem(self, space, w_key):
- w_meth = self.getattr(space, space.wrap('__getitem__'))
+ w_meth = self.getattr(space, '__getitem__')
return space.call_function(w_meth, w_key)
def descr_setitem(self, space, w_key, w_value):
- w_meth = self.getattr(space, space.wrap('__setitem__'))
+ w_meth = self.getattr(space, '__setitem__')
space.call_function(w_meth, w_key, w_value)
def descr_delitem(self, space, w_key):
- w_meth = self.getattr(space, space.wrap('__delitem__'))
+ w_meth = self.getattr(space, '__delitem__')
space.call_function(w_meth, w_key)
def descr_iter(self, space):
- w_meth = self.getattr(space, space.wrap('__iter__'), False)
+ w_meth = self.getattr(space, '__iter__', False)
if w_meth is not None:
return space.call_function(w_meth)
- w_meth = self.getattr(space, space.wrap('__getitem__'), False)
+ w_meth = self.getattr(space, '__getitem__', False)
if w_meth is None:
raise OperationError(
space.w_TypeError,
@@ -486,14 +494,14 @@
# don't see the point
def descr_getslice(self, space, w_i, w_j):
- w_meth = self.getattr(space, space.wrap('__getslice__'), False)
+ w_meth = self.getattr(space, '__getslice__', False)
if w_meth is not None:
return space.call_function(w_meth, w_i, w_j)
else:
return space.getitem(self, space.newslice(w_i, w_j, space.w_None))
def descr_setslice(self, space, w_i, w_j, w_sequence):
- w_meth = self.getattr(space, space.wrap('__setslice__'), False)
+ w_meth = self.getattr(space, '__setslice__', False)
if w_meth is not None:
space.call_function(w_meth, w_i, w_j, w_sequence)
else:
@@ -501,20 +509,20 @@
w_sequence)
def descr_delslice(self, space, w_i, w_j):
- w_meth = self.getattr(space, space.wrap('__delslice__'), False)
+ w_meth = self.getattr(space, '__delslice__', False)
if w_meth is not None:
space.call_function(w_meth, w_i, w_j)
else:
return space.delitem(self, space.newslice(w_i, w_j, space.w_None))
def descr_call(self, space, __args__):
- w_meth = self.getattr(space, space.wrap('__call__'))
+ w_meth = self.getattr(space, '__call__')
return space.call_args(w_meth, __args__)
def descr_nonzero(self, space):
- w_func = self.getattr(space, space.wrap('__nonzero__'), False)
+ w_func = self.getattr(space, '__nonzero__', False)
if w_func is None:
- w_func = self.getattr(space, space.wrap('__len__'), False)
+ w_func = self.getattr(space, '__len__', False)
if w_func is None:
return space.w_True
w_result = space.call_function(w_func)
@@ -538,7 +546,7 @@
not isinstance(w_b, W_InstanceObject)):
return space.cmp(w_a, w_b)
if isinstance(w_a, W_InstanceObject):
- w_func = w_a.getattr(space, space.wrap('__cmp__'), False)
+ w_func = w_a.getattr(space, '__cmp__', False)
if w_func is not None:
w_res = space.call_function(w_func, w_b)
if space.is_w(w_res, space.w_NotImplemented):
@@ -557,7 +565,7 @@
return space.wrap(-1)
return space.wrap(0)
if isinstance(w_b, W_InstanceObject):
- w_func = w_b.getattr(space, space.wrap('__cmp__'), False)
+ w_func = w_b.getattr(space, '__cmp__', False)
if w_func is not None:
w_res = space.call_function(w_func, w_a)
if space.is_w(w_res, space.w_NotImplemented):
@@ -578,10 +586,10 @@
return space.w_NotImplemented
def descr_hash(self, space):
- w_func = self.getattr(space, space.wrap('__hash__'), False)
+ w_func = self.getattr(space, '__hash__', False)
if w_func is None:
- w_eq = self.getattr(space, space.wrap('__eq__'), False)
- w_cmp = self.getattr(space, space.wrap('__cmp__'), False)
+ w_eq = self.getattr(space, '__eq__', False)
+ w_cmp = self.getattr(space, '__cmp__', False)
if w_eq is not None or w_cmp is not None:
raise OperationError(space.w_TypeError,
space.wrap("unhashable instance"))
@@ -617,7 +625,7 @@
return self.descr_int(space)
def descr_index(self, space):
- w_func = self.getattr(space, space.wrap('__index__'), False)
+ w_func = self.getattr(space, '__index__', False)
if w_func is not None:
return space.call_function(w_func)
raise OperationError(
@@ -625,7 +633,7 @@
space.wrap("object cannot be interpreted as an index"))
def descr_contains(self, space, w_obj):
- w_func = self.getattr(space, space.wrap('__contains__'), False)
+ w_func = self.getattr(space, '__contains__', False)
if w_func is not None:
return space.wrap(space.is_true(space.call_function(w_func, w_obj)))
# now do it ourselves
@@ -648,7 +656,7 @@
w_a = self
w_b = w_other
if w_a is self:
- w_func = self.getattr(space, space.wrap('__pow__'), False)
+ w_func = self.getattr(space, '__pow__', False)
if w_func is not None:
return space.call_function(w_func, w_other)
return space.w_NotImplemented
@@ -656,7 +664,7 @@
return space.pow(w_a, w_b, space.w_None)
else:
# CPython also doesn't try coercion in this case
- w_func = self.getattr(space, space.wrap('__pow__'), False)
+ w_func = self.getattr(space, '__pow__', False)
if w_func is not None:
return space.call_function(w_func, w_other, w_modulo)
return space.w_NotImplemented
@@ -668,7 +676,7 @@
w_a = self
w_b = w_other
if w_a is self:
- w_func = self.getattr(space, space.wrap('__rpow__'), False)
+ w_func = self.getattr(space, '__rpow__', False)
if w_func is not None:
return space.call_function(w_func, w_other)
return space.w_NotImplemented
@@ -676,13 +684,13 @@
return space.pow(w_b, w_a, space.w_None)
else:
# CPython also doesn't try coercion in this case
- w_func = self.getattr(space, space.wrap('__rpow__'), False)
+ w_func = self.getattr(space, '__rpow__', False)
if w_func is not None:
return space.call_function(w_func, w_other, w_modulo)
return space.w_NotImplemented
def descr_next(self, space):
- w_func = self.getattr(space, space.wrap('next'), False)
+ w_func = self.getattr(space, 'next', False)
if w_func is None:
raise OperationError(space.w_TypeError,
space.wrap("instance has no next() method"))
@@ -691,10 +699,9 @@
def descr_del(self, space):
# Note that this is called from executioncontext.UserDelAction
# via the space.userdel() method.
- w_name = space.wrap('__del__')
- w_func = space.finditem(self.w_dict, w_name)
+ w_func = self.getdictvalue(space, '__del__')
if w_func is None:
- w_func = self.getattr_from_class(space, w_name)
+ w_func = self.getattr_from_class(space, '__del__')
if w_func is not None:
space.call_function(w_func)
@@ -744,6 +751,14 @@
rmeth,
unwrap_spec=["self", ObjSpace, W_Root])
+
+def descr_del_dict(space, w_inst):
+ # use setdict to raise the error
+ w_inst.setdict(space, space.w_None)
+
+dict_descr = GetSetProperty(descr_get_dict, descr_set_dict, descr_del_dict)
+dict_descr.name = '__dict__'
+
W_InstanceObject.typedef = TypeDef("instance",
__new__ = interp2app(descr_instance_new),
__getattribute__ = interp2app(W_InstanceObject.descr_getattribute,
@@ -797,14 +812,11 @@
unwrap_spec=['self', ObjSpace, W_Root, W_Root]),
next = interp2app(W_InstanceObject.descr_next,
unwrap_spec=['self', ObjSpace]),
- __weakref__ = make_weakref_descr(W_InstanceObject),
__del__ = interp2app(W_InstanceObject.descr_del,
unwrap_spec=['self', ObjSpace]),
__exit__ = interp2app(W_InstanceObject.descr_exit,
unwrap_spec=['self', ObjSpace, W_Root, W_Root, W_Root]),
+ __dict__ = dict_descr,
**rawdict
)
-
-class W_InstanceObjectWithDel(W_InstanceObject):
- def __del__(self):
- self._enqueue_for_destruction(self.space)
+W_InstanceObject.typedef.acceptable_as_base_class = False
Modified: pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py (original)
+++ pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py Fri Oct 22 23:09:43 2010
@@ -955,6 +955,31 @@
assert x is b
assert y == 5
+ def test_cant_subclass_instance(self):
+ class A:
+ pass
+ try:
+ class B(type(A())):
+ pass
+ except TypeError:
+ pass
+ else:
+ assert 0, "should have raised"
+
+ def test_dict_descriptor(self):
+ import sys
+ if not hasattr(sys, 'pypy_objspaceclass'):
+ skip("on CPython old-style instances don't have a __dict__ descriptor")
+ class A:
+ pass
+ a = A()
+ a.x = 1
+ descr = type(a).__dict__['__dict__']
+ assert descr.__get__(a) == {'x': 1}
+ descr.__set__(a, {'x': 2})
+ assert a.x == 2
+ raises(TypeError, descr.__delete__, a)
+
class AppTestOldStyleSharing(AppTestOldstyle):
def setup_class(cls):
@@ -993,3 +1018,22 @@
a = 1
b = 2
assert self.is_strdict(A)
+
+class AppTestOldStyleMapDict(AppTestOldstyle):
+ def setup_class(cls):
+ cls.space = gettestobjspace(**{"objspace.std.withmapdict": True})
+ if option.runappdirect:
+ py.test.skip("can only be run on py.py")
+ def has_mapdict(space, w_inst):
+ return space.wrap(w_inst._get_mapdict_map() is not None)
+ cls.w_has_mapdict = cls.space.wrap(gateway.interp2app(has_mapdict))
+
+
+ def test_has_mapdict(self):
+ class A:
+ def __init__(self):
+ self.x = 42
+ a = A()
+ assert a.x == 42
+ assert self.has_mapdict(a)
+
Modified: pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py Fri Oct 22 23:09:43 2010
@@ -1,5 +1,5 @@
-""" Low-level interface to libffi
+""" Low-level interface to clibffi
"""
from pypy.interpreter.mixedmodule import MixedModule
@@ -46,12 +46,12 @@
if hasattr(interp_rawffi, 'check_HRESULT'):
Module.interpleveldefs['check_HRESULT'] = 'interp_rawffi.check_HRESULT'
- from pypy.rlib import libffi
+ from pypy.rlib import clibffi
for name in ['FUNCFLAG_STDCALL', 'FUNCFLAG_CDECL', 'FUNCFLAG_PYTHONAPI',
'FUNCFLAG_USE_ERRNO', 'FUNCFLAG_USE_LASTERROR',
]:
- if hasattr(libffi, name):
- Module.interpleveldefs[name] = "space.wrap(%r)" % getattr(libffi, name)
+ if hasattr(clibffi, name):
+ Module.interpleveldefs[name] = "space.wrap(%r)" % getattr(clibffi, name)
super(Module, cls).buildloaders()
buildloaders = classmethod(buildloaders)
Modified: pypy/branch/fast-forward/pypy/module/_rawffi/array.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_rawffi/array.py (original)
+++ pypy/branch/fast-forward/pypy/module/_rawffi/array.py Fri Oct 22 23:09:43 2010
@@ -97,7 +97,15 @@
class W_ArrayInstance(W_DataInstance):
def __init__(self, space, shape, length, address=r_uint(0)):
- W_DataInstance.__init__(self, space, shape.size * length, address)
+ # Workaround for a strange behavior of libffi: make sure that
+ # we always have at least 8 bytes. For W_ArrayInstances that are
+ # used as the result value of a function call, ffi_call() writes
+ # 8 bytes into it even if the function's result type asks for less.
+ # This strange behavior is documented.
+ memsize = shape.size * length
+ if memsize < 8:
+ memsize = 8
+ W_DataInstance.__init__(self, space, memsize, address)
self.length = length
self.shape = shape
Modified: pypy/branch/fast-forward/pypy/module/_rawffi/callback.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_rawffi/callback.py (original)
+++ pypy/branch/fast-forward/pypy/module/_rawffi/callback.py Fri Oct 22 23:09:43 2010
@@ -8,8 +8,8 @@
from pypy.module._rawffi.array import get_elem, push_elem
from pypy.module._rawffi.interp_rawffi import W_DataInstance, letter2tp, \
wrap_value, unwrap_value, unwrap_truncate_int
-from pypy.rlib.libffi import USERDATA_P, CallbackFuncPtr, FUNCFLAG_CDECL
-from pypy.rlib.libffi import ffi_type_void
+from pypy.rlib.clibffi import USERDATA_P, CallbackFuncPtr, FUNCFLAG_CDECL
+from pypy.rlib.clibffi import ffi_type_void
from pypy.module._rawffi.tracker import tracker
from pypy.interpreter.error import OperationError
from pypy.interpreter import gateway
Modified: pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py (original)
+++ pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py Fri Oct 22 23:09:43 2010
@@ -5,7 +5,7 @@
from pypy.interpreter.gateway import interp2app, NoneNotWrapped
from pypy.interpreter.typedef import TypeDef, GetSetProperty
-from pypy.rlib.libffi import *
+from pypy.rlib.clibffi import *
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.rlib.unroll import unrolling_iterable
import pypy.rlib.rposix as rposix
Modified: pypy/branch/fast-forward/pypy/module/_rawffi/structure.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_rawffi/structure.py (original)
+++ pypy/branch/fast-forward/pypy/module/_rawffi/structure.py Fri Oct 22 23:09:43 2010
@@ -15,7 +15,7 @@
from pypy.module._rawffi.interp_rawffi import wrap_value, unwrap_value
from pypy.module._rawffi.interp_rawffi import unpack_shape_with_length
from pypy.module._rawffi.interp_rawffi import size_alignment
-from pypy.rlib import libffi
+from pypy.rlib import clibffi
from pypy.rlib.rarithmetic import intmask, r_uint
def unpack_fields(space, w_fields):
@@ -34,7 +34,7 @@
def round_up(size, alignment):
return (size + alignment - 1) & -alignment
-def size_alignment_pos(fields):
+def size_alignment_pos(fields, is_union=False):
size = 0
alignment = 1
pos = []
@@ -42,16 +42,20 @@
# fieldtype is a W_Array
fieldsize = fieldtype.size
fieldalignment = fieldtype.alignment
- size = round_up(size, fieldalignment)
alignment = max(alignment, fieldalignment)
- pos.append(size)
- size += intmask(fieldsize)
+ if is_union:
+ pos.append(0)
+ size = max(size, fieldsize)
+ else:
+ size = round_up(size, fieldalignment)
+ pos.append(size)
+ size += intmask(fieldsize)
size = round_up(size, alignment)
return size, alignment, pos
class W_Structure(W_DataShape):
- def __init__(self, space, fields, size, alignment):
+ def __init__(self, space, fields, size, alignment, is_union=False):
name_to_index = {}
if fields is not None:
for i in range(len(fields)):
@@ -60,7 +64,7 @@
raise operationerrfmt(space.w_ValueError,
"duplicate field name %s", name)
name_to_index[name] = i
- size, alignment, pos = size_alignment_pos(fields)
+ size, alignment, pos = size_alignment_pos(fields, is_union)
else: # opaque case
fields = []
pos = []
@@ -104,14 +108,14 @@
descr_fieldoffset.unwrap_spec = ['self', ObjSpace, str]
# get the corresponding ffi_type
- ffi_struct = lltype.nullptr(libffi.FFI_STRUCT_P.TO)
+ ffi_struct = lltype.nullptr(clibffi.FFI_STRUCT_P.TO)
def get_basic_ffi_type(self):
if not self.ffi_struct:
# Repeated fields are delicate. Consider for example
# struct { int a[5]; }
# or struct { struct {int x;} a[5]; }
- # Seeing no corresponding doc in libffi, let's just repeat
+ # Seeing no corresponding doc in clibffi, let's just repeat
# the field 5 times...
fieldtypes = []
for name, tp in self.fields:
@@ -122,7 +126,7 @@
while count + basic_size <= total_size:
fieldtypes.append(basic_ffi_type)
count += basic_size
- self.ffi_struct = libffi.make_struct_ffitype_e(self.size,
+ self.ffi_struct = clibffi.make_struct_ffitype_e(self.size,
self.alignment,
fieldtypes)
return self.ffi_struct.ffistruct
@@ -133,15 +137,17 @@
-def descr_new_structure(space, w_type, w_shapeinfo):
+def descr_new_structure(space, w_type, w_shapeinfo, union=0):
+ is_union = bool(union)
if space.is_true(space.isinstance(w_shapeinfo, space.w_tuple)):
w_size, w_alignment = space.fixedview(w_shapeinfo, expected_length=2)
S = W_Structure(space, None, space.int_w(w_size),
- space.int_w(w_alignment))
+ space.int_w(w_alignment), is_union)
else:
fields = unpack_fields(space, w_shapeinfo)
- S = W_Structure(space, fields, 0, 0)
+ S = W_Structure(space, fields, 0, 0, is_union)
return space.wrap(S)
+descr_new_structure.unwrap_spec = [ObjSpace, W_Root, W_Root, int]
W_Structure.typedef = TypeDef(
'Structure',
Modified: pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py (original)
+++ pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py Fri Oct 22 23:09:43 2010
@@ -71,7 +71,7 @@
return s[num];
}
- char *char_check(char x, char y)
+ const char *char_check(char x, char y)
{
if (y == static_str[0])
return static_str;
@@ -186,13 +186,14 @@
sum_x_y
give perturb get_s2a check_s2a
AAA_first_ordinal_function
+ ret_un_func
""".split()
eci = ExternalCompilationInfo(export_symbols=symbols)
return str(platform.compile([c_file], eci, 'x', standalone=False))
prepare_c_example = staticmethod(prepare_c_example)
def setup_class(cls):
- from pypy.rlib.libffi import get_libc_name
+ from pypy.rlib.clibffi import get_libc_name
space = gettestobjspace(usemodules=('_rawffi', 'struct'))
cls.space = space
cls.w_lib_name = space.wrap(cls.prepare_c_example())
@@ -948,14 +949,15 @@
assert a[4] == 't'
def test_union(self):
- skip("segfaulting")
import _rawffi
longsize = _rawffi.sizeof('l')
- S = _rawffi.Structure((longsize, longsize))
+ S = _rawffi.Structure([('x', 'h'), ('y', 'l')], union=True)
s = S(autofree=False)
+ s.x = 12345
lib = _rawffi.CDLL(self.lib_name)
f = lib.ptr('ret_un_func', [(S, 1)], (S, 1))
ret = f(s)
+ assert ret.y == 1234500, "ret.y == %d" % (ret.y,)
s.free()
class AppTestAutoFree:
Modified: pypy/branch/fast-forward/pypy/module/_socket/test/test_sock_app.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_socket/test/test_sock_app.py (original)
+++ pypy/branch/fast-forward/pypy/module/_socket/test/test_sock_app.py Fri Oct 22 23:09:43 2010
@@ -254,6 +254,7 @@
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
fd = s.fileno()
w_obj = rsocket.make_address(c_addr, addrlen).as_object(fd, space)
+ lltype.free(c_addr_ll, flavor='raw')
assert space.is_true(space.eq(w_obj, space.newtuple([
space.wrap('lo'),
space.wrap(socket.ntohs(8)),
Modified: pypy/branch/fast-forward/pypy/module/_sre/interp_sre.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_sre/interp_sre.py (original)
+++ pypy/branch/fast-forward/pypy/module/_sre/interp_sre.py Fri Oct 22 23:09:43 2010
@@ -290,10 +290,8 @@
def SRE_Pattern__new__(space, w_subtype, w_pattern, flags, w_code,
groups=0, w_groupindex=None, w_indexgroup=None):
n = space.int_w(space.len(w_code))
- code = [0] * n
- for i in range(n):
- x = space.uint_w(space.getitem(w_code, space.wrap(i)))
- code[i] = intmask(x)
+ code = [intmask(space.uint_w(space.getitem(w_code, space.wrap(i))))
+ for i in range(n)]
#
w_srepat = space.allocate_instance(W_SRE_Pattern, w_subtype)
srepat = space.interp_w(W_SRE_Pattern, w_srepat)
Modified: pypy/branch/fast-forward/pypy/module/_weakref/interp__weakref.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_weakref/interp__weakref.py (original)
+++ pypy/branch/fast-forward/pypy/module/_weakref/interp__weakref.py Fri Oct 22 23:09:43 2010
@@ -7,7 +7,7 @@
import weakref
-class WeakrefLifeline(object):
+class WeakrefLifeline(W_Root):
def __init__(self, space):
self.space = space # this is here for W_Root.clear_all_weakrefs()
self.refs_weak = []
Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py (original)
+++ pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py Fri Oct 22 23:09:43 2010
@@ -213,13 +213,10 @@
subkey = None
else:
subkey = space.str_w(w_subkey)
- dataptr = rffi.str2charp(value)
- try:
+ with rffi.scoped_str2charp(value) as dataptr:
ret = rwinreg.RegSetValue(hkey, subkey, rwinreg.REG_SZ, dataptr, len(value))
- finally:
- rffi.free_charp(dataptr)
- if ret != 0:
- raiseWindowsError(space, ret, 'RegSetValue')
+ if ret != 0:
+ raiseWindowsError(space, ret, 'RegSetValue')
SetValue.unwrap_spec = [ObjSpace, W_Root, W_Root, int, str]
def QueryValue(space, w_hkey, w_subkey):
@@ -238,23 +235,15 @@
subkey = None
else:
subkey = space.str_w(w_subkey)
- bufsize_p = lltype.malloc(rwin32.PLONG.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.PLONG.TO, 1) as bufsize_p:
ret = rwinreg.RegQueryValue(hkey, subkey, None, bufsize_p)
if ret != 0:
raiseWindowsError(space, ret, 'RegQueryValue')
- buf = lltype.malloc(rffi.CCHARP.TO, bufsize_p[0], flavor='raw')
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, bufsize_p[0]) as buf:
ret = rwinreg.RegQueryValue(hkey, subkey, buf, bufsize_p)
if ret != 0:
raiseWindowsError(space, ret, 'RegQueryValue')
return space.wrap(rffi.charp2strn(buf, bufsize_p[0] - 1))
- finally:
- lltype.free(buf, flavor='raw')
- finally:
- lltype.free(bufsize_p, flavor='raw')
- if ret != 0:
- raiseWindowsError(space, ret, 'RegQueryValue')
QueryValue.unwrap_spec = [ObjSpace, W_Root, W_Root]
def convert_to_regdata(space, w_value, typ):
@@ -413,16 +402,14 @@
value_name is a string indicating the value to query"""
hkey = hkey_w(w_hkey, space)
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
- retDataSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retDataSize:
ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword, null_dword,
None, retDataSize)
if ret != 0:
raiseWindowsError(space, ret, 'RegQueryValueEx')
- databuf = lltype.malloc(rffi.CCHARP.TO, retDataSize[0], flavor='raw')
- try:
- retType = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+
+ with lltype.scoped_alloc(rffi.CCHARP.TO, retDataSize[0]) as databuf:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType:
ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword,
retType, databuf, retDataSize)
@@ -433,12 +420,6 @@
retDataSize[0], retType[0]),
space.wrap(retType[0]),
])
- finally:
- lltype.free(retType, flavor='raw')
- finally:
- lltype.free(databuf, flavor='raw')
- finally:
- lltype.free(retDataSize, flavor='raw')
QueryValueEx.unwrap_spec = [ObjSpace, W_Root, str]
@@ -455,14 +436,11 @@
The return value is the handle of the opened key.
If the function fails, an exception is raised."""
hkey = hkey_w(w_hkey, space)
- rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey:
ret = rwinreg.RegCreateKey(hkey, subkey, rethkey)
if ret != 0:
raiseWindowsError(space, ret, 'CreateKey')
return space.wrap(W_HKEY(rethkey[0]))
- finally:
- lltype.free(rethkey, flavor='raw')
CreateKey.unwrap_spec = [ObjSpace, W_Root, str]
def DeleteKey(space, w_hkey, subkey):
@@ -505,14 +483,11 @@
The result is a new handle to the specified key
If the function fails, an EnvironmentError exception is raised."""
hkey = hkey_w(w_hkey, space)
- rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey:
ret = rwinreg.RegOpenKeyEx(hkey, subkey, res, sam, rethkey)
if ret != 0:
raiseWindowsError(space, ret, 'RegOpenKeyEx')
return space.wrap(W_HKEY(rethkey[0]))
- finally:
- lltype.free(rethkey, flavor='raw')
OpenKey.unwrap_spec = [ObjSpace, W_Root, str, int, rffi.r_uint]
def EnumValue(space, w_hkey, index):
@@ -532,10 +507,8 @@
hkey = hkey_w(w_hkey, space)
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
- retValueSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
- retDataSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retValueSize:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retDataSize:
ret = rwinreg.RegQueryInfoKey(
hkey, None, null_dword, null_dword,
null_dword, null_dword, null_dword,
@@ -547,14 +520,9 @@
retValueSize[0] += 1
retDataSize[0] += 1
- valuebuf = lltype.malloc(rffi.CCHARP.TO, retValueSize[0],
- flavor='raw')
- try:
- databuf = lltype.malloc(rffi.CCHARP.TO, retDataSize[0],
- flavor='raw')
- try:
- retType = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, retValueSize[0]) as valuebuf:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, retDataSize[0]) as databuf:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType:
ret = rwinreg.RegEnumValue(
hkey, index, valuebuf, retValueSize,
null_dword, retType, databuf, retDataSize)
@@ -567,16 +535,6 @@
retDataSize[0], retType[0]),
space.wrap(retType[0]),
])
- finally:
- lltype.free(retType, flavor='raw')
- finally:
- lltype.free(databuf, flavor='raw')
- finally:
- lltype.free(valuebuf, flavor='raw')
- finally:
- lltype.free(retDataSize, flavor='raw')
- finally:
- lltype.free(retValueSize, flavor='raw')
EnumValue.unwrap_spec = [ObjSpace, W_Root, int]
@@ -593,10 +551,8 @@
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
# max key name length is 255
- buf = lltype.malloc(rffi.CCHARP.TO, 256, flavor='raw')
- try:
- retValueSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, 256) as buf:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retValueSize:
retValueSize[0] = r_uint(256) # includes NULL terminator
ret = rwinreg.RegEnumKeyEx(hkey, index, buf, retValueSize,
null_dword, None, null_dword,
@@ -604,10 +560,6 @@
if ret != 0:
raiseWindowsError(space, ret, 'RegEnumKeyEx')
return space.wrap(rffi.charp2str(buf))
- finally:
- lltype.free(retValueSize, flavor='raw')
- finally:
- lltype.free(buf, flavor='raw')
EnumKey.unwrap_spec = [ObjSpace, W_Root, int]
@@ -622,12 +574,9 @@
A long integer that identifies when the key was last modified (if available)
as 100's of nanoseconds since Jan 1, 1600."""
hkey = hkey_w(w_hkey, space)
- nSubKeys = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
- nValues = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
- ft = lltype.malloc(rwin32.PFILETIME.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as nSubKeys:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as nValues:
+ with lltype.scoped_alloc(rwin32.PFILETIME.TO, 1) as ft:
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
ret = rwinreg.RegQueryInfoKey(
hkey, None, null_dword, null_dword,
@@ -641,12 +590,6 @@
return space.newtuple([space.wrap(nSubKeys[0]),
space.wrap(nValues[0]),
space.wrap(l)])
- finally:
- lltype.free(ft, flavor='raw')
- finally:
- lltype.free(nValues, flavor='raw')
- finally:
- lltype.free(nSubKeys, flavor='raw')
QueryInfoKey.unwrap_spec = [ObjSpace, W_Root]
def str_or_None_w(space, w_obj):
@@ -667,12 +610,9 @@
If the function fails, an EnvironmentError exception is raised."""
machine = str_or_None_w(space, w_machine)
hkey = hkey_w(w_hkey, space)
- rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey:
ret = rwinreg.RegConnectRegistry(machine, hkey, rethkey)
if ret != 0:
raiseWindowsError(space, ret, 'RegConnectRegistry')
return space.wrap(W_HKEY(rethkey[0]))
- finally:
- lltype.free(rethkey, flavor='raw')
ConnectRegistry.unwrap_spec = [ObjSpace, W_Root, W_Root]
Modified: pypy/branch/fast-forward/pypy/module/array/interp_array.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/array/interp_array.py (original)
+++ pypy/branch/fast-forward/pypy/module/array/interp_array.py Fri Oct 22 23:09:43 2010
@@ -193,32 +193,30 @@
mytype.bytes
raise OperationError(space.w_OverflowError,
space.wrap(msg))
- elif mytype.unwrap == 'str_w' or mytype.unwrap == 'unicode_w':
+ return rffi.cast(mytype.itemtype, item)
+ if mytype.unwrap == 'str_w' or mytype.unwrap == 'unicode_w':
if len(item) != 1:
msg = 'array item must be char'
raise OperationError(space.w_TypeError, space.wrap(msg))
item = item[0]
-
+ return rffi.cast(mytype.itemtype, item)
+ #
+ # "regular" case: it fits in an rpython integer (lltype.Signed)
+ result = rffi.cast(mytype.itemtype, item)
if mytype.canoverflow:
- msg = None
- if mytype.signed:
- if item < -1 << (mytype.bytes * 8 - 1):
+ if rffi.cast(lltype.Signed, result) != item:
+ # overflow. build the correct message
+ if item < 0:
msg = ('signed %d-byte integer is less than minimum' %
mytype.bytes)
- elif item > (1 << (mytype.bytes * 8 - 1)) - 1:
+ else:
msg = ('signed %d-byte integer is greater than maximum'
% mytype.bytes)
- else:
- if item < 0:
- msg = ('unsigned %d-byte integer is less than minimum'
- % mytype.bytes)
- elif item > (1 << (mytype.bytes * 8)) - 1:
- msg = ('unsigned %d-byte integer is greater'
- ' than maximum' % mytype.bytes)
- if msg is not None:
+ if not mytype.signed:
+ msg = 'un' + msg # 'signed' => 'unsigned'
raise OperationError(space.w_OverflowError,
space.wrap(msg))
- return rffi.cast(mytype.itemtype, item)
+ return result
def __del__(self):
self.setlen(0)
Modified: pypy/branch/fast-forward/pypy/module/bz2/interp_bz2.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/bz2/interp_bz2.py (original)
+++ pypy/branch/fast-forward/pypy/module/bz2/interp_bz2.py Fri Oct 22 23:09:43 2010
@@ -225,6 +225,11 @@
if self.current_size > 0:
rffi.keep_buffer_alive_until_here(self.raw_buf, self.gc_buf)
+ def __enter__(self):
+ return self
+ def __exit__(self, *args):
+ self.free()
+
# ____________________________________________________________
#
# Make the BZ2File type by internally inheriting from W_File.
@@ -531,33 +536,30 @@
if not self.running:
raise OperationError(self.space.w_ValueError,
self.space.wrap("this object was already flushed"))
-
- out = OutBuffer(self.bzs)
+
in_bufsize = datasize
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(datasize):
- in_buf[i] = data[i]
- try:
-
- self.bzs.c_next_in = in_buf
- rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize)
+ with OutBuffer(self.bzs) as out:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
- while True:
- bzerror = BZ2_bzCompress(self.bzs, BZ_RUN)
- if bzerror != BZ_RUN_OK:
- _catch_bz2_error(self.space, bzerror)
+ for i in range(datasize):
+ in_buf[i] = data[i]
- if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
- break
- elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
+ self.bzs.c_next_in = in_buf
+ rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize)
- res = out.make_result_string()
- return self.space.wrap(res)
- finally:
- lltype.free(in_buf, flavor='raw')
- out.free()
+ while True:
+ bzerror = BZ2_bzCompress(self.bzs, BZ_RUN)
+ if bzerror != BZ_RUN_OK:
+ _catch_bz2_error(self.space, bzerror)
+
+ if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
+ break
+ elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
+
+ res = out.make_result_string()
+ return self.space.wrap(res)
compress.unwrap_spec = ['self', 'bufferstr']
@@ -566,9 +568,8 @@
raise OperationError(self.space.w_ValueError,
self.space.wrap("this object was already flushed"))
self.running = False
-
- out = OutBuffer(self.bzs)
- try:
+
+ with OutBuffer(self.bzs) as out:
while True:
bzerror = BZ2_bzCompress(self.bzs, BZ_FINISH)
if bzerror == BZ_STREAM_END:
@@ -581,8 +582,6 @@
res = out.make_result_string()
return self.space.wrap(res)
- finally:
- out.free()
flush.unwrap_spec = ['self']
W_BZ2Compressor.typedef = TypeDef("BZ2Compressor",
@@ -641,38 +640,37 @@
if not self.running:
raise OperationError(self.space.w_EOFError,
self.space.wrap("end of stream was already found"))
-
+
in_bufsize = len(data)
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(in_bufsize):
- in_buf[i] = data[i]
- out = OutBuffer(self.bzs)
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
+ for i in range(in_bufsize):
+ in_buf[i] = data[i]
self.bzs.c_next_in = in_buf
rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize)
- while True:
- bzerror = BZ2_bzDecompress(self.bzs)
- if bzerror == BZ_STREAM_END:
- if rffi.getintfield(self.bzs, 'c_avail_in') != 0:
- unused = [self.bzs.c_next_in[i] for i in range(rffi.getintfield(self.bzs, 'c_avail_in'))]
- self.unused_data = "".join(unused)
- self.running = False
- break
- if bzerror != BZ_OK:
- _catch_bz2_error(self.space, bzerror)
-
- if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
- break
- elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
+ with OutBuffer(self.bzs) as out:
+ while True:
+ bzerror = BZ2_bzDecompress(self.bzs)
+ if bzerror == BZ_STREAM_END:
+ if rffi.getintfield(self.bzs, 'c_avail_in') != 0:
+ unused = [self.bzs.c_next_in[i]
+ for i in range(
+ rffi.getintfield(self.bzs,
+ 'c_avail_in'))]
+ self.unused_data = "".join(unused)
+ self.running = False
+ break
+ if bzerror != BZ_OK:
+ _catch_bz2_error(self.space, bzerror)
+
+ if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
+ break
+ elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
- res = out.make_result_string()
- return self.space.wrap(res)
- finally:
- lltype.free(in_buf, flavor='raw')
- out.free()
+ res = out.make_result_string()
+ return self.space.wrap(res)
decompress.unwrap_spec = ['self', 'bufferstr']
@@ -695,43 +693,39 @@
if compresslevel < 1 or compresslevel > 9:
raise OperationError(space.w_ValueError,
space.wrap("compresslevel must be between 1 and 9"))
-
- bzs = lltype.malloc(bz_stream.TO, flavor='raw', zero=True)
- in_bufsize = len(data)
- # conforming to bz2 manual, this is large enough to fit compressed
- # data in one shot. We will check it later anyway.
- out = OutBuffer(bzs, in_bufsize + (in_bufsize / 100 + 1) + 600)
-
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(in_bufsize):
- in_buf[i] = data[i]
-
- try:
- bzs.c_next_in = in_buf
- rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
- bzerror = BZ2_bzCompressInit(bzs, compresslevel, 0, 0)
- if bzerror != BZ_OK:
- _catch_bz2_error(space, bzerror)
+ with lltype.scoped_alloc(bz_stream.TO, zero=True) as bzs:
+ in_bufsize = len(data)
+
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
+ for i in range(in_bufsize):
+ in_buf[i] = data[i]
+ bzs.c_next_in = in_buf
+ rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
+
+ # conforming to bz2 manual, this is large enough to fit compressed
+ # data in one shot. We will check it later anyway.
+ with OutBuffer(bzs,
+ in_bufsize + (in_bufsize / 100 + 1) + 600) as out:
+
+ bzerror = BZ2_bzCompressInit(bzs, compresslevel, 0, 0)
+ if bzerror != BZ_OK:
+ _catch_bz2_error(space, bzerror)
- while True:
- bzerror = BZ2_bzCompress(bzs, BZ_FINISH)
- if bzerror == BZ_STREAM_END:
- break
- elif bzerror != BZ_FINISH_OK:
+ while True:
+ bzerror = BZ2_bzCompress(bzs, BZ_FINISH)
+ if bzerror == BZ_STREAM_END:
+ break
+ elif bzerror != BZ_FINISH_OK:
+ BZ2_bzCompressEnd(bzs)
+ _catch_bz2_error(space, bzerror)
+
+ if rffi.getintfield(bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
+
+ res = out.make_result_string()
BZ2_bzCompressEnd(bzs)
- _catch_bz2_error(space, bzerror)
-
- if rffi.getintfield(bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
-
- res = out.make_result_string()
- BZ2_bzCompressEnd(bzs)
- return space.wrap(res)
- finally:
- lltype.free(bzs, flavor='raw')
- lltype.free(in_buf, flavor='raw')
- out.free()
+ return space.wrap(res)
compress.unwrap_spec = [ObjSpace, 'bufferstr', int]
def decompress(space, data):
@@ -744,40 +738,34 @@
if in_bufsize == 0:
return space.wrap("")
- bzs = lltype.malloc(bz_stream.TO, flavor='raw', zero=True)
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(in_bufsize):
- in_buf[i] = data[i]
-
- out = OutBuffer(bzs)
- try:
- bzs.c_next_in = in_buf
- rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
-
- bzerror = BZ2_bzDecompressInit(bzs, 0, 0)
- if bzerror != BZ_OK:
- _catch_bz2_error(space, bzerror)
-
- while True:
- bzerror = BZ2_bzDecompress(bzs)
- if bzerror == BZ_STREAM_END:
- break
- if bzerror != BZ_OK:
- BZ2_bzDecompressEnd(bzs)
- _catch_bz2_error(space, bzerror)
-
- if rffi.getintfield(bzs, 'c_avail_in') == 0:
+ with lltype.scoped_alloc(bz_stream.TO, zero=True) as bzs:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
+ for i in range(in_bufsize):
+ in_buf[i] = data[i]
+ bzs.c_next_in = in_buf
+ rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
+
+ with OutBuffer(bzs) as out:
+ bzerror = BZ2_bzDecompressInit(bzs, 0, 0)
+ if bzerror != BZ_OK:
+ _catch_bz2_error(space, bzerror)
+
+ while True:
+ bzerror = BZ2_bzDecompress(bzs)
+ if bzerror == BZ_STREAM_END:
+ break
+ if bzerror != BZ_OK:
+ BZ2_bzDecompressEnd(bzs)
+ _catch_bz2_error(space, bzerror)
+
+ if rffi.getintfield(bzs, 'c_avail_in') == 0:
+ BZ2_bzDecompressEnd(bzs)
+ raise OperationError(space.w_ValueError, space.wrap(
+ "couldn't find end of stream"))
+ elif rffi.getintfield(bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
+
+ res = out.make_result_string()
BZ2_bzDecompressEnd(bzs)
- raise OperationError(space.w_ValueError,
- space.wrap("couldn't find end of stream"))
- elif rffi.getintfield(bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
-
- res = out.make_result_string()
- BZ2_bzDecompressEnd(bzs)
- return space.wrap(res)
- finally:
- lltype.free(bzs, flavor='raw')
- lltype.free(in_buf, flavor='raw')
- out.free()
+ return space.wrap(res)
decompress.unwrap_spec = [ObjSpace, 'bufferstr']
Modified: pypy/branch/fast-forward/pypy/module/cpyext/classobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/cpyext/classobject.py (original)
+++ pypy/branch/fast-forward/pypy/module/cpyext/classobject.py Fri Oct 22 23:09:43 2010
@@ -15,16 +15,20 @@
class is the class of new object. The dict parameter will be used as the
object's __dict__; if NULL, a new dictionary will be created for the
instance."""
- if not PyClass_Check(space, w_class):
+ if not isinstance(w_class, W_ClassObject):
return PyErr_BadInternalCall(space)
- return W_InstanceObject(space, w_class, w_dict)
+ w_result = w_class.instantiate(space)
+ if w_dict is not None:
+ w_result.setdict(space, w_dict)
+ return w_result
@cpython_api([PyObject, PyObject], PyObject, error=CANNOT_FAIL)
def _PyInstance_Lookup(space, w_instance, w_name):
+ name = space.str_w(w_name)
assert isinstance(w_instance, W_InstanceObject)
- w_result = space.finditem(w_instance.w_dict, w_name)
+ w_result = w_instance.getdictvalue(space, name)
if w_result is not None:
return w_result
- return w_instance.w_class.lookup(space, w_name)
+ return w_instance.w_class.lookup(space, name)
Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py (original)
+++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py Fri Oct 22 23:09:43 2010
@@ -16,6 +16,7 @@
from pypy.module.cpyext.pyobject import Py_DecRef, InvalidPointerException
from pypy.translator.goal import autopath
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
@api.cpython_api([], api.PyObject)
def PyPy_Crash1(space):
@@ -78,7 +79,6 @@
self.frozen_refcounts[w_obj] = obj.c_ob_refcnt
#state.print_refcounts()
self.frozen_ll2callocations = set(ll2ctypes.ALLOCATED.values())
- lltype.start_tracking_allocations()
class LeakCheckingTest(object):
def check_and_print_leaks(self):
@@ -126,17 +126,8 @@
for w_obj in lost_objects_w:
print >>sys.stderr, "Lost object %r" % (w_obj, )
leaking = True
- for llvalue in set(ll2ctypes.ALLOCATED.values()) - self.frozen_ll2callocations:
- if getattr(llvalue, "_traceback", None): # this means that the allocation should be tracked
- leaking = True
- print >>sys.stderr, "Did not deallocate %r (ll2ctypes)" % (llvalue, )
- print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines())
- for llvalue in lltype.ALLOCATED.keys():
- leaking = True
- print >>sys.stderr, "Did not deallocate %r (llvalue)" % (llvalue, )
- print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines())
-
- lltype.stop_tracking_allocations()
+ # the actual low-level leak checking is done by pypy.tool.leakfinder,
+ # enabled automatically by pypy.conftest.
return leaking
class AppTestCpythonExtensionBase(LeakCheckingTest):
Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py (original)
+++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py Fri Oct 22 23:09:43 2010
@@ -177,13 +177,14 @@
encoded_charp = rffi.str2charp(encoded)
strict_charp = rffi.str2charp("strict")
if endian is not None:
- pendian = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
if endian < 0:
- pendian[0] = -1
+ value = -1
elif endian > 0:
- pendian[0] = 1
+ value = 1
else:
- pendian[0] = 0
+ value = 0
+ pendian = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
+ pendian[0] = rffi.cast(rffi.INT, value)
else:
pendian = None
Modified: pypy/branch/fast-forward/pypy/module/gc/referents.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/gc/referents.py (original)
+++ pypy/branch/fast-forward/pypy/module/gc/referents.py Fri Oct 22 23:09:43 2010
@@ -15,8 +15,16 @@
def try_cast_gcref_to_w_root(gcref):
w_obj = rgc.try_cast_gcref_to_instance(W_Root, gcref)
- if not we_are_translated() and not hasattr(w_obj, 'typedef'):
- w_obj = None
+ # Ignore the instances of W_Root that are not really valid as Python
+ # objects. There is e.g. WeakrefLifeline in module/_weakref that
+ # inherits from W_Root for internal reasons. Such instances don't
+ # have a typedef at all (or have a null typedef after translation).
+ if not we_are_translated():
+ if not hasattr(w_obj, 'typedef'):
+ return None
+ else:
+ if w_obj is None or not w_obj.typedef:
+ return None
return w_obj
def wrap(space, gcref):
Modified: pypy/branch/fast-forward/pypy/module/imp/test/test_import.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/imp/test/test_import.py (original)
+++ pypy/branch/fast-forward/pypy/module/imp/test/test_import.py Fri Oct 22 23:09:43 2010
@@ -792,6 +792,7 @@
extrapath = udir.ensure("pythonpath", dir=1)
extrapath.join("urllib.py").write("print 42\n")
old = os.environ.get('PYTHONPATH', None)
+ oldlang = os.environ.pop('LANG', None)
try:
os.environ['PYTHONPATH'] = str(extrapath)
output = py.process.cmdexec('''"%s" "%s" -c "import urllib"''' %
@@ -800,6 +801,8 @@
finally:
if old:
os.environ['PYTHONPATH'] = old
+ if oldlang:
+ os.environ['LANG'] = oldlang
class AppTestImportHooks(object):
def test_meta_path(self):
Modified: pypy/branch/fast-forward/pypy/module/pypyjit/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/pypyjit/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/pypyjit/__init__.py Fri Oct 22 23:09:43 2010
@@ -6,6 +6,7 @@
interpleveldefs = {
'set_param': 'interp_jit.set_param',
+ 'residual_call': 'interp_jit.residual_call',
}
def setup_after_space_initialization(self):
Modified: pypy/branch/fast-forward/pypy/module/pypyjit/interp_jit.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/pypyjit/interp_jit.py (original)
+++ pypy/branch/fast-forward/pypy/module/pypyjit/interp_jit.py Fri Oct 22 23:09:43 2010
@@ -5,10 +5,10 @@
from pypy.tool.pairtype import extendabletype
from pypy.rlib.rarithmetic import r_uint, intmask
-from pypy.rlib.jit import JitDriver, hint, we_are_jitted
+from pypy.rlib.jit import JitDriver, hint, we_are_jitted, dont_look_inside
import pypy.interpreter.pyopcode # for side-effects
from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.gateway import ObjSpace, Arguments
+from pypy.interpreter.gateway import ObjSpace, Arguments, W_Root
from pypy.interpreter.pycode import PyCode, CO_GENERATOR
from pypy.interpreter.pyframe import PyFrame
from pypy.interpreter.pyopcode import ExitFrame
@@ -131,3 +131,10 @@
"no JIT parameter '%s'", key)
set_param.unwrap_spec = [ObjSpace, Arguments]
+
+ at dont_look_inside
+def residual_call(space, w_callable, args):
+ '''For testing. Invokes callable(...), but without letting
+ the JIT follow the call.'''
+ return space.call_args(w_callable, args)
+residual_call.unwrap_spec = [ObjSpace, W_Root, Arguments]
Modified: pypy/branch/fast-forward/pypy/module/pypyjit/policy.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/pypyjit/policy.py (original)
+++ pypy/branch/fast-forward/pypy/module/pypyjit/policy.py Fri Oct 22 23:09:43 2010
@@ -11,7 +11,7 @@
if '.' in modname:
modname, _ = modname.split('.', 1)
if modname in ['pypyjit', 'signal', 'micronumpy', 'math', 'exceptions',
- 'imp', 'sys', 'array']:
+ 'imp', 'sys', 'array', '_ffi']:
return True
return False
Modified: pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py Fri Oct 22 23:09:43 2010
@@ -79,8 +79,11 @@
class PyPyCJITTests(object):
- def run_source(self, source, expected_max_ops, *testcases):
+ def run_source(self, source, expected_max_ops, *testcases, **kwds):
assert isinstance(expected_max_ops, int)
+ threshold = kwds.pop('threshold', 3)
+ if kwds:
+ raise TypeError, 'Unsupported keyword arguments: %s' % kwds.keys()
source = py.code.Source(source)
filepath = self.tmpdir.join('case%d.py' % self.counter)
logfilepath = filepath.new(ext='.log')
@@ -92,7 +95,7 @@
import sys
try: # make the file runnable by CPython
import pypyjit
- pypyjit.set_param(threshold=3)
+ pypyjit.set_param(threshold=%d)
except ImportError:
pass
@@ -102,7 +105,7 @@
print >> sys.stderr, 'got:', repr(result)
assert result == expected
assert type(result) is type(expected)
- """)
+ """ % threshold)
for testcase in testcases * 2:
print >> f, "check(%r, %r)" % testcase
print >> f, "print 'OK :-)'"
@@ -116,6 +119,8 @@
result = child_stdout.read()
child_stdout.close()
assert result
+ if result.strip().startswith('SKIP:'):
+ py.test.skip(result.strip())
assert result.splitlines()[-1].strip() == 'OK :-)'
self.parse_loops(logfilepath)
self.print_loops()
@@ -123,9 +128,10 @@
if self.total_ops > expected_max_ops:
assert 0, "too many operations: got %d, expected maximum %d" % (
self.total_ops, expected_max_ops)
+ return result
def parse_loops(self, opslogfile):
- from pypy.jit.metainterp.test.oparser import parse
+ from pypy.jit.tool.oparser import parse
from pypy.tool import logparser
assert opslogfile.check()
log = logparser.parse_log_file(str(opslogfile))
@@ -272,7 +278,7 @@
assert len(ops) == 2
assert not ops[0].get_opnames("call")
assert not ops[0].get_opnames("new")
- assert len(ops[0].get_opnames("guard")) <= 7
+ assert len(ops[0].get_opnames("guard")) <= 3 # we get 2 withmapdict
assert not ops[1] # second LOOKUP_METHOD folded away
ops = self.get_by_bytecode("CALL_METHOD")
@@ -283,7 +289,7 @@
else:
assert not bytecode.get_opnames("call")
assert not bytecode.get_opnames("new")
- assert len(bytecode.get_opnames("guard")) <= 9
+ assert len(bytecode.get_opnames("guard")) <= 6
assert len(ops[1]) < len(ops[0])
ops = self.get_by_bytecode("LOAD_ATTR")
@@ -317,8 +323,8 @@
assert len(ops) == 2
assert not ops[0].get_opnames("call")
assert not ops[0].get_opnames("new")
- assert len(ops[0].get_opnames("guard")) <= 7
- assert len(ops[0].get_opnames("getfield")) < 6
+ assert len(ops[0].get_opnames("guard")) <= 3 # we get 2 withmapdict
+ assert len(ops[0].get_opnames("getfield")) <= 5 # we get <5 withmapdict
assert not ops[1] # second LOOKUP_METHOD folded away
def test_default_and_kw(self):
@@ -382,7 +388,7 @@
a.x = 2
i = i + a.x
return i
- ''', 67,
+ ''', 69,
([20], 20),
([31], 32))
@@ -390,7 +396,7 @@
self.get_by_bytecode("CALL_FUNCTION"))
assert not callA.get_opnames("call")
assert not callA.get_opnames("new")
- assert len(callA.get_opnames("guard")) <= 8
+ assert len(callA.get_opnames("guard")) <= 2
assert not callisinstance1.get_opnames("call")
assert not callisinstance1.get_opnames("new")
assert len(callisinstance1.get_opnames("guard")) <= 2
@@ -554,17 +560,13 @@
def test_blockstack_virtualizable(self):
self.run_source('''
- def g(k):
- s = 0
- for i in range(k, k+2):
- s += 1
- return s
+ from pypyjit import residual_call
def main():
i = 0
while i < 100:
try:
- g(i)
+ residual_call(len, [])
except:
pass
i += 1
@@ -606,16 +608,17 @@
# call that can raise is not exchanged into getarrayitem_gc
def test_overflow_checking(self):
+ startvalue = sys.maxint - 2147483647
self.run_source('''
def main():
def f(a,b):
if a < 0: return -1
return a-b
- total = 0
+ total = %d
for i in range(100000):
total += f(i, 5)
return total
- ''', 170, ([], 4999450000L))
+ ''' % startvalue, 170, ([], startvalue + 4999450000L))
def test_boolrewrite_invers(self):
for a, b, res, ops in (('2000', '2000', 20001000, 51),
@@ -742,6 +745,8 @@
'''%(op1, float(a)/4.0, float(b)/4.0, op2), 109, ([], res))
def test_boolrewrite_ptr(self):
+ # XXX this test is way too imprecise in what it is actually testing
+ # it should count the number of guards instead
compares = ('a == b', 'b == a', 'a != b', 'b != a', 'a == c', 'c != b')
for e1 in compares:
for e2 in compares:
@@ -765,7 +770,7 @@
print
print 'Test:', e1, e2, n, res
self.run_source('''
- class tst:
+ class tst(object):
pass
def main():
a = tst()
@@ -781,24 +786,6 @@
return sa
'''%(e1, e2), n, ([], res))
- def test_boolrewrite_ptr_single(self):
- self.run_source('''
- class tst:
- pass
- def main():
- a = tst()
- b = tst()
- c = tst()
- sa = 0
- for i in range(1000):
- if a == b: sa += 1
- else: sa += 2
- if a != b: sa += 10000
- else: sa += 20000
- if i > 750: a = b
- return sa
- ''', 215, ([], 12481752))
-
def test_array_sum(self):
for tc, maxops in zip('bhilBHILfd', (38,) * 6 + (40, 40, 41, 38)):
res = 19352859
@@ -847,7 +834,12 @@
''', 65, ([], 122880))
def test_array_intimg(self):
- for tc, maxops in zip('ilILd', (67, 67, 69, 69, 61)):
+ # XXX this test is way too imprecise in what it is actually testing
+ # it should count the number of guards instead
+ for tc, maxops in zip('ilILd', (67, 67, 70, 70, 61)):
+ print
+ print '='*65
+ print '='*20, 'running test for tc=%r' % (tc,), '='*20
res = 73574560
if tc in 'IL':
res = long(res)
@@ -1130,6 +1122,44 @@
return sa
''', 88, ([], 1997001))
+ def test__ffi_call(self):
+ from pypy.rlib.test.test_libffi import get_libm_name
+ libm_name = get_libm_name(sys.platform)
+ out = self.run_source('''
+ def main():
+ try:
+ from _ffi import CDLL, types
+ except ImportError:
+ sys.stdout.write('SKIP: cannot import _ffi')
+ return 0
+
+ libm = CDLL('%(libm_name)s')
+ pow = libm.getfunc('pow', [types.double, types.double],
+ types.double)
+ print pow.getaddr()
+ i = 0
+ res = 0
+ while i < 2000:
+ res += pow(2, 3)
+ i += 1
+ return res
+ ''' % locals(),
+ 76, ([], 8.0*2000), threshold=1000)
+ pow_addr = int(out.splitlines()[0])
+ ops = self.get_by_bytecode('CALL_FUNCTION')
+ assert len(ops) == 2 # we get two loops, because of specialization
+ call_function = ops[0]
+ last_ops = [op.getopname() for op in call_function[-5:]]
+ assert last_ops == ['force_token',
+ 'setfield_gc',
+ 'call_may_force',
+ 'guard_not_forced',
+ 'guard_no_exception']
+ call = call_function[-3]
+ assert call.getarg(0).value == pow_addr
+ assert call.getarg(1).value == 2.0
+ assert call.getarg(2).value == 3.0
+
# test_circular
class AppTestJIT(PyPyCJITTests):
@@ -1155,6 +1185,17 @@
cls.pypy_c = option.pypy_c
+def test_interface_residual_call():
+ space = gettestobjspace(usemodules=['pypyjit'])
+ space.appexec([], """():
+ import pypyjit
+ def f(*args, **kwds):
+ return (args, kwds)
+ res = pypyjit.residual_call(f, 4, x=6)
+ assert res == ((4,), {'x': 6})
+ """)
+
+
def has_info(pypy_c, option):
g = os.popen('"%s" --info' % pypy_c, 'r')
lines = g.readlines()
Modified: pypy/branch/fast-forward/pypy/module/signal/interp_signal.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/signal/interp_signal.py (original)
+++ pypy/branch/fast-forward/pypy/module/signal/interp_signal.py Fri Oct 22 23:09:43 2010
@@ -1,7 +1,6 @@
from pypy.interpreter.error import OperationError
from pypy.interpreter.baseobjspace import W_Root, ObjSpace
from pypy.interpreter.executioncontext import AsyncAction, AbstractActionFlag
-from pypy.rlib.rarithmetic import LONG_BIT, intmask
import signal as cpy_signal
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.translator.tool.cbuild import ExternalCompilationInfo
@@ -64,8 +63,8 @@
class CheckSignalAction(AsyncAction):
"""An action that is automatically invoked when a signal is received."""
- # The C-level signal handler sets the highest bit of pypysig_occurred:
- bitmask = intmask(1 << (LONG_BIT-1))
+ # The C-level signal handler sets the bit 30 of pypysig_occurred:
+ bitmask = 1 << 30
def __init__(self, space):
AsyncAction.__init__(self, space)
Modified: pypy/branch/fast-forward/pypy/module/sys/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/sys/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/sys/__init__.py Fri Oct 22 23:09:43 2010
@@ -7,13 +7,15 @@
"""Sys Builtin Module. """
def __init__(self, space, w_name):
"""NOT_RPYTHON""" # because parent __init__ isn't
+ if space.config.translating:
+ del self.__class__.interpleveldefs['pypy_getudir']
super(Module, self).__init__(space, w_name)
self.checkinterval = 100
self.recursionlimit = 100
self.w_default_encoder = None
self.defaultencoding = "ascii"
self.filesystemencoding = None
-
+
interpleveldefs = {
'__name__' : '(space.wrap("sys"))',
'__doc__' : '(space.wrap("PyPy sys module"))',
@@ -39,7 +41,7 @@
'py3kwarning' : 'space.w_False',
'warnoptions' : 'state.get(space).w_warnoptions',
'builtin_module_names' : 'state.w_None',
- 'pypy_getudir' : 'state.pypy_getudir',
+ 'pypy_getudir' : 'state.pypy_getudir', # not translated
'pypy_initial_path' : 'state.pypy_initial_path',
'_getframe' : 'vm._getframe',
Modified: pypy/branch/fast-forward/pypy/module/sys/state.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/sys/state.py (original)
+++ pypy/branch/fast-forward/pypy/module/sys/state.py Fri Oct 22 23:09:43 2010
@@ -95,15 +95,8 @@
def getio(space):
return space.fromcache(IOState)
-def _pypy_getudir(space):
- """NOT_RPYTHON"""
+def pypy_getudir(space):
+ """NOT_RPYTHON
+ (should be removed from interpleveldefs before translation)"""
from pypy.tool.udir import udir
return space.wrap(str(udir))
-_pypy_getudir._annspecialcase_ = "override:ignore"
-
-# we need the indirection because this function will live in a dictionary with other
-# RPYTHON functions and share call sites with them. Better it not be a special-case
-# directly.
-def pypy_getudir(space):
- return _pypy_getudir(space)
-
Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c
==============================================================================
--- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c (original)
+++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c Fri Oct 22 23:09:43 2010
@@ -533,3 +533,7 @@
return inp;
}
+int my_unused_function(void)
+{
+ return 42;
+}
Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py (original)
+++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py Fri Oct 22 23:09:43 2010
@@ -148,3 +148,7 @@
# but it segfaults for some reason.
if sys.platform == 'win32':
assert f() == 0x12345678
+
+ def test_restype(self):
+ foo = lib.my_unused_function
+ assert foo.restype is c_int # by default
Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py (original)
+++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py Fri Oct 22 23:09:43 2010
@@ -10,7 +10,7 @@
"""
def test_array_of_pointers(self):
# tests array item assignements & pointer.contents = ...
- A = POINTER(c_int) * 24
+ A = POINTER(c_long) * 24
a = A()
l = c_long(2)
p = pointer(l)
Modified: pypy/branch/fast-forward/pypy/module/thread/ll_thread.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/thread/ll_thread.py (original)
+++ pypy/branch/fast-forward/pypy/module/thread/ll_thread.py Fri Oct 22 23:09:43 2010
@@ -111,7 +111,7 @@
c_thread_releaselock(self._lock)
def __del__(self):
- lltype.free(self._lock, flavor='raw')
+ lltype.free(self._lock, flavor='raw', track_allocation=False)
# ____________________________________________________________
#
@@ -128,10 +128,13 @@
null_ll_lock = lltype.nullptr(TLOCKP.TO)
def allocate_ll_lock():
- ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw')
+ # track_allocation=False here; be careful to lltype.free() it. The
+ # reason it is set to False is that we get it from all app-level
+ # lock objects, as well as from the GIL, which exists at shutdown.
+ ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw', track_allocation=False)
res = c_thread_lock_init(ll_lock)
if res == -1:
- lltype.free(ll_lock, flavor='raw')
+ lltype.free(ll_lock, flavor='raw', track_allocation=False)
raise error("out of resources")
return ll_lock
Modified: pypy/branch/fast-forward/pypy/objspace/flow/flowcontext.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/flow/flowcontext.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/flow/flowcontext.py Fri Oct 22 23:09:43 2010
@@ -398,6 +398,20 @@
break
class FlowSpaceFrame(pyframe.CPythonFrame):
+
+ def SETUP_WITH(self, offsettoend, next_instr):
+ # A simpler version than the 'real' 2.7 one:
+ # directly call manager.__enter__(), don't use special lookup functions
+ # which don't make sense on the RPython type system.
+ from pypy.interpreter.pyopcode import WithBlock
+ w_manager = self.peekvalue()
+ w_exit = self.space.getattr(w_manager, self.space.wrap("__exit__"))
+ self.settopvalue(w_exit)
+ w_result = self.space.call_method(w_manager, "__enter__")
+ block = WithBlock(self, next_instr + offsettoend)
+ self.append_block(block)
+ self.pushvalue(w_result)
+
# XXX Unimplemented 2.7 opcodes ----------------
# Set literals, set comprehensions
@@ -413,9 +427,6 @@
def MAP_ADD(self, oparg, next_instr):
raise NotImplementedError("MAP_ADD")
- # `with` statement
-
-
def make_arguments(self, nargs):
return ArgumentsForTranslation(self.space, self.peekvalues(nargs))
def argument_factory(self, *args):
@@ -427,3 +438,12 @@
raise operr
return pyframe.PyFrame.handle_operation_error(self, ec, operr,
*args, **kwds)
+
+ def call_contextmanager_exit_function(self, w_func, w_typ, w_val, w_tb):
+ if w_typ is not self.space.w_None:
+ # The annotator won't allow to merge exception types with None.
+ # Replace it with an object which will break translation when used
+ # (except maybe with 'exc_typ is None')
+ w_typ = self.space.wrap(self.space)
+ return self.space.call_function(w_func, w_typ, w_val, w_tb)
+
Modified: pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py Fri Oct 22 23:09:43 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
import new
import py
from pypy.objspace.flow.model import Constant, Block, Link, Variable, traverse
@@ -828,6 +829,25 @@
simplify_graph(graph)
assert self.all_operations(graph) == {'getitem': 1}
+ def test_context_manager(self):
+ def f(c, x):
+ with x:
+ pass
+ graph = self.codetest(f)
+ # 2 method calls: x.__enter__() and x.__exit__(None, None, None)
+ assert self.all_operations(graph) == {'getattr': 2,
+ 'simple_call': 2}
+ #
+ def g(): pass
+ def f(c, x):
+ with x:
+ g()
+ graph = self.codetest(f)
+ assert self.all_operations(graph) == {
+ 'getattr': 2, # __enter__ and __exit__
+ 'simple_call': 4, # __enter__, g and 2 possible calls to __exit__
+ 'is_true': 1} # check the result of __exit__()
+
def monkey_patch_code(self, code, stacksize, flags, codestring, names, varnames):
c = code
return new.code(c.co_argcount, c.co_nlocals, stacksize, flags,
Modified: pypy/branch/fast-forward/pypy/objspace/std/celldict.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/celldict.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/celldict.py Fri Oct 22 23:09:43 2010
@@ -45,7 +45,7 @@
if space.is_w(space.type(w_key), space.w_str):
self.impl_setitem_str(self.space.str_w(w_key), w_value)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
def impl_setitem_str(self, name, w_value, shadows_type=True):
self.getcell(name, True).w_value = w_value
@@ -66,7 +66,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_length(self):
# inefficient, but do we care?
@@ -85,7 +85,7 @@
elif _is_sane_hash(space, w_lookup_type):
return None
else:
- return self._as_rdict().getitem(w_lookup)
+ return self._as_rdict().impl_fallback_getitem(w_lookup)
def impl_getitem_str(self, lookup):
res = self.getcell(lookup, False)
Modified: pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py Fri Oct 22 23:09:43 2010
@@ -102,17 +102,17 @@
else:
return None
- # _________________________________________________________________
+ # _________________________________________________________________
# implementation methods
def impl_getitem(self, w_key):
#return w_value or None
raise NotImplementedError("abstract base class")
- def impl_getitem_str(self, w_key):
+ def impl_getitem_str(self, key):
#return w_value or None
raise NotImplementedError("abstract base class")
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value, shadows_type=True):
raise NotImplementedError("abstract base class")
def impl_setitem(self, w_key, w_value):
@@ -120,7 +120,7 @@
def impl_delitem(self, w_key):
raise NotImplementedError("abstract base class")
-
+
def impl_length(self):
raise NotImplementedError("abstract base class")
@@ -310,7 +310,7 @@
if space.is_w(space.type(w_key), space.w_str):
self.impl_setitem_str(self.space.str_w(w_key), w_value)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
def impl_setitem_str(self, key, w_value, shadows_type=True):
self.content[key] = w_value
@@ -324,7 +324,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_length(self):
return len(self.content)
@@ -344,7 +344,7 @@
elif _is_sane_hash(space, w_lookup_type):
return None
else:
- return self._as_rdict().getitem(w_key)
+ return self._as_rdict().impl_fallback_getitem(w_key)
def impl_iter(self):
return StrIteratorImplementation(self.space, self)
@@ -414,7 +414,7 @@
StrDictImplementation.impl_setitem_str(
self, self.space.str_w(w_key), w_value, False)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
def impl_shadows_anything(self):
return (self._shadows_anything or
@@ -446,7 +446,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_get_builtin_indexed(self, i):
return self.shadowed[i]
Modified: pypy/branch/fast-forward/pypy/objspace/std/fake.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/fake.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/fake.py Fri Oct 22 23:09:43 2010
@@ -21,7 +21,6 @@
#debug_print("faking obj %s" % x)
ft = fake_type(type(x))
return ft(space, x)
-fake_object._annspecialcase_ = "override:fake_object"
import sys
@@ -47,7 +46,6 @@
w_exc = space.wrap(exc)
w_value = space.wrap(value)
raise OperationError, OperationError(w_exc, w_value), tb
-wrap_exception._annspecialcase_ = "override:ignore"
def fake_type(cpy_type):
assert type(cpy_type) is type
Modified: pypy/branch/fast-forward/pypy/objspace/std/model.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/model.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/model.py Fri Oct 22 23:09:43 2010
@@ -18,6 +18,7 @@
"withsmallint" : ["smallintobject.W_SmallIntObject"],
"withstrslice" : ["strsliceobject.W_StringSliceObject"],
"withstrjoin" : ["strjoinobject.W_StringJoinObject"],
+ "withstrbuf" : ["strbufobject.W_StringBufferObject"],
"withrope" : ["ropeobject.W_RopeObject",
"ropeobject.W_RopeIterObject"],
"withropeunicode": ["ropeunicodeobject.W_RopeUnicodeObject",
@@ -77,6 +78,7 @@
from pypy.objspace.std import ropeunicodeobject
from pypy.objspace.std import strsliceobject
from pypy.objspace.std import strjoinobject
+ from pypy.objspace.std import strbufobject
from pypy.objspace.std import typeobject
from pypy.objspace.std import sliceobject
from pypy.objspace.std import longobject
@@ -228,6 +230,13 @@
(unicodeobject.W_UnicodeObject,
strjoinobject.delegate_join2unicode)
]
+ elif config.objspace.std.withstrbuf:
+ self.typeorder[strbufobject.W_StringBufferObject] += [
+ (stringobject.W_StringObject,
+ strbufobject.delegate_buf2str),
+ (unicodeobject.W_UnicodeObject,
+ strbufobject.delegate_buf2unicode)
+ ]
if config.objspace.std.withrangelist:
self.typeorder[rangeobject.W_RangeListObject] += [
(listobject.W_ListObject,
Modified: pypy/branch/fast-forward/pypy/objspace/std/objspace.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/objspace.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/objspace.py Fri Oct 22 23:09:43 2010
@@ -23,6 +23,7 @@
from pypy.objspace.std.listobject import W_ListObject
from pypy.objspace.std.longobject import W_LongObject
from pypy.objspace.std.noneobject import W_NoneObject
+from pypy.objspace.std.objectobject import W_ObjectObject
from pypy.objspace.std.ropeobject import W_RopeObject
from pypy.objspace.std.iterobject import W_SeqIterObject
from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject
@@ -242,7 +243,6 @@
w_result = getattr(self, 'w_' + x.__name__)
return w_result
return None
- wrap_exception_cls._annspecialcase_ = "override:wrap_exception_cls"
def unwrap(self, w_obj):
if isinstance(w_obj, Wrappable):
@@ -318,9 +318,14 @@
w_subtype = w_type.check_user_subclass(w_subtype)
if cls.typedef.applevel_subclasses_base is not None:
cls = cls.typedef.applevel_subclasses_base
- subcls = get_unique_interplevel_subclass(
- self.config, cls, w_subtype.hasdict, w_subtype.nslots != 0,
- w_subtype.needsdel, w_subtype.weakrefable)
+ if (self.config.objspace.std.withmapdict and cls is W_ObjectObject
+ and not w_subtype.needsdel):
+ from pypy.objspace.std.mapdict import get_subclass_of_correct_size
+ subcls = get_subclass_of_correct_size(self, cls, w_subtype)
+ else:
+ subcls = get_unique_interplevel_subclass(
+ self.config, cls, w_subtype.hasdict, w_subtype.nslots != 0,
+ w_subtype.needsdel, w_subtype.weakrefable)
instance = instantiate(subcls)
assert isinstance(instance, cls)
instance.user_setup(self, w_subtype)
Modified: pypy/branch/fast-forward/pypy/objspace/std/sharingdict.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/sharingdict.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/sharingdict.py Fri Oct 22 23:09:43 2010
@@ -71,7 +71,7 @@
elif _is_sane_hash(space, w_lookup_type):
return None
else:
- return self._as_rdict().getitem(w_lookup)
+ return self._as_rdict().impl_fallback_getitem(w_lookup)
def impl_getitem_str(self, lookup):
i = self.structure.lookup_position(lookup)
@@ -84,7 +84,7 @@
if space.is_w(space.type(w_key), space.w_str):
self.impl_setitem_str(self.space.str_w(w_key), w_value)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
@unroll_safe
def impl_setitem_str(self, key, w_value, shadows_type=True):
@@ -132,7 +132,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_length(self):
return self.structure.length
Modified: pypy/branch/fast-forward/pypy/objspace/std/stringobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/stringobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/stringobject.py Fri Oct 22 23:09:43 2010
@@ -14,7 +14,7 @@
from pypy.rlib.rstring import StringBuilder, string_repeat
from pypy.interpreter.buffer import StringBuffer
-from pypy.objspace.std.stringtype import sliced, joined, wrapstr, wrapchar, \
+from pypy.objspace.std.stringtype import sliced, wrapstr, wrapchar, \
stringendswith, stringstartswith, joined2
from pypy.objspace.std.formatting import mod_format
Modified: pypy/branch/fast-forward/pypy/objspace/std/stringtype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/stringtype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/stringtype.py Fri Oct 22 23:09:43 2010
@@ -55,19 +55,14 @@
return W_StringSliceObject(s, start, stop)
return wrapstr(space, s[start:stop])
-def joined(space, strlist):
- assert not space.config.objspace.std.withrope
- if space.config.objspace.std.withstrjoin:
- from pypy.objspace.std.strjoinobject import W_StringJoinObject
- return W_StringJoinObject(strlist)
- else:
- return wrapstr(space, "".join(strlist))
-
def joined2(space, str1, str2):
assert not space.config.objspace.std.withrope
if space.config.objspace.std.withstrjoin:
from pypy.objspace.std.strjoinobject import W_StringJoinObject
return W_StringJoinObject([str1, str2])
+ elif space.config.objspace.std.withstrbuf:
+ from pypy.objspace.std.strbufobject import joined2
+ return joined2(str1, str2)
else:
return wrapstr(space, str1 + str2)
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py Fri Oct 22 23:09:43 2010
@@ -602,6 +602,15 @@
classofinstance=classofinstance,
from_strdict_shared=from_strdict_shared)
+ def finditem_str(self, w_dict, s):
+ return w_dict.getitem_str(s) # assume it's a multidict
+
+ def setitem_str(self, w_dict, s, w_value):
+ return w_dict.setitem_str(s, w_value) # assume it's a multidict
+
+ def delitem(self, w_dict, w_s):
+ return w_dict.delitem(w_s) # assume it's a multidict
+
def allocate_instance(self, cls, type):
return object.__new__(cls)
@@ -611,7 +620,7 @@
w_StopIteration = StopIteration
w_None = None
StringObjectCls = FakeString
- w_dict = None
+ w_dict = W_DictMultiObject
iter = iter
fixedview = list
listview = list
@@ -687,6 +696,14 @@
assert self.impl.length() == 0
self.check_not_devolved()
+ def test_clear(self):
+ self.fill_impl()
+ assert self.impl.length() == 2
+ self.impl.clear()
+ assert self.impl.length() == 0
+ self.check_not_devolved()
+
+
def test_keys(self):
self.fill_impl()
keys = self.impl.keys()
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_shadowtracking.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_shadowtracking.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_shadowtracking.py Fri Oct 22 23:09:43 2010
@@ -3,7 +3,8 @@
class TestShadowTracking(object):
def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withshadowtracking": True})
+ cls.space = gettestobjspace(**{"objspace.std.withshadowtracking": True,
+ "objspace.std.withmapdict": False})
def test_simple_shadowing(self):
space = self.space
Modified: pypy/branch/fast-forward/pypy/objspace/std/typeobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/typeobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/typeobject.py Fri Oct 22 23:09:43 2010
@@ -76,7 +76,9 @@
'weakrefable',
'hasdict',
'nslots',
- 'instancetypedef']
+ 'instancetypedef',
+ 'terminator',
+ ]
# for config.objspace.std.getattributeshortcut
# (False is a conservative default, fixed during real usage)
@@ -118,6 +120,12 @@
# dict_w of any of the types in the mro changes, or if the mro
# itself changes
w_self._version_tag = VersionTag()
+ if space.config.objspace.std.withmapdict:
+ from pypy.objspace.std.mapdict import DictTerminator, NoDictTerminator
+ if w_self.hasdict:
+ w_self.terminator = DictTerminator(space, w_self)
+ else:
+ w_self.terminator = NoDictTerminator(space, w_self)
def mutated(w_self):
space = w_self.space
Modified: pypy/branch/fast-forward/pypy/rlib/_rsocket_rffi.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/_rsocket_rffi.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/_rsocket_rffi.py Fri Oct 22 23:09:43 2010
@@ -332,10 +332,11 @@
('sll_hatype', rffi.INT),
('sll_addr', rffi.CFixedArray(rffi.CHAR, 8)),
('sll_halen', rffi.INT)],
- )
+ ifdef='AF_PACKET')
CConfig.ifreq = platform.Struct('struct ifreq', [('ifr_ifindex', rffi.INT),
- ('ifr_name', rffi.CFixedArray(rffi.CHAR, 8))])
+ ('ifr_name', rffi.CFixedArray(rffi.CHAR, 8))],
+ ifdef='AF_PACKET')
if _WIN32:
CConfig.WSAEVENT = platform.SimpleType('WSAEVENT', rffi.VOIDP)
@@ -546,8 +547,9 @@
socketpair_t = rffi.CArray(socketfd_type)
socketpair = external('socketpair', [rffi.INT, rffi.INT, rffi.INT,
lltype.Ptr(socketpair_t)], rffi.INT)
- ioctl = external('ioctl', [socketfd_type, rffi.INT, lltype.Ptr(ifreq)],
- rffi.INT)
+ if ifreq is not None:
+ ioctl = external('ioctl', [socketfd_type, rffi.INT, lltype.Ptr(ifreq)],
+ rffi.INT)
if _WIN32:
ioctlsocket = external('ioctlsocket',
Modified: pypy/branch/fast-forward/pypy/rlib/jit.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/jit.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/jit.py Fri Oct 22 23:09:43 2010
@@ -77,6 +77,12 @@
return result
return decorator
+def oopspec(spec):
+ def decorator(func):
+ func.oopspec = spec
+ return func
+ return decorator
+
class Entry(ExtRegistryEntry):
_about_ = hint
@@ -139,6 +145,24 @@
return hop.inputconst(lltype.Signed, _we_are_jitted)
+def jit_debug(string, arg1=-sys.maxint-1, arg2=-sys.maxint-1,
+ arg3=-sys.maxint-1, arg4=-sys.maxint-1):
+ """When JITted, cause an extra operation DEBUG_MERGE_POINT to appear in
+ the graphs. Should not be left after debugging."""
+ keepalive_until_here(string) # otherwise the whole function call is removed
+jit_debug.oopspec = 'jit.debug(string, arg1, arg2, arg3, arg4)'
+
+def assert_green(value):
+ """Very strong assert: checks that 'value' is a green
+ (a JIT compile-time constant)."""
+ keepalive_until_here(value)
+assert_green._annspecialcase_ = 'specialize:argtype(0)'
+assert_green.oopspec = 'jit.assert_green(value)'
+
+class AssertGreenFailed(Exception):
+ pass
+
+
##def force_virtualizable(virtualizable):
## pass
@@ -250,8 +274,9 @@
several independent JITting interpreters in it.
"""
+ active = True # if set to False, this JitDriver is ignored
virtualizables = []
-
+
def __init__(self, greens=None, reds=None, virtualizables=None,
get_jitcell_at=None, set_jitcell_at=None,
get_printable_location=None, confirm_enter_jit=None,
@@ -266,7 +291,8 @@
self.virtualizables = virtualizables
for v in self.virtualizables:
assert v in self.reds
- self._alllivevars = dict.fromkeys(self.greens + self.reds)
+ self._alllivevars = dict.fromkeys(
+ [name for name in self.greens + self.reds if '.' not in name])
self._make_extregistryentries()
self.get_jitcell_at = get_jitcell_at
self.set_jitcell_at = set_jitcell_at
@@ -355,10 +381,16 @@
def compute_result_annotation(self, **kwds_s):
from pypy.annotation import model as annmodel
+
+ if self.instance.__name__ == 'jit_merge_point':
+ if not self.annotate_hooks(**kwds_s):
+ return None # wrong order, try again later
+
driver = self.instance.im_self
keys = kwds_s.keys()
keys.sort()
- expected = ['s_' + name for name in driver.greens + driver.reds]
+ expected = ['s_' + name for name in driver.greens + driver.reds
+ if '.' not in name]
expected.sort()
if keys != expected:
raise JitHintError("%s expects the following keyword "
@@ -382,30 +414,35 @@
key[2:])
cache[key] = s_value
- if self.instance.__name__ == 'jit_merge_point':
- self.annotate_hooks(**kwds_s)
-
return annmodel.s_None
def annotate_hooks(self, **kwds_s):
driver = self.instance.im_self
s_jitcell = self.bookkeeper.valueoftype(BaseJitCell)
- self.annotate_hook(driver.get_jitcell_at, driver.greens, **kwds_s)
- self.annotate_hook(driver.set_jitcell_at, driver.greens, [s_jitcell],
- **kwds_s)
- self.annotate_hook(driver.get_printable_location, driver.greens, **kwds_s)
+ h = self.annotate_hook
+ return (h(driver.get_jitcell_at, driver.greens, **kwds_s)
+ and h(driver.set_jitcell_at, driver.greens, [s_jitcell], **kwds_s)
+ and h(driver.get_printable_location, driver.greens, **kwds_s))
def annotate_hook(self, func, variables, args_s=[], **kwds_s):
if func is None:
- return
+ return True
bk = self.bookkeeper
s_func = bk.immutablevalue(func)
uniquekey = 'jitdriver.%s' % func.func_name
args_s = args_s[:]
for name in variables:
- s_arg = kwds_s['s_' + name]
+ if '.' not in name:
+ s_arg = kwds_s['s_' + name]
+ else:
+ objname, fieldname = name.split('.')
+ s_instance = kwds_s['s_' + objname]
+ s_arg = s_instance.classdef.about_attribute(fieldname)
+ if s_arg is None:
+ return False # wrong order, try again later
args_s.append(s_arg)
bk.emulate_pbc_call(uniquekey, s_func, args_s)
+ return True
def specialize_call(self, hop, **kwds_i):
# XXX to be complete, this could also check that the concretetype
@@ -416,9 +453,42 @@
greens_v = []
reds_v = []
for name in driver.greens:
- i = kwds_i['i_' + name]
- r_green = hop.args_r[i]
- v_green = hop.inputarg(r_green, arg=i)
+ if '.' not in name:
+ i = kwds_i['i_' + name]
+ r_green = hop.args_r[i]
+ v_green = hop.inputarg(r_green, arg=i)
+ else:
+ if hop.rtyper.type_system.name == 'ootypesystem':
+ py.test.skip("lltype only")
+ objname, fieldname = name.split('.') # see test_green_field
+ assert objname in driver.reds
+ i = kwds_i['i_' + objname]
+ s_red = hop.args_s[i]
+ r_red = hop.args_r[i]
+ while True:
+ try:
+ mangled_name, r_field = r_red._get_field(fieldname)
+ break
+ except KeyError:
+ pass
+ assert r_red.rbase is not None, (
+ "field %r not found in %r" % (name,
+ r_red.lowleveltype.TO))
+ r_red = r_red.rbase
+ GTYPE = r_red.lowleveltype.TO
+ assert GTYPE._immutable_field(mangled_name), (
+ "field %r must be declared as immutable" % name)
+ if not hasattr(driver, 'll_greenfields'):
+ driver.ll_greenfields = {}
+ driver.ll_greenfields[name] = GTYPE, mangled_name
+ #
+ v_red = hop.inputarg(r_red, arg=i)
+ c_llname = hop.inputconst(lltype.Void, mangled_name)
+ v_green = hop.genop('getfield', [v_red, c_llname],
+ resulttype = r_field)
+ s_green = s_red.classdef.about_attribute(fieldname)
+ assert s_green is not None
+ hop.rtyper.annotator.setbinding(v_green, s_green)
greens_v.append(v_green)
for name in driver.reds:
i = kwds_i['i_' + name]
Modified: pypy/branch/fast-forward/pypy/rlib/rgc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rgc.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rgc.py Fri Oct 22 23:09:43 2010
@@ -170,7 +170,14 @@
return hop.genop('gc_set_max_heap_size', [v_nbytes],
resulttype=lltype.Void)
-def can_move(p): # NB. must not be called with NULL pointers
+def can_move(p):
+ """Check if the GC object 'p' is at an address that can move.
+ Must not be called with None. With non-moving GCs, it is always False.
+ With some moving GCs like the SemiSpace GC, it is always True.
+ With other moving GCs like the MiniMark GC, it can be True for some
+ time, then False for the same object, when we are sure that it won't
+ move any more.
+ """
return True
class CanMoveEntry(ExtRegistryEntry):
Modified: pypy/branch/fast-forward/pypy/rlib/rmmap.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rmmap.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rmmap.py Fri Oct 22 23:09:43 2010
@@ -50,7 +50,7 @@
constant_names = ['MAP_SHARED', 'MAP_PRIVATE',
'PROT_READ', 'PROT_WRITE',
'MS_SYNC']
- opt_constant_names = ['MAP_ANON', 'MAP_ANONYMOUS',
+ opt_constant_names = ['MAP_ANON', 'MAP_ANONYMOUS', 'MAP_NORESERVE',
'PROT_EXEC',
'MAP_DENYWRITE', 'MAP_EXECUTABLE']
for name in constant_names:
Modified: pypy/branch/fast-forward/pypy/rlib/rsre/rsre_char.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rsre/rsre_char.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rsre/rsre_char.py Fri Oct 22 23:09:43 2010
@@ -4,6 +4,7 @@
import sys
from pypy.rlib.rlocale import tolower, isalnum
from pypy.rlib.unroll import unrolling_iterable
+from pypy.rlib import jit
# Note: the unicode parts of this module require you to call
# rsre_char.set_unicode_db() first, to select one of the modules
@@ -43,6 +44,7 @@
# XXX can we import those safely from sre_constants?
SRE_INFO_PREFIX = 1
SRE_INFO_LITERAL = 2
+SRE_INFO_CHARSET = 4
SRE_FLAG_LOCALE = 4 # honour system locale
SRE_FLAG_UNICODE = 32 # use unicode locale
OPCODE_INFO = 17
@@ -64,33 +66,27 @@
#### Category helpers
-ascii_char_info = [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 6, 2,
-2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 25, 25, 25, 25, 25, 25, 25, 25,
-25, 25, 0, 0, 0, 0, 0, 0, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0,
-0, 0, 16, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0, 0, 0, 0 ]
-
+is_a_word = [(chr(i).isalnum() or chr(i) == '_') for i in range(256)]
linebreak = ord("\n")
underline = ord("_")
def is_digit(code):
- return code < 128 and (ascii_char_info[code] & 1 != 0)
+ return code <= 57 and code >= 48
def is_uni_digit(code):
assert unicodedb is not None
return unicodedb.isdecimal(code)
def is_space(code):
- return code < 128 and (ascii_char_info[code] & 2 != 0)
+ return code == 32 or (code <= 13 and code >= 9)
def is_uni_space(code):
assert unicodedb is not None
return unicodedb.isspace(code)
def is_word(code):
- return code < 128 and (ascii_char_info[code] & 16 != 0)
+ assert code >= 0
+ return code < 256 and is_a_word[code]
def is_uni_word(code):
assert unicodedb is not None
@@ -142,6 +138,7 @@
SET_OK = -1
SET_NOT_OK = -2
+ at jit.unroll_safe
def check_charset(pattern, ppos, char_code):
"""Checks whether a character matches set of arbitrary length.
The set starts at pattern[ppos]."""
Modified: pypy/branch/fast-forward/pypy/rlib/rsre/rsre_core.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rsre/rsre_core.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rsre/rsre_core.py Fri Oct 22 23:09:43 2010
@@ -1,9 +1,11 @@
import sys
-from pypy.rlib.debug import check_nonneg
+from pypy.rlib.debug import check_nonneg, make_sure_not_modified
from pypy.rlib.unroll import unrolling_iterable
from pypy.rlib.rsre import rsre_char
from pypy.tool.sourcetools import func_with_new_name
from pypy.rlib.objectmodel import we_are_translated
+from pypy.rlib import jit
+from pypy.rlib.rsre.rsre_jit import install_jitdriver, install_jitdriver_spec
OPCODE_FAILURE = 0
@@ -56,16 +58,19 @@
_seen_specname[specname] = True
# Install a copy of the function under the name '_spec_funcname' in each
# concrete subclass
+ specialized_methods = []
for prefix, concreteclass in [('str', StrMatchContext),
('uni', UnicodeMatchContext)]:
newfunc = func_with_new_name(func, prefix + specname)
assert not hasattr(concreteclass, specname)
setattr(concreteclass, specname, newfunc)
+ specialized_methods.append(newfunc)
# Return a dispatcher function, specialized on the exact type of 'ctx'
def dispatch(ctx, *args):
return getattr(ctx, specname)(*args)
dispatch._annspecialcase_ = 'specialize:argtype(0)'
- return dispatch
+ dispatch._specialized_methods_ = specialized_methods
+ return func_with_new_name(dispatch, specname)
# ____________________________________________________________
@@ -75,6 +80,7 @@
class AbstractMatchContext(object):
"""Abstract base class"""
+ _immutable_fields_ = ['pattern[*]', 'flags', 'end']
match_start = 0
match_end = 0
match_marks = None
@@ -164,6 +170,8 @@
def __init__(self, pattern, string, match_start, end, flags):
AbstractMatchContext.__init__(self, pattern, match_start, end, flags)
self._string = string
+ if not we_are_translated() and isinstance(string, unicode):
+ self.flags |= rsre_char.SRE_FLAG_UNICODE # for rsre_re.py
def str(self, index):
check_nonneg(index)
@@ -238,8 +246,9 @@
self.start_ptr = ptr
self.start_marks = marks
+ @jit.unroll_safe
def find_first_result(self, ctx):
- ppos = self.ppos
+ ppos = jit.hint(self.ppos, promote=True)
while ctx.pat(ppos):
result = sre_match(ctx, ppos + 1, self.start_ptr, self.start_marks)
ppos += ctx.pat(ppos)
@@ -250,6 +259,10 @@
find_next_result = find_first_result
class RepeatOneMatchResult(MatchResult):
+ install_jitdriver('RepeatOne',
+ greens=['nextppos', 'ctx.pattern'],
+ reds=['ptr', 'self', 'ctx'],
+ debugprint=(1, 0)) # indices in 'greens'
def __init__(self, nextppos, minptr, ptr, marks):
self.nextppos = nextppos
@@ -259,8 +272,11 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
+ nextppos = self.nextppos
while ptr >= self.minptr:
- result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
+ ctx.jitdriver_RepeatOne.jit_merge_point(
+ self=self, ptr=ptr, ctx=ctx, nextppos=nextppos)
+ result = sre_match(ctx, nextppos, ptr, self.start_marks)
ptr -= 1
if result is not None:
self.subresult = result
@@ -270,6 +286,10 @@
class MinRepeatOneMatchResult(MatchResult):
+ install_jitdriver('MinRepeatOne',
+ greens=['nextppos', 'ppos3', 'ctx.pattern'],
+ reds=['ptr', 'self', 'ctx'],
+ debugprint=(2, 0)) # indices in 'greens'
def __init__(self, nextppos, ppos3, maxptr, ptr, marks):
self.nextppos = nextppos
@@ -280,29 +300,32 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
+ nextppos = self.nextppos
+ ppos3 = self.ppos3
while ptr <= self.maxptr:
- result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
+ ctx.jitdriver_MinRepeatOne.jit_merge_point(
+ self=self, ptr=ptr, ctx=ctx, nextppos=nextppos, ppos3=ppos3)
+ result = sre_match(ctx, nextppos, ptr, self.start_marks)
if result is not None:
self.subresult = result
self.start_ptr = ptr
return self
- if not self.next_char_ok(ctx, ptr):
+ if not self.next_char_ok(ctx, ptr, ppos3):
break
ptr += 1
def find_next_result(self, ctx):
ptr = self.start_ptr
- if not self.next_char_ok(ctx, ptr):
+ if not self.next_char_ok(ctx, ptr, self.ppos3):
return
self.start_ptr = ptr + 1
return self.find_first_result(ctx)
- def next_char_ok(self, ctx, ptr):
+ def next_char_ok(self, ctx, ptr, ppos):
if ptr == ctx.end:
return False
- ppos = self.ppos3
op = ctx.pat(ppos)
- for op1, (checkerfn, _) in unroll_char_checker:
+ for op1, checkerfn in unroll_char_checker:
if op1 == op:
return checkerfn(ctx, ptr, ppos)
raise Error("next_char_ok[%d]" % op)
@@ -325,41 +348,34 @@
self.next = next # chained list
class MaxUntilMatchResult(AbstractUntilMatchResult):
+ install_jitdriver('MaxUntil',
+ greens=['ppos', 'tailppos', 'match_more', 'ctx.pattern'],
+ reds=['ptr', 'marks', 'self', 'ctx'],
+ debugprint=(3, 0, 2))
def find_first_result(self, ctx):
- enum = sre_match(ctx, self.ppos + 3, self.cur_ptr, self.cur_marks)
- return self.search_next(ctx, enum, resume=False)
+ return self.search_next(ctx, match_more=True)
def find_next_result(self, ctx):
- return self.search_next(ctx, None, resume=True)
+ return self.search_next(ctx, match_more=False)
- def search_next(self, ctx, enum, resume):
+ def search_next(self, ctx, match_more):
ppos = self.ppos
- min = ctx.pat(ppos+1)
- max = ctx.pat(ppos+2)
+ tailppos = self.tailppos
ptr = self.cur_ptr
marks = self.cur_marks
while True:
- while True:
- if (enum is not None and
- (ptr != ctx.match_end or self.num_pending < min)):
- # ^^^^^^^^^^ zero-width match protection
- # matched one more 'item'. record it and continue.
- self.pending = Pending(ptr, marks, enum, self.pending)
- self.num_pending += 1
- ptr = ctx.match_end
- marks = ctx.match_marks
- break
- # 'item' no longer matches.
- if not resume and self.num_pending >= min:
- # try to match 'tail' if we have enough 'item'
- result = sre_match(ctx, self.tailppos, ptr, marks)
- if result is not None:
- self.subresult = result
- self.cur_ptr = ptr
- self.cur_marks = marks
- return self
- resume = False
+ ctx.jitdriver_MaxUntil.jit_merge_point(
+ ppos=ppos, tailppos=tailppos, match_more=match_more,
+ ptr=ptr, marks=marks, self=self, ctx=ctx)
+ if match_more:
+ max = ctx.pat(ppos+2)
+ if max == 65535 or self.num_pending < max:
+ # try to match one more 'item'
+ enum = sre_match(ctx, ppos + 3, ptr, marks)
+ else:
+ enum = None # 'max' reached, no more matches
+ else:
p = self.pending
if p is None:
return
@@ -369,11 +385,27 @@
marks = p.marks
enum = p.enum.move_to_next_result(ctx)
#
- if max == 65535 or self.num_pending < max:
- # try to match one more 'item'
- enum = sre_match(ctx, ppos + 3, ptr, marks)
+ min = ctx.pat(ppos+1)
+ if (enum is not None and
+ (ptr != ctx.match_end or self.num_pending < min)):
+ # ^^^^^^^^^^ zero-width match protection
+ # matched one more 'item'. record it and continue.
+ self.pending = Pending(ptr, marks, enum, self.pending)
+ self.num_pending += 1
+ ptr = ctx.match_end
+ marks = ctx.match_marks
+ match_more = True
else:
- enum = None # 'max' reached, no more matches
+ # 'item' no longer matches.
+ if self.num_pending >= min:
+ # try to match 'tail' if we have enough 'item'
+ result = sre_match(ctx, tailppos, ptr, marks)
+ if result is not None:
+ self.subresult = result
+ self.cur_ptr = ptr
+ self.cur_marks = marks
+ return self
+ match_more = False
class MinUntilMatchResult(AbstractUntilMatchResult):
@@ -384,6 +416,7 @@
return self.search_next(ctx, resume=True)
def search_next(self, ctx, resume):
+ # XXX missing jit support here
ppos = self.ppos
min = ctx.pat(ppos+1)
max = ctx.pat(ppos+2)
@@ -429,6 +462,7 @@
# ____________________________________________________________
@specializectx
+ at jit.unroll_safe
def sre_match(ctx, ppos, ptr, marks):
"""Returns either None or a MatchResult object. Usually we only need
the first result, but there is the case of REPEAT...UNTIL where we
@@ -437,6 +471,13 @@
while True:
op = ctx.pat(ppos)
ppos += 1
+ make_sure_not_modified(ctx.pattern)
+
+ #jit.jit_debug("sre_match", op, ppos, ptr)
+ #
+ # When using the JIT, calls to sre_match() must always have a constant
+ # (green) argument for 'ppos'. If not, the following assert fails.
+ jit.assert_green(op)
if op == OPCODE_FAILURE:
return
@@ -712,13 +753,23 @@
@specializectx
def find_repetition_end(ctx, ppos, ptr, maxcount):
end = ctx.end
- # adjust end
- if maxcount != 65535:
+ if maxcount <= 1:
+ if maxcount == 1 and ptr < end:
+ # Relatively common case: maxcount == 1. If we are not at the
+ # end of the string, it's done by a single direct check.
+ op = ctx.pat(ppos)
+ for op1, checkerfn in unroll_char_checker:
+ if op1 == op:
+ if checkerfn(ctx, ptr, ppos):
+ return ptr + 1
+ return ptr
+ elif maxcount != 65535:
+ # adjust end
end1 = ptr + maxcount
if end1 <= end:
end = end1
op = ctx.pat(ppos)
- for op1, (_, fre) in unroll_char_checker:
+ for op1, fre in unroll_fre_checker:
if op1 == op:
return fre(ctx, ptr, end, ppos)
raise Error("rsre.find_repetition_end[%d]" % op)
@@ -751,23 +802,60 @@
if checkerfn == match_ANY_ALL:
def fre(ctx, ptr, end, ppos):
return end
+ elif checkerfn == match_IN:
+ install_jitdriver_spec('MatchIn',
+ greens=['ppos', 'ctx.pattern'],
+ reds=['ptr', 'end', 'ctx'],
+ debugprint=(1, 0))
+ @specializectx
+ def fre(ctx, ptr, end, ppos):
+ while True:
+ ctx.jitdriver_MatchIn.jit_merge_point(ctx=ctx, ptr=ptr,
+ end=end, ppos=ppos)
+ if ptr < end and checkerfn(ctx, ptr, ppos):
+ ptr += 1
+ else:
+ return ptr
+ elif checkerfn == match_IN_IGNORE:
+ install_jitdriver_spec('MatchInIgnore',
+ greens=['ppos', 'ctx.pattern'],
+ reds=['ptr', 'end', 'ctx'],
+ debugprint=(1, 0))
+ @specializectx
+ def fre(ctx, ptr, end, ppos):
+ while True:
+ ctx.jitdriver_MatchInIgnore.jit_merge_point(ctx=ctx, ptr=ptr,
+ end=end, ppos=ppos)
+ if ptr < end and checkerfn(ctx, ptr, ppos):
+ ptr += 1
+ else:
+ return ptr
else:
+ # in the other cases, the fre() function is not JITted at all
+ # and is present as a residual call.
+ @specializectx
def fre(ctx, ptr, end, ppos):
while ptr < end and checkerfn(ctx, ptr, ppos):
ptr += 1
return ptr
- return checkerfn, fre
+ fre = func_with_new_name(fre, 'fre_' + checkerfn.__name__)
+ return fre
+
+unroll_char_checker = [
+ (OPCODE_ANY, match_ANY),
+ (OPCODE_ANY_ALL, match_ANY_ALL),
+ (OPCODE_IN, match_IN),
+ (OPCODE_IN_IGNORE, match_IN_IGNORE),
+ (OPCODE_LITERAL, match_LITERAL),
+ (OPCODE_LITERAL_IGNORE, match_LITERAL_IGNORE),
+ (OPCODE_NOT_LITERAL, match_NOT_LITERAL),
+ (OPCODE_NOT_LITERAL_IGNORE, match_NOT_LITERAL_IGNORE),
+ ]
+unroll_fre_checker = [(_op, _make_fre(_fn))
+ for (_op, _fn) in unroll_char_checker]
-unroll_char_checker = unrolling_iterable([
- (OPCODE_ANY, _make_fre(match_ANY)),
- (OPCODE_ANY_ALL, _make_fre(match_ANY_ALL)),
- (OPCODE_IN, _make_fre(match_IN)),
- (OPCODE_IN_IGNORE, _make_fre(match_IN_IGNORE)),
- (OPCODE_LITERAL, _make_fre(match_LITERAL)),
- (OPCODE_LITERAL_IGNORE, _make_fre(match_LITERAL_IGNORE)),
- (OPCODE_NOT_LITERAL, _make_fre(match_NOT_LITERAL)),
- (OPCODE_NOT_LITERAL_IGNORE, _make_fre(match_NOT_LITERAL_IGNORE)),
- ])
+unroll_char_checker = unrolling_iterable(unroll_char_checker)
+unroll_fre_checker = unrolling_iterable(unroll_fre_checker)
##### At dispatch
@@ -873,74 +961,139 @@
else:
return None
+install_jitdriver('Match',
+ greens=['ctx.pattern'], reds=['ctx'],
+ debugprint=(0,))
+
def match_context(ctx):
ctx.original_pos = ctx.match_start
if ctx.end < ctx.match_start:
return False
+ ctx.jitdriver_Match.jit_merge_point(ctx=ctx)
return sre_match(ctx, 0, ctx.match_start, None) is not None
def search_context(ctx):
ctx.original_pos = ctx.match_start
if ctx.end < ctx.match_start:
return False
- if ctx.pat(0) == OPCODE_INFO:
- if ctx.pat(2) & rsre_char.SRE_INFO_PREFIX and ctx.pat(5) > 1:
- return fast_search(ctx)
- return regular_search(ctx)
+ base = 0
+ charset = False
+ if ctx.pat(base) == OPCODE_INFO:
+ flags = ctx.pat(2)
+ if flags & rsre_char.SRE_INFO_PREFIX:
+ if ctx.pat(5) > 1:
+ return fast_search(ctx)
+ else:
+ charset = (flags & rsre_char.SRE_INFO_CHARSET)
+ base += 1 + ctx.pat(1)
+ if ctx.pat(base) == OPCODE_LITERAL:
+ return literal_search(ctx, base)
+ if charset:
+ return charset_search(ctx, base)
+ return regular_search(ctx, base)
+
+install_jitdriver('RegularSearch',
+ greens=['base', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(1, 0))
-def regular_search(ctx):
+def regular_search(ctx, base):
start = ctx.match_start
while start <= ctx.end:
- if sre_match(ctx, 0, start, None) is not None:
+ ctx.jitdriver_RegularSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base)
+ if sre_match(ctx, base, start, None) is not None:
ctx.match_start = start
return True
start += 1
return False
+install_jitdriver_spec("LiteralSearch",
+ greens=['base', 'character', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(2, 0, 1))
+ at specializectx
+def literal_search(ctx, base):
+ # pattern starts with a literal character. this is used
+ # for short prefixes, and if fast search is disabled
+ character = ctx.pat(base + 1)
+ base += 2
+ start = ctx.match_start
+ while start < ctx.end:
+ ctx.jitdriver_LiteralSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base, character=character)
+ if ctx.str(start) == character:
+ if sre_match(ctx, base, start + 1, None) is not None:
+ ctx.match_start = start
+ return True
+ start += 1
+ return False
+
+install_jitdriver_spec("CharsetSearch",
+ greens=['base', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(1, 0))
+ at specializectx
+def charset_search(ctx, base):
+ # pattern starts with a character from a known set
+ start = ctx.match_start
+ while start < ctx.end:
+ ctx.jitdriver_CharsetSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base)
+ if rsre_char.check_charset(ctx.pattern, 5, ctx.str(start)):
+ if sre_match(ctx, base, start, None) is not None:
+ ctx.match_start = start
+ return True
+ start += 1
+ return False
+
+install_jitdriver_spec('FastSearch',
+ greens=['i', 'prefix_len', 'ctx.pattern'],
+ reds=['string_position', 'ctx'],
+ debugprint=(2, 0))
@specializectx
def fast_search(ctx):
# skips forward in a string as fast as possible using information from
# an optimization info block
# <1=skip> <2=flags> <3=min> <4=...>
# <5=length> <6=skip> <7=prefix data>
- flags = ctx.pat(2)
+ string_position = ctx.match_start
+ if string_position >= ctx.end:
+ return False
prefix_len = ctx.pat(5)
assert prefix_len >= 0
- prefix_skip = ctx.pat(6)
- assert prefix_skip >= 0
- overlap_offset = 7 + prefix_len - 1
- assert overlap_offset >= 0
- pattern_offset = ctx.pat(1) + 1
- ppos_start = pattern_offset + 2 * prefix_skip
- assert ppos_start >= 0
i = 0
- string_position = ctx.match_start
- end = ctx.end
- while string_position < end:
- while True:
- char_ord = ctx.str(string_position)
- if char_ord != ctx.pat(7 + i):
- if i == 0:
- break
- else:
- i = ctx.pat(overlap_offset + i)
- else:
- i += 1
- if i == prefix_len:
- # found a potential match
- start = string_position + 1 - prefix_len
- assert start >= 0
- ptr = start + prefix_skip
- if flags & rsre_char.SRE_INFO_LITERAL:
- # matched all of pure literal pattern
- ctx.match_start = start
- ctx.match_end = ptr
- ctx.match_marks = None
- return True
- if sre_match(ctx, ppos_start, ptr, None) is not None:
- ctx.match_start = start
- return True
- i = ctx.pat(overlap_offset + i)
- break
+ while True:
+ ctx.jitdriver_FastSearch.jit_merge_point(ctx=ctx,
+ string_position=string_position, i=i, prefix_len=prefix_len)
+ char_ord = ctx.str(string_position)
+ if char_ord != ctx.pat(7 + i):
+ if i > 0:
+ overlap_offset = prefix_len + (7 - 1)
+ i = ctx.pat(overlap_offset + i)
+ continue
+ else:
+ i += 1
+ if i == prefix_len:
+ # found a potential match
+ start = string_position + 1 - prefix_len
+ assert start >= 0
+ prefix_skip = ctx.pat(6)
+ ptr = start + prefix_skip
+ #flags = ctx.pat(2)
+ #if flags & rsre_char.SRE_INFO_LITERAL:
+ # # matched all of pure literal pattern
+ # ctx.match_start = start
+ # ctx.match_end = ptr
+ # ctx.match_marks = None
+ # return True
+ pattern_offset = ctx.pat(1) + 1
+ ppos_start = pattern_offset + 2 * prefix_skip
+ if sre_match(ctx, ppos_start, ptr, None) is not None:
+ ctx.match_start = start
+ return True
+ overlap_offset = prefix_len + (7 - 1)
+ i = ctx.pat(overlap_offset + i)
string_position += 1
- return False
+ if string_position >= ctx.end:
+ return False
Modified: pypy/branch/fast-forward/pypy/rlib/rsre/test/test_match.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rsre/test/test_match.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rsre/test/test_match.py Fri Oct 22 23:09:43 2010
@@ -1,24 +1,49 @@
-import _sre, re, sre_compile
-from pypy.rlib.rsre import rsre_core
+import re
+from pypy.rlib.rsre import rsre_core, rsre_char
-def get_code(regexp, flags=0, allargs=False):
- class GotIt(Exception):
- pass
- def my_compile(pattern, flags, code, *args):
- print code
- raise GotIt(code, flags, args)
- saved = _sre.compile
- try:
- _sre.compile = my_compile
- try:
- sre_compile.compile(regexp, flags)
- except GotIt, e:
- pass
+def get_hacked_sre_compile(my_compile):
+ """Return a copy of the sre_compile module for which the _sre
+ module is a custom module that has _sre.compile == my_compile
+ and CODESIZE == rsre_char.CODESIZE.
+ """
+ import sre_compile, __builtin__, new
+ sre_hacked = new.module("_sre_hacked")
+ sre_hacked.compile = my_compile
+ sre_hacked.MAGIC = sre_compile.MAGIC
+ sre_hacked.CODESIZE = rsre_char.CODESIZE
+ sre_hacked.getlower = rsre_char.getlower
+ def my_import(name, *args):
+ if name == '_sre':
+ return sre_hacked
else:
- raise ValueError("did not reach _sre.compile()!")
+ return default_import(name, *args)
+ src = sre_compile.__file__
+ if src.lower().endswith('.pyc') or src.lower().endswith('.pyo'):
+ src = src[:-1]
+ mod = new.module("sre_compile_hacked")
+ default_import = __import__
+ try:
+ __builtin__.__import__ = my_import
+ execfile(src, mod.__dict__)
finally:
- _sre.compile = saved
+ __builtin__.__import__ = default_import
+ return mod
+
+class GotIt(Exception):
+ pass
+def my_compile(pattern, flags, code, *args):
+ print code
+ raise GotIt(code, flags, args)
+sre_compile_hacked = get_hacked_sre_compile(my_compile)
+
+def get_code(regexp, flags=0, allargs=False):
+ try:
+ sre_compile_hacked.compile(regexp, flags)
+ except GotIt, e:
+ pass
+ else:
+ raise ValueError("did not reach _sre.compile()!")
if allargs:
return e.args
else:
Modified: pypy/branch/fast-forward/pypy/rlib/rstring.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rstring.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rstring.py Fri Oct 22 23:09:43 2010
@@ -69,6 +69,9 @@
def build(self):
return self.tp("").join(self.l)
+ def getlength(self):
+ return len(self.build())
+
class StringBuilder(AbstractStringBuilder):
tp = str
@@ -122,9 +125,12 @@
assert isinstance(s_times, SomeInteger)
return s_None
+ def method_getlength(self):
+ return SomeInteger(nonneg=True)
+
def method_build(self):
return SomeString()
-
+
def rtyper_makerepr(self, rtyper):
return rtyper.type_system.rbuilder.stringbuilder_repr
@@ -144,6 +150,9 @@
assert isinstance(s_times, SomeInteger)
return s_None
+ def method_getlength(self):
+ return SomeInteger(nonneg=True)
+
def method_build(self):
return SomeUnicodeString()
Modified: pypy/branch/fast-forward/pypy/rlib/test/test_jit.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/test/test_jit.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/test/test_jit.py Fri Oct 22 23:09:43 2010
@@ -1,10 +1,17 @@
import py
+from pypy.conftest import option
from pypy.rlib.jit import hint, we_are_jitted, JitDriver, purefunction_promote
-from pypy.rlib.jit import JitHintError
+from pypy.rlib.jit import JitHintError, oopspec
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.rpython.lltypesystem import lltype
+def test_oopspec():
+ @oopspec('foobar')
+ def fn():
+ pass
+ assert fn.oopspec == 'foobar'
+
class BaseTestJIT(BaseRtypingTest):
def test_hint(self):
def f():
@@ -104,6 +111,26 @@
return n
py.test.raises(JitHintError, self.gengraph, fn, [int])
+ def test_green_field(self):
+ def get_printable_location(xfoo):
+ return str(ord(xfoo)) # xfoo must be annotated as a character
+ myjitdriver = JitDriver(greens=['x.foo'], reds=['n', 'x'],
+ get_printable_location=get_printable_location)
+ class A(object):
+ _immutable_fields_ = ['foo']
+ def fn(n):
+ x = A()
+ x.foo = chr(n)
+ while n > 0:
+ myjitdriver.can_enter_jit(x=x, n=n)
+ myjitdriver.jit_merge_point(x=x, n=n)
+ n -= 1
+ return n
+ t = self.gengraph(fn, [int])[0]
+ if option.view:
+ t.view()
+ # assert did not raise
+
class TestJITLLtype(BaseTestJIT, LLRtypeMixin):
pass
Modified: pypy/branch/fast-forward/pypy/rlib/test/test_rdynload.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/test/test_rdynload.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/test/test_rdynload.py Fri Oct 22 23:09:43 2010
@@ -1,15 +1,22 @@
from pypy.rlib.rdynload import *
-from pypy.rlib.libffi import get_libc_name
+from pypy.rlib.clibffi import get_libc_name
from pypy.rpython.lltypesystem import rffi, lltype
import py
class TestDLOperations:
def test_dlopen(self):
- py.test.raises(DLOpenError, "dlopen(rffi.str2charp('xxxxxxxxxxxx'))")
- assert dlopen(rffi.str2charp(get_libc_name()))
+ s = rffi.str2charp('xxxxxxxxxxxx')
+ py.test.raises(DLOpenError, "dlopen(s)")
+ rffi.free_charp(s)
+ #
+ s = rffi.str2charp(get_libc_name())
+ assert dlopen(s)
+ rffi.free_charp(s)
def test_dlsym(self):
- lib = dlopen(rffi.str2charp(get_libc_name()))
+ s = rffi.str2charp(get_libc_name())
+ lib = dlopen(s)
+ rffi.free_charp(s)
handle = rffi.cast(lltype.Ptr(lltype.FuncType([lltype.Signed],
lltype.Signed)), dlsym(lib, 'abs'))
assert 1 == handle(1)
Modified: pypy/branch/fast-forward/pypy/rlib/test/test_rsocket.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/test/test_rsocket.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/test/test_rsocket.py Fri Oct 22 23:09:43 2010
@@ -437,3 +437,31 @@
foo = self.serv.accept()
py.test.raises(SocketError, raise_error)
+def _test_cond_include(cond):
+ # Test that _rsocket_rffi is importable even on platforms where
+ # AF_PACKET or AF_NETLINK is not defined.
+ import re
+ from pypy.rlib import _rsocket_rffi
+ srcfile = _rsocket_rffi.__file__
+ if srcfile.lower().endswith('c') or srcfile.lower().endswith('o'):
+ srcfile = srcfile[:-1] # .pyc => .py
+ assert srcfile.lower().endswith('.py')
+ sourcelines = open(srcfile, 'rb').read().splitlines()
+ found = False
+ for i, line in enumerate(sourcelines):
+ line2 = re.sub(r"(\s*COND_HEADER\s*=)",
+ r"\1'#undef %s\\n'+" % cond,
+ line)
+ if line2 != line:
+ found = True
+ sourcelines[i] = line2
+ assert found
+ d = {}
+ sourcelines.append('')
+ exec '\n'.join(sourcelines) in d
+
+def test_no_AF_PACKET():
+ _test_cond_include('AF_PACKET')
+
+def test_no_AF_NETLINK():
+ _test_cond_include('AF_NETLINK')
Modified: pypy/branch/fast-forward/pypy/rlib/test/test_rstring.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/test/test_rstring.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/test/test_rstring.py Fri Oct 22 23:09:43 2010
@@ -29,6 +29,7 @@
s = StringBuilder()
s.append("a")
s.append("abc")
+ assert s.getlength() == len('aabc')
s.append("a")
s.append_slice("abc", 1, 2)
s.append_multiple_char('d', 4)
@@ -39,6 +40,7 @@
s.append(u'a')
s.append(u'abc')
s.append_slice(u'abcdef', 1, 2)
+ assert s.getlength() == len('aabcb')
s.append_multiple_char(u'd', 4)
assert s.build() == 'aabcbdddd'
assert isinstance(s.build(), unicode)
Modified: pypy/branch/fast-forward/pypy/rlib/test/test_rzlib.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/test/test_rzlib.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/test/test_rzlib.py Fri Oct 22 23:09:43 2010
@@ -189,6 +189,8 @@
assert unused3 == len('more_garbage')
assert data3 == ''
+ rzlib.deflateEnd(stream)
+
def test_decompress_max_length():
"""
@@ -205,6 +207,8 @@
assert finished2 is True
assert unused2 == 0
+ rzlib.deflateEnd(stream)
+
def test_cornercases():
"""
@@ -215,6 +219,7 @@
bytes += rzlib.compress(stream, "")
bytes += rzlib.compress(stream, "", rzlib.Z_FINISH)
assert zlib.decompress(bytes) == ""
+ rzlib.deflateEnd(stream)
stream = rzlib.inflateInit()
data, finished, unused = rzlib.decompress(stream, "")
@@ -228,3 +233,4 @@
assert finished is False
assert unused > 0
buf = buf[-unused:]
+ rzlib.deflateEnd(stream)
Modified: pypy/branch/fast-forward/pypy/rpython/llinterp.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/llinterp.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/llinterp.py Fri Oct 22 23:09:43 2010
@@ -48,8 +48,7 @@
current_interpreter = None
- def __init__(self, typer, tracing=True, exc_data_ptr=None,
- malloc_check=True):
+ def __init__(self, typer, tracing=True, exc_data_ptr=None):
self.bindings = {}
self.typer = typer
# 'heap' is module or object that provides malloc, etc for lltype ops
@@ -57,9 +56,7 @@
self.exc_data_ptr = exc_data_ptr
self.frame_stack = []
self.tracer = None
- self.malloc_check = malloc_check
self.frame_class = LLFrame
- self.mallocs = {}
if tracing:
self.tracer = Tracer()
@@ -163,24 +160,6 @@
return self.exc_data_ptr
return None
- def remember_malloc(self, ptr, llframe):
- # err....
- self.mallocs[ptr._obj] = llframe
-
- def remember_free(self, ptr):
- try:
- del self.mallocs[ptr._obj]
- except KeyError:
- self._rehash_mallocs()
- del self.mallocs[ptr._obj]
-
- def _rehash_mallocs(self):
- # rehashing is needed because some objects' hash may change
- # when being turned to
- items = self.mallocs.items()
- self.mallocs = {}
- self.mallocs.update(items)
-
def _store_exception(self, exc):
raise PleaseOverwriteStoreException("You just invoked ll2ctypes callback without overwriting _store_exception on llinterpreter")
@@ -726,23 +705,23 @@
def op_malloc(self, obj, flags):
flavor = flags['flavor']
zero = flags.get('zero', False)
+ track_allocation = flags.get('track_allocation', True)
if flavor == "stack":
result = self.heap.malloc(obj, zero=zero, flavor='raw')
self.alloca_objects.append(result)
return result
- ptr = self.heap.malloc(obj, zero=zero, flavor=flavor)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_malloc(ptr, self)
+ ptr = self.heap.malloc(obj, zero=zero, flavor=flavor,
+ track_allocation=track_allocation)
return ptr
def op_malloc_varsize(self, obj, flags, size):
flavor = flags['flavor']
zero = flags.get('zero', False)
+ track_allocation = flags.get('track_allocation', True)
assert flavor in ('gc', 'raw')
try:
- ptr = self.heap.malloc(obj, size, zero=zero, flavor=flavor)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_malloc(ptr, self)
+ ptr = self.heap.malloc(obj, size, zero=zero, flavor=flavor,
+ track_allocation=track_allocation)
return ptr
except MemoryError:
self.make_llexception()
@@ -759,11 +738,10 @@
zero = flags.get('zero', False)
return self.heap.malloc_nonmovable(TYPE, size, zero=zero)
- def op_free(self, obj, flavor):
- assert isinstance(flavor, str)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_free(obj)
- self.heap.free(obj, flavor=flavor)
+ def op_free(self, obj, flags):
+ assert flags['flavor'] == 'raw'
+ track_allocation = flags.get('track_allocation', True)
+ self.heap.free(obj, flavor='raw', track_allocation=track_allocation)
def op_shrink_array(self, obj, smallersize):
return self.heap.shrink_array(obj, smallersize)
@@ -1037,6 +1015,13 @@
def op_stack_malloc(self, size): # mmh
raise NotImplementedError("backend only")
+ def op_track_alloc_start(self, addr):
+ # we don't do tracking at this level
+ checkadr(addr)
+
+ def op_track_alloc_stop(self, addr):
+ checkadr(addr)
+
# ____________________________________________________________
# Overflow-detecting variants
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/ll2ctypes.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/ll2ctypes.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/ll2ctypes.py Fri Oct 22 23:09:43 2010
@@ -72,7 +72,7 @@
PIECESIZE = 0x08000000
PIECES = 10
m = rmmap.mmap(-1, PIECES * PIECESIZE,
- rmmap.MAP_PRIVATE|rmmap.MAP_ANONYMOUS,
+ rmmap.MAP_PRIVATE|rmmap.MAP_ANONYMOUS|rmmap.MAP_NORESERVE,
rmmap.PROT_READ|rmmap.PROT_WRITE)
m.close = lambda : None # leak instead of giving a spurious
# error at CPython's shutdown
@@ -827,6 +827,8 @@
except (ValueError, OverflowError):
for tc in 'HIL':
if array(tc).itemsize == array('u').itemsize:
+ import struct
+ cobj &= 256 ** struct.calcsize(tc) - 1
llobj = array('u', array(tc, (cobj,)).tostring())[0]
break
else:
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py Fri Oct 22 23:09:43 2010
@@ -440,7 +440,7 @@
[rffi.INT],
rffi.INT,
sandboxsafe=True, _nowrapper=True)
- _dev_zero = rffi.str2charp('/dev/zero') # prebuilt
+ _dev_zero = rffi.str2charp_immortal('/dev/zero') # prebuilt
def clear_large_memory_chunk(baseaddr, size):
# on some Unixy platforms, reading from /dev/zero is the fastest way
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/llmemory.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/llmemory.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/llmemory.py Fri Oct 22 23:09:43 2010
@@ -105,11 +105,13 @@
if (isinstance(self.TYPE, lltype.ContainerType)
and self.TYPE._gckind == 'gc'):
assert self.repeat == 1
- p = lltype.malloc(self.TYPE, flavor='raw', zero=zero)
+ p = lltype.malloc(self.TYPE, flavor='raw', zero=zero,
+ track_allocation=False)
return cast_ptr_to_adr(p)
else:
T = lltype.FixedSizeArray(self.TYPE, self.repeat)
- p = lltype.malloc(T, flavor='raw', zero=zero)
+ p = lltype.malloc(T, flavor='raw', zero=zero,
+ track_allocation=False)
array_adr = cast_ptr_to_adr(p)
return array_adr + ArrayItemsOffset(T)
@@ -288,7 +290,8 @@
count = 0
p = lltype.malloc(parenttype or self.TYPE, count,
immortal = self.TYPE._gckind == 'raw',
- zero = zero)
+ zero = zero,
+ track_allocation = False)
return cast_ptr_to_adr(p)
def raw_memcopy(self, srcadr, dstadr):
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/lloperation.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/lloperation.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/lloperation.py Fri Oct 22 23:09:43 2010
@@ -405,6 +405,8 @@
'raw_load': LLOp(sideeffects=False),
'raw_store': LLOp(),
'stack_malloc': LLOp(), # mmh
+ 'track_alloc_start': LLOp(),
+ 'track_alloc_stop': LLOp(),
'adr_add': LLOp(canfold=True),
'adr_sub': LLOp(canfold=True),
'adr_delta': LLOp(canfold=True),
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/lltype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/lltype.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/lltype.py Fri Oct 22 23:09:43 2010
@@ -1,7 +1,3 @@
-import StringIO
-import traceback
-import sys
-
import py
from pypy.rlib.rarithmetic import (r_int, r_uint, intmask, r_singlefloat,
r_ulonglong, r_longlong, r_longfloat,
@@ -10,25 +6,13 @@
from pypy.tool.uid import Hashable
from pypy.tool.tls import tlsobject
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
from types import NoneType
from sys import maxint
import weakref
TLS = tlsobject()
-# Track allocations to detect memory leaks
-# Don't track 'gc' and immortal mallocs
-TRACK_ALLOCATIONS = False
-ALLOCATED = identity_dict()
-
-def start_tracking_allocations():
- global TRACK_ALLOCATIONS
- TRACK_ALLOCATIONS = True
- ALLOCATED.clear()
-
-def stop_tracking_allocations():
- global TRACK_ALLOCATIONS
- TRACK_ALLOCATIONS = False
class _uninitialized(object):
def __init__(self, TYPE):
@@ -801,6 +785,8 @@
return llmemory.cast_adr_to_ptr(value, TGT)
elif TGT == llmemory.Address and isinstance(ORIG, Ptr):
return llmemory.cast_ptr_to_adr(value)
+ elif TGT == Signed and isinstance(ORIG, Ptr) and ORIG.TO._gckind == 'raw':
+ return llmemory.cast_adr_to_int(llmemory.cast_ptr_to_adr(value), 'symbolic')
raise TypeError("don't know how to cast from %r to %r" % (ORIG, TGT))
@@ -1387,41 +1373,21 @@
__slots__ = ('_TYPE',
'_parent_type', '_parent_index', '_keepparent',
'_wrparent',
- '__weakref__', '_traceback',
- '__storage')
+ '__weakref__',
+ '_storage')
- def __init__(self, TYPE, track_allocation=None):
+ def __init__(self, TYPE):
self._wrparent = None
self._TYPE = TYPE
self._storage = True # means "use default storage", as opposed to:
# None - container was freed
# - using ctypes
# (see ll2ctypes.py)
- if track_allocation is not False and TRACK_ALLOCATIONS:
- self._traceback = self._get_traceback()
- ALLOCATED[self] = None
- else:
- self._traceback = None
-
- def _get_traceback(self):
- frame = sys._getframe().f_back.f_back.f_back.f_back
- sio = StringIO.StringIO()
- traceback.print_stack(frame, file=sio)
- return sio.getvalue()
def _free(self):
self._check() # no double-frees
self._storage = None
- def _storage_get(self):
- return self.__storage
-
- def _storage_set(self, value):
- self.__storage = value
- if value is not True and self in ALLOCATED:
- del ALLOCATED[self]
- _storage = property(_storage_get, _storage_set)
-
def _was_freed(self):
if self._storage is None:
return True
@@ -1500,12 +1466,12 @@
__slots__ = ('_hash_cache_', '_compilation_info')
- def __new__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None, track_allocation=None):
+ def __new__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None):
my_variety = _struct_variety(TYPE._names)
return object.__new__(my_variety)
- def __init__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None, track_allocation=None):
- _parentable.__init__(self, TYPE, track_allocation)
+ def __init__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None):
+ _parentable.__init__(self, TYPE)
if n is not None and TYPE._arrayfld is None:
raise TypeError("%r is not variable-sized" % (TYPE,))
if n is None and TYPE._arrayfld is not None:
@@ -1513,8 +1479,7 @@
first, FIRSTTYPE = TYPE._first_struct()
for fld, typ in TYPE._flds.items():
if fld == TYPE._arrayfld:
- value = _array(typ, n, initialization=initialization, parent=self, parentindex=fld,
- track_allocation=track_allocation)
+ value = _array(typ, n, initialization=initialization, parent=self, parentindex=fld)
else:
value = typ._allocate(initialization=initialization, parent=self, parentindex=fld)
setattr(self, fld, value)
@@ -1575,12 +1540,12 @@
__slots__ = ('items',)
- def __init__(self, TYPE, n, initialization=None, parent=None, parentindex=None, track_allocation=None):
+ def __init__(self, TYPE, n, initialization=None, parent=None, parentindex=None):
if not isinstance(n, int):
raise TypeError, "array length must be an int"
if n < 0:
raise ValueError, "negative array length"
- _parentable.__init__(self, TYPE, track_allocation)
+ _parentable.__init__(self, TYPE)
try:
myrange = range(n)
except OverflowError:
@@ -1647,7 +1612,7 @@
_cache = weakref.WeakKeyDictionary() # parentarray -> {subarrays}
def __init__(self, TYPE, parent, baseoffset_or_fieldname):
- _parentable.__init__(self, TYPE, track_allocation=False)
+ _parentable.__init__(self, TYPE)
self._setparentstructure(parent, baseoffset_or_fieldname)
# Keep the parent array alive, we share the same allocation.
# Don't do it if we are inside a GC object, though -- it's someone
@@ -1655,6 +1620,13 @@
if typeOf(top_container(parent))._gckind == 'raw':
self._keepparent = parent
+ def __str__(self):
+ parent = self._wrparent()
+ if parent is None:
+ return '_subarray at %s in already freed' % (self._parent_index,)
+ return '_subarray at %r in %s' % (self._parent_index,
+ parent._TYPE)
+
def __repr__(self):
parent = self._wrparent()
if parent is None:
@@ -1868,8 +1840,9 @@
return id(self.value)
-def malloc(T, n=None, flavor='gc', immortal=False, zero=False):
- assert flavor != 'cpy'
+def malloc(T, n=None, flavor='gc', immortal=False, zero=False,
+ track_allocation=True):
+ assert flavor in ('gc', 'raw')
if zero or immortal:
initialization = 'example'
elif flavor == 'raw':
@@ -1877,9 +1850,9 @@
else:
initialization = 'malloc'
if isinstance(T, Struct):
- o = _struct(T, n, initialization=initialization, track_allocation=flavor == "raw" and not immortal)
+ o = _struct(T, n, initialization=initialization)
elif isinstance(T, Array):
- o = _array(T, n, initialization=initialization, track_allocation=flavor == "raw" and not immortal)
+ o = _array(T, n, initialization=initialization)
elif isinstance(T, OpaqueType):
assert n is None
o = _opaque(T, initialization=initialization)
@@ -1887,17 +1860,50 @@
raise TypeError, "malloc for Structs and Arrays only"
if T._gckind != 'gc' and not immortal and flavor.startswith('gc'):
raise TypeError, "gc flavor malloc of a non-GC non-immortal structure"
+ if flavor == "raw" and not immortal and track_allocation:
+ leakfinder.remember_malloc(o, framedepth=2)
solid = immortal or not flavor.startswith('gc') # immortal or non-gc case
return _ptr(Ptr(T), o, solid)
-def free(p, flavor):
+def free(p, flavor, track_allocation=True):
if flavor.startswith('gc'):
raise TypeError, "gc flavor free"
T = typeOf(p)
if not isinstance(T, Ptr) or p._togckind() != 'raw':
raise TypeError, "free(): only for pointers to non-gc containers"
+ if track_allocation:
+ leakfinder.remember_free(p._obj0)
p._obj0._free()
+def _make_scoped_allocator(T):
+ class ScopedAlloc:
+ def __init__(self, n=None, zero=False):
+ if n is None:
+ self.buf = malloc(T, flavor='raw', zero=zero)
+ else:
+ self.buf = malloc(T, n, flavor='raw', zero=zero)
+
+ def __enter__(self):
+ return self.buf
+
+ def __exit__(self, *args):
+ free(self.buf, flavor='raw')
+
+ ScopedAlloc.__name__ = 'ScopedAlloc_%s' % (T,)
+ return ScopedAlloc
+_make_scoped_allocator._annspecialcase_ = 'specialize:memo'
+
+def scoped_alloc(T, n=None, zero=False):
+ """Returns a context manager which handles allocation and
+ deallocation of temporary memory. Use it in a with statement::
+
+ with scoped_alloc(Array(Signed), 1) as array:
+ ...use array...
+ ...it's freed now.
+ """
+ return _make_scoped_allocator(T)(n=n, zero=zero)
+scoped_alloc._annspecialcase_ = 'specialize:arg(0)'
+
def functionptr(TYPE, name, **attrs):
if not isinstance(TYPE, FuncType):
raise TypeError, "functionptr() for FuncTypes only"
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/rbuilder.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/rbuilder.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/rbuilder.py Fri Oct 22 23:09:43 2010
@@ -100,6 +100,10 @@
ll_builder.used = used
@staticmethod
+ def ll_getlength(ll_builder):
+ return ll_builder.used
+
+ @staticmethod
def ll_build(ll_builder):
final_size = ll_builder.used
assert final_size >= 0
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/rclass.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/rclass.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/rclass.py Fri Oct 22 23:09:43 2010
@@ -329,16 +329,33 @@
fields['__class__'] = 'typeptr', get_type_repr(self.rtyper)
else:
# instance attributes
- if llfields is None:
- llfields = []
attrs = self.classdef.attrs.items()
attrs.sort()
+ myllfields = []
for name, attrdef in attrs:
if not attrdef.readonly:
r = self.rtyper.getrepr(attrdef.s_value)
mangled_name = 'inst_' + name
fields[name] = mangled_name, r
- llfields.append((mangled_name, r.lowleveltype))
+ myllfields.append((mangled_name, r.lowleveltype))
+
+ # Sort the instance attributes by decreasing "likely size",
+ # as reported by rffi.sizeof(), to minimize padding holes in C.
+ # Fields of the same size are sorted by name (by attrs.sort()
+ # above) just to minimize randomness.
+ def keysize((_, T)):
+ if T is lltype.Void:
+ return None
+ from pypy.rpython.lltypesystem.rffi import sizeof
+ try:
+ return -sizeof(T)
+ except StandardError:
+ return None
+ myllfields.sort(key = keysize)
+ if llfields is None:
+ llfields = myllfields
+ else:
+ llfields = llfields + myllfields
self.rbase = getinstancerepr(self.rtyper, self.classdef.basedef,
self.gcflavor)
@@ -403,7 +420,7 @@
return cast_pointer(self.lowleveltype, result)
def create_instance(self):
- return malloc(self.object_type, flavor=self.gcflavor)
+ return malloc(self.object_type, flavor=self.gcflavor, immortal=True)
def initialize_prebuilt_data(self, value, classdef, result):
if self.classdef is not None:
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py Fri Oct 22 23:09:43 2010
@@ -609,6 +609,15 @@
return array
str2charp._annenforceargs_ = [strtype]
+ def str2charp_immortal(s):
+ "NOT_RPYTHON"
+ array = lltype.malloc(TYPEP.TO, len(s) + 1, flavor='raw',
+ immortal=True)
+ for i in range(len(s)):
+ array[i] = s[i]
+ array[len(s)] = lastchar
+ return array
+
def free_charp(cp):
lltype.free(cp, flavor='raw')
@@ -646,10 +655,18 @@
"""
Either free a non-moving buffer or keep the original storage alive.
"""
- if rgc.can_move(data):
+ # We cannot rely on rgc.can_move(data) here, because its result
+ # might have changed since get_nonmovingbuffer(). Instead we check
+ # if 'buf' points inside 'data'. This is only possible if we
+ # followed the 2nd case in get_nonmovingbuffer(); in the first case,
+ # 'buf' points to its own raw-malloced memory.
+ data = llstrtype(data)
+ data_start = cast_ptr_to_adr(data) + \
+ offsetof(STRTYPE, 'chars') + itemoffsetof(STRTYPE.chars, 0)
+ followed_2nd_path = (buf == cast(TYPEP, data_start))
+ keepalive_until_here(data)
+ if not followed_2nd_path:
lltype.free(buf, flavor='raw')
- else:
- keepalive_until_here(data)
# int -> (char*, str)
def alloc_buffer(count):
@@ -719,19 +736,19 @@
l = [cp[i] for i in range(size)]
return emptystr.join(l)
- return (str2charp, free_charp, charp2str,
+ return (str2charp, str2charp_immortal, free_charp, charp2str,
get_nonmovingbuffer, free_nonmovingbuffer,
alloc_buffer, str_from_buffer, keep_buffer_alive_until_here,
charp2strn, charpsize2str,
)
-(str2charp, free_charp, charp2str,
+(str2charp, str2charp_immortal, free_charp, charp2str,
get_nonmovingbuffer, free_nonmovingbuffer,
alloc_buffer, str_from_buffer, keep_buffer_alive_until_here,
charp2strn, charpsize2str,
) = make_string_mappings(str)
-(unicode2wcharp, free_wcharp, wcharp2unicode,
+(unicode2wcharp, unicode2wcharp_immortal, free_wcharp, wcharp2unicode,
get_nonmoving_unicodebuffer, free_nonmoving_unicodebuffer,
alloc_unicodebuffer, unicode_from_buffer, keep_unicodebuffer_alive_until_here,
wcharp2unicoden, wcharpsize2unicode,
@@ -918,3 +935,11 @@
"""
return cast(lltype.Signed, getattr(pdst, fieldname))
getintfield._annspecialcase_ = 'specialize:ll_and_arg(1)'
+
+class scoped_str2charp:
+ def __init__(self, value):
+ self.buf = str2charp(value)
+ def __enter__(self):
+ return self.buf
+ def __exit__(self, *args):
+ free_charp(self.buf)
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_ll2ctypes.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_ll2ctypes.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_ll2ctypes.py Fri Oct 22 23:09:43 2010
@@ -765,6 +765,7 @@
assert abs(float(b[1]) - 1.1) < 1E-6
assert isinstance(b[2], rffi.r_singlefloat)
assert abs(float(b[2]) - 2.2) < 1E-6
+ lltype.free(a, flavor='raw')
def test_different_signatures(self):
if sys.platform=='win32':
@@ -879,6 +880,7 @@
qsort(rffi.cast(rffi.VOIDP, a), 5, rffi.sizeof(rffi.INT), compare)
for i in range(5):
assert a[i] == i + 1
+ lltype.free(a, flavor='raw')
def test_array_type_bug(self):
A = lltype.Array(lltype.Signed)
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_llmemory.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_llmemory.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_llmemory.py Fri Oct 22 23:09:43 2010
@@ -324,12 +324,14 @@
p_t = lltype.malloc(T)
assert p_t.s == lltype.nullptr(S)
# raw malloc does not
- p_raw_t = lltype.malloc(T, flavor="raw")
- py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.s")
+ U = lltype.Struct("U", ('x', lltype.Signed))
+ p_raw_t = lltype.malloc(U, flavor="raw")
+ py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.x")
+ lltype.free(p_raw_t, flavor="raw")
# this sort of raw_malloc too
- p_raw_t = cast_adr_to_ptr(raw_malloc(sizeof(T)), lltype.Ptr(T))
- py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.s")
-
+ p_raw_t = cast_adr_to_ptr(raw_malloc(sizeof(U)), lltype.Ptr(U))
+ py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.x")
+
def test_raw_malloc_signed_bunch():
adr = raw_malloc(sizeof(lltype.Signed) * 50)
@@ -601,7 +603,8 @@
a = lltype.malloc(A, flavor='raw')
src = cast_ptr_to_adr(a) + itemoffsetof(A, 0)
raw_memclear(src, sizeof(lltype.Signed) * 0)
-
+ lltype.free(a, flavor="raw")
+
def test_nonneg():
S1 = lltype.GcStruct('S1', ('x', lltype.Float))
A1 = lltype.GcArray(lltype.Float)
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_lltype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_lltype.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_lltype.py Fri Oct 22 23:09:43 2010
@@ -1,7 +1,9 @@
+from __future__ import with_statement
import py
from pypy.rpython.lltypesystem.lltype import *
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
def isweak(p, T):
try:
@@ -804,22 +806,20 @@
class TestTrackAllocation:
- def setup_method(self, func):
- start_tracking_allocations()
-
- def teardown_method(self, func):
- assert not lltype.ALLOCATED, "Memory was not correctly freed"
- stop_tracking_allocations()
+ def test_automatic_tracking(self):
+ # calls to start_tracking_allocations/stop_tracking_allocations
+ # should occur automatically from pypy/conftest.py. Check that.
+ assert leakfinder.TRACK_ALLOCATIONS
def test_track_allocation(self):
"""A malloc'd buffer fills the ALLOCATED dictionary"""
- assert lltype.TRACK_ALLOCATIONS
- assert not lltype.ALLOCATED
+ assert leakfinder.TRACK_ALLOCATIONS
+ assert not leakfinder.ALLOCATED
buf = malloc(Array(Signed), 1, flavor="raw")
- assert len(lltype.ALLOCATED) == 1
- assert lltype.ALLOCATED.keys() == [buf._obj]
+ assert len(leakfinder.ALLOCATED) == 1
+ assert leakfinder.ALLOCATED.keys() == [buf._obj]
free(buf, flavor="raw")
- assert not lltype.ALLOCATED
+ assert not leakfinder.ALLOCATED
def test_str_from_buffer(self):
"""gc-managed memory does not need to be freed"""
@@ -828,16 +828,28 @@
for i in range(size): raw_buf[i] = 'a'
rstr = rffi.str_from_buffer(raw_buf, gc_buf, size, size)
rffi.keep_buffer_alive_until_here(raw_buf, gc_buf)
- assert not lltype.ALLOCATED
+ assert not leakfinder.ALLOCATED
def test_leak_traceback(self):
"""Test info stored for allocated items"""
buf = malloc(Array(Signed), 1, flavor="raw")
- traceback = lltype.ALLOCATED.keys()[0]._traceback
+ traceback = leakfinder.ALLOCATED.values()[0]
lines = traceback.splitlines()
assert 'malloc(' in lines[-1] and 'flavor="raw")' in lines[-1]
- # XXX The traceback should not be too long
+ # The traceback should not be too long
print traceback
free(buf, flavor="raw")
+
+ def test_no_tracking(self):
+ p1 = malloc(Array(Signed), 1, flavor='raw', track_allocation=False)
+ p2 = malloc(Array(Signed), 1, flavor='raw', track_allocation=False)
+ free(p2, flavor='raw', track_allocation=False)
+ # p1 is not freed
+
+ def test_scoped_allocator(self):
+ with scoped_alloc(Array(Signed), 1) as array:
+ array[0] = -42
+ x = array[0]
+ assert x == -42
Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_rffi.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_rffi.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/test/test_rffi.py Fri Oct 22 23:09:43 2010
@@ -9,7 +9,7 @@
from pypy.rpython.lltypesystem.rstr import STR
from pypy.rpython.lltypesystem import lltype
from pypy.tool.udir import udir
-from pypy.rpython.test.test_llinterp import interpret, MallocMismatch
+from pypy.rpython.test.test_llinterp import interpret
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.annotation.annrpython import RPythonAnnotator
from pypy.rpython.rtyper import RPythonTyper
@@ -787,3 +787,10 @@
mixann.getgraph(f2, [], s_None)
mixann.finish()
+def test_force_cast_unichar():
+ x = cast(lltype.UniChar, -1)
+ assert isinstance(x, unicode)
+ if sys.maxunicode == 65535:
+ assert cast(LONG, x) == 65535
+ else:
+ assert cast(LONG, cast(INT, x)) == -1
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/minimarkpage.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gc/minimarkpage.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gc/minimarkpage.py Fri Oct 22 23:09:43 2010
@@ -100,11 +100,14 @@
# allocation of the given size.
length = small_request_threshold / WORD + 1
self.page_for_size = lltype.malloc(rffi.CArray(PAGE_PTR), length,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
self.full_page_for_size = lltype.malloc(rffi.CArray(PAGE_PTR), length,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
self.nblocks_for_size = lltype.malloc(rffi.CArray(lltype.Signed),
- length, flavor='raw')
+ length, flavor='raw',
+ immortal=True)
self.hdrsize = llmemory.raw_malloc_usage(llmemory.sizeof(PAGE_HEADER))
assert page_size > self.hdrsize
self.nblocks_for_size[0] = 0 # unused
@@ -114,11 +117,13 @@
self.max_pages_per_arena = arena_size // page_size
self.arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
self.max_pages_per_arena,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
# this is used in mass_free() only
self.old_arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
self.max_pages_per_arena,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
#
# the arena currently consumed; it must have at least one page
# available, or be NULL. The arena object that we point to is
@@ -281,7 +286,7 @@
npages = (arena_end - firstpage) // self.page_size
#
# Allocate an ARENA object and initialize it
- arena = lltype.malloc(ARENA, flavor='raw')
+ arena = lltype.malloc(ARENA, flavor='raw', track_allocation=False)
arena.base = arena_base
arena.nfreepages = 0 # they are all uninitialized pages
arena.totalpages = npages
@@ -332,7 +337,7 @@
#
# The whole arena is empty. Free it.
llarena.arena_free(arena.base)
- lltype.free(arena, flavor='raw')
+ lltype.free(arena, flavor='raw', track_allocation=False)
#
else:
# Insert 'arena' in the correct arenas_lists[n]
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gctransform/asmgcroot.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gctransform/asmgcroot.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gctransform/asmgcroot.py Fri Oct 22 23:09:43 2010
@@ -75,7 +75,8 @@
key = (TYPE, num)
if key not in sradict:
CONTAINER = lltype.FixedSizeArray(TYPE, 1)
- p = lltype.malloc(CONTAINER, flavor='raw', zero=True)
+ p = lltype.malloc(CONTAINER, flavor='raw', zero=True,
+ immortal=True)
sradict[key] = Constant(p, lltype.Ptr(CONTAINER))
sra.append(sradict[key])
#
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gctransform/transform.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gctransform/transform.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gctransform/transform.py Fri Oct 22 23:09:43 2010
@@ -430,7 +430,8 @@
return self.parenttransformer.gct_malloc_varsize(hop)
def gct_free(self, hop):
- flavor = hop.spaceop.args[1].value
+ flags = hop.spaceop.args[1].value
+ flavor = flags['flavor']
assert flavor == 'raw'
return self.parenttransformer.gct_free(hop)
@@ -532,6 +533,8 @@
resulttype=llmemory.Address)
if flags.get('zero'):
hop.genop("raw_memclear", [v_raw, c_size])
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_start", [v_raw])
return v_raw
def gct_fv_stack_malloc(self, hop, flags, TYPE, c_size):
@@ -602,15 +605,20 @@
[self.raw_malloc_varsize_ptr, v_length,
c_const_size, c_item_size, c_offset_to_length],
resulttype=llmemory.Address)
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_start", [v_raw])
return v_raw
def gct_free(self, hop):
op = hop.spaceop
- flavor = op.args[1].value
+ flags = op.args[1].value
+ flavor = flags['flavor']
v = op.args[0]
assert flavor != 'cpy', "cannot free CPython objects directly"
if flavor == 'raw':
v = hop.genop("cast_ptr_to_adr", [v], resulttype=llmemory.Address)
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_stop", [v])
hop.genop('raw_free', [v])
else:
assert False, "%s has no support for free with flavor %r" % (self, flavor)
Modified: pypy/branch/fast-forward/pypy/rpython/memory/gcwrapper.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/gcwrapper.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/gcwrapper.py Fri Oct 22 23:09:43 2010
@@ -42,7 +42,8 @@
#
# Interface for the llinterp
#
- def malloc(self, TYPE, n=None, flavor='gc', zero=False):
+ def malloc(self, TYPE, n=None, flavor='gc', zero=False,
+ track_allocation=True):
if flavor == 'gc':
typeid = self.get_type_id(TYPE)
addr = self.gc.malloc(typeid, n, zero=zero)
@@ -51,7 +52,8 @@
gctypelayout.zero_gc_pointers(result)
return result
else:
- return lltype.malloc(TYPE, n, flavor=flavor, zero=zero)
+ return lltype.malloc(TYPE, n, flavor=flavor, zero=zero,
+ track_allocation=track_allocation)
def malloc_nonmovable(self, TYPE, n=None, zero=False):
typeid = self.get_type_id(TYPE)
@@ -69,9 +71,10 @@
return self.gc.shrink_array(addr, smallersize)
return False
- def free(self, TYPE, flavor='gc'):
+ def free(self, TYPE, flavor='gc', track_allocation=True):
assert flavor != 'gc'
- return lltype.free(TYPE, flavor=flavor)
+ return lltype.free(TYPE, flavor=flavor,
+ track_allocation=track_allocation)
def setfield(self, obj, fieldname, fieldvalue):
STRUCT = lltype.typeOf(obj).TO
Modified: pypy/branch/fast-forward/pypy/rpython/memory/support.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/support.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/support.py Fri Oct 22 23:09:43 2010
@@ -30,7 +30,8 @@
# we zero-initialize the chunks to make the translation
# backends happy, but we don't need to do it at run-time.
zero = not we_are_translated()
- return lltype.malloc(CHUNK, flavor="raw", zero=zero)
+ return lltype.malloc(CHUNK, flavor="raw", zero=zero,
+ track_allocation=False)
result = self.free_list
self.free_list = result.next
@@ -44,7 +45,7 @@
# Don't cache the old chunks but free them immediately.
# Helps debugging, and avoids that old chunks full of
# addresses left behind by a test end up in genc...
- lltype.free(chunk, flavor="raw")
+ lltype.free(chunk, flavor="raw", track_allocation=False)
unused_chunks = FreeList()
cache[chunk_size] = unused_chunks, null_chunk
Modified: pypy/branch/fast-forward/pypy/rpython/memory/test/test_gc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/memory/test/test_gc.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/memory/test/test_gc.py Fri Oct 22 23:09:43 2010
@@ -368,6 +368,14 @@
res = self.interpret(f, [4, 42])
assert res == 12
+ def test_print_leak(self):
+ def f(n):
+ for i in range(n):
+ print i
+ return 42
+ res = self.interpret(f, [10])
+ assert res == 42
+
def test_weakref_across_minor_collection(self):
import weakref
class A:
Modified: pypy/branch/fast-forward/pypy/rpython/module/ll_time.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/module/ll_time.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/module/ll_time.py Fri Oct 22 23:09:43 2010
@@ -108,7 +108,7 @@
errcode = -1
if self.GETTIMEOFDAY_NO_TZ:
- errcode = g_gettimeofday(t)
+ errcode = c_gettimeofday(t)
else:
errcode = c_gettimeofday(t, void)
Modified: pypy/branch/fast-forward/pypy/rpython/rbuilder.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/rbuilder.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/rbuilder.py Fri Oct 22 23:09:43 2010
@@ -36,8 +36,12 @@
hop.exception_cannot_occur()
return hop.gendirectcall(self.ll_append_multiple_char, *vlist)
+ def rtype_method_getlength(self, hop):
+ vlist = hop.inputargs(self)
+ hop.exception_cannot_occur()
+ return hop.gendirectcall(self.ll_getlength, *vlist)
+
def rtype_method_build(self, hop):
vlist = hop.inputargs(self)
hop.exception_cannot_occur()
return hop.gendirectcall(self.ll_build, *vlist)
-
Modified: pypy/branch/fast-forward/pypy/rpython/rbuiltin.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/rbuiltin.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/rbuiltin.py Fri Oct 22 23:09:43 2010
@@ -345,17 +345,22 @@
BUILTIN_TYPER[object.__init__] = rtype_object__init__
# annotation of low-level types
-def rtype_malloc(hop, i_flavor=None, i_zero=None):
+def rtype_malloc(hop, i_flavor=None, i_zero=None, i_track_allocation=None):
assert hop.args_s[0].is_constant()
vlist = [hop.inputarg(lltype.Void, arg=0)]
opname = 'malloc'
- v_flavor, v_zero = parse_kwds(hop, (i_flavor, lltype.Void), (i_zero, None))
+ v_flavor, v_zero, v_track_allocation = parse_kwds(hop,
+ (i_flavor, lltype.Void),
+ (i_zero, None),
+ (i_track_allocation, None))
flags = {'flavor': 'gc'}
if v_flavor is not None:
flags['flavor'] = v_flavor.value
if i_zero is not None:
flags['zero'] = v_zero.value
+ if i_track_allocation is not None:
+ flags['track_allocation'] = v_track_allocation.value
vlist.append(hop.inputconst(lltype.Void, flags))
if hop.nb_args == 2:
@@ -366,10 +371,19 @@
hop.exception_is_here()
return hop.genop(opname, vlist, resulttype = hop.r_result.lowleveltype)
-def rtype_free(hop, i_flavor):
- assert i_flavor == 1
+def rtype_free(hop, i_flavor, i_track_allocation=None):
+ vlist = [hop.inputarg(hop.args_r[0], arg=0)]
+ v_flavor, v_track_allocation = parse_kwds(hop,
+ (i_flavor, lltype.Void),
+ (i_track_allocation, None))
+ #
+ assert v_flavor is not None and v_flavor.value == 'raw'
+ flags = {'flavor': 'raw'}
+ if i_track_allocation is not None:
+ flags['track_allocation'] = v_track_allocation.value
+ vlist.append(hop.inputconst(lltype.Void, flags))
+ #
hop.exception_cannot_occur()
- vlist = hop.inputargs(hop.args_r[0], lltype.Void)
hop.genop('free', vlist)
def rtype_const_result(hop):
@@ -584,8 +598,9 @@
vinst, = hop.inputargs(hop.args_r[0])
flavor = hop.args_r[0].gcflavor
assert flavor != 'gc'
- cflavor = hop.inputconst(lltype.Void, flavor)
- return hop.genop('free', [vinst, cflavor])
+ flags = {'flavor': flavor}
+ cflags = hop.inputconst(lltype.Void, flags)
+ return hop.genop('free', [vinst, cflags])
BUILTIN_TYPER[objectmodel.free_non_gc_object] = rtype_free_non_gc_object
Modified: pypy/branch/fast-forward/pypy/rpython/rpbc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/rpbc.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/rpbc.py Fri Oct 22 23:09:43 2010
@@ -256,6 +256,8 @@
def convert_const(self, value):
if isinstance(value, types.MethodType) and value.im_self is None:
value = value.im_func # unbound method -> bare function
+ elif isinstance(value, staticmethod):
+ value = value.__get__(42) # hackish, get the function wrapped by staticmethod
if self.lowleveltype is Void:
return None
if value is None:
Modified: pypy/branch/fast-forward/pypy/rpython/rtyper.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/rtyper.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/rtyper.py Fri Oct 22 23:09:43 2010
@@ -421,7 +421,7 @@
assert noexclink.exitcase is None
if pos == "removed":
# the exception cannot actually occur at all.
- # See for example rspecialcase.rtype_call_specialcase().
+ # This is set by calling exception_cannot_occur().
# We just remove all exception links.
block.exitswitch = None
block.exits = block.exits[:1]
@@ -1019,7 +1019,7 @@
from pypy.rpython import rint, rbool, rfloat
from pypy.rpython import rrange
from pypy.rpython import rstr, rdict, rlist
-from pypy.rpython import rclass, rbuiltin, rpbc, rspecialcase
+from pypy.rpython import rclass, rbuiltin, rpbc
from pypy.rpython import rexternalobj
from pypy.rpython import rptr
from pypy.rpython import rgeneric
Modified: pypy/branch/fast-forward/pypy/rpython/test/test_llinterp.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/test/test_llinterp.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/test/test_llinterp.py Fri Oct 22 23:09:43 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
import py
import sys
from pypy.rpython.lltypesystem.lltype import typeOf, pyobjectptr, Ptr,\
@@ -5,6 +6,7 @@
from pypy.rpython.lltypesystem.lloperation import llop
from pypy.rpython.llinterp import LLInterpreter, LLException, log
from pypy.rpython.rmodel import inputconst
+from pypy.rpython.annlowlevel import hlstr
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.rint import signed_repr
from pypy.rpython.lltypesystem import rstr, lltype
@@ -12,13 +14,11 @@
from pypy.annotation.model import lltype_to_annotation
from pypy.rlib.rarithmetic import r_uint, ovfcheck
from pypy.rpython.ootypesystem import ootype
+from pypy.tool import leakfinder
from pypy import conftest
# switch on logging of interp to show more info on failing tests
-class MallocMismatch(Exception):
- pass
-
def setup_module(mod):
mod.logstate = py.log._getstate()
py.log.setconsumer("llinterp", py.log.STDOUT)
@@ -72,7 +72,7 @@
def get_interpreter(func, values, view='auto', viewbefore='auto', policy=None,
someobjects=False, type_system="lltype", backendopt=False,
- config=None, malloc_check=True, **extraconfigopts):
+ config=None, **extraconfigopts):
extra_key = [(key, value) for key, value in extraconfigopts.iteritems()]
extra_key.sort()
extra_key = tuple(extra_key)
@@ -97,7 +97,7 @@
viewbefore, policy, type_system=type_system,
backendopt=backendopt, config=config,
**extraconfigopts)
- interp = LLInterpreter(typer, malloc_check=malloc_check)
+ interp = LLInterpreter(typer)
_tcache[key] = (t, interp, graph)
# keep the cache small
_lastinterpreted.append(key)
@@ -115,10 +115,17 @@
interp, graph = get_interpreter(func, values, view, viewbefore, policy,
someobjects, type_system=type_system,
backendopt=backendopt, config=config,
- malloc_check=malloc_check, **kwargs)
- result = interp.eval_graph(graph, values)
- if malloc_check and interp.mallocs:
- raise MallocMismatch(interp.mallocs)
+ **kwargs)
+ if not malloc_check:
+ result = interp.eval_graph(graph, values)
+ else:
+ prev = leakfinder.start_tracking_allocations()
+ try:
+ result = interp.eval_graph(graph, values)
+ finally:
+ leaks = leakfinder.stop_tracking_allocations(False, prev)
+ if leaks:
+ raise leakfinder.MallocMismatch(leaks)
return result
def interpret_raises(exc, func, values, view='auto', viewbefore='auto',
@@ -418,6 +425,7 @@
assert result
def test_stack_malloc():
+ py.test.skip("stack-flavored mallocs no longer supported")
class A(object):
pass
def f():
@@ -430,6 +438,7 @@
assert result == 1
def test_invalid_stack_access():
+ py.test.skip("stack-flavored mallocs no longer supported")
class A(object):
pass
globala = A()
@@ -605,7 +614,7 @@
if x:
free(t, flavor='raw')
interpret(f, [1])
- py.test.raises(MallocMismatch, "interpret(f, [0])")
+ py.test.raises(leakfinder.MallocMismatch, "interpret(f, [0])")
def f():
t1 = malloc(T, flavor='raw')
@@ -615,3 +624,37 @@
interpret(f, [])
+def test_context_manager():
+ state = []
+ class C:
+ def __enter__(self):
+ state.append('acquire')
+ return self
+ def __exit__(self, *args):
+ if args[1] is not None:
+ state.append('raised')
+ state.append('release')
+ def f():
+ try:
+ with C() as c:
+ state.append('use')
+ raise ValueError
+ except ValueError:
+ pass
+ return ', '.join(state)
+ res = interpret(f, [])
+ assert hlstr(res) == 'acquire, use, raised, release'
+
+
+def test_scoped_allocator():
+ from pypy.rpython.lltypesystem.lltype import scoped_alloc, Array, Signed
+ T = Array(Signed)
+
+ def f():
+ x = 0
+ with scoped_alloc(T, 1) as array:
+ array[0] = -42
+ x = array[0]
+ assert x == -42
+
+ res = interpret(f, [])
Modified: pypy/branch/fast-forward/pypy/rpython/test/test_nongc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/test/test_nongc.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/test/test_nongc.py Fri Oct 22 23:09:43 2010
@@ -79,7 +79,7 @@
py.test.raises(TypeError,rtyper.specialize) # results in an invalid cast
def test_isinstance():
- class A:
+ class A(object):
_alloc_flavor_ = "raw"
class B(A):
pass
@@ -95,7 +95,24 @@
o = B()
else:
o = C()
- return 100*isinstance(o, A)+10*isinstance(o, B)+1*isinstance(o ,C)
+ res = 100*isinstance(o, A) + 10*isinstance(o, B) + 1*isinstance(o, C)
+ if i == 0:
+ pass
+ elif i == 1:
+ assert isinstance(o, A)
+ free_non_gc_object(o)
+ elif i == 2:
+ assert isinstance(o, B)
+ free_non_gc_object(o)
+ else:
+ assert isinstance(o, C)
+ free_non_gc_object(o)
+ return res
+
+ assert f(1) == 100
+ assert f(2) == 110
+ assert f(3) == 111
+ assert f(0) == 0
a = RPythonAnnotator()
#does not raise:
@@ -131,10 +148,14 @@
d = b
elif i == 2:
e = c
- return (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
+ res = (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
0x0008*(a is e) | 0x0010*(b is c) | 0x0020*(b is d) |
0x0040*(b is e) | 0x0080*(c is d) | 0x0100*(c is e) |
0x0200*(d is e))
+ free_non_gc_object(a)
+ free_non_gc_object(b)
+ free_non_gc_object(c)
+ return res
a = RPythonAnnotator()
#does not raise:
s = a.build_types(f, [int])
@@ -169,10 +190,13 @@
d = b
elif i == 2:
e = c
- return (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
+ res = (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
0x0008*(a is e) | 0x0010*(b is c) | 0x0020*(b is d) |
0x0040*(b is e) | 0x0080*(c is d) | 0x0100*(c is e) |
0x0200*(d is e))
+ free_non_gc_object(a)
+ free_non_gc_object(b)
+ return res
a = RPythonAnnotator()
#does not raise:
s = a.build_types(f, [int])
Modified: pypy/branch/fast-forward/pypy/rpython/test/test_rbuilder.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/test/test_rbuilder.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/test/test_rbuilder.py Fri Oct 22 23:09:43 2010
@@ -1,4 +1,4 @@
-
+import py
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.rpython.lltypesystem.rbuilder import *
from pypy.rpython.annlowlevel import llstr, hlstr
@@ -55,8 +55,29 @@
assert res == 'aabcabcdefbuuuu'
assert isinstance(res, unicode)
+ def test_string_getlength(self):
+ def func():
+ s = StringBuilder()
+ s.append("a")
+ s.append("abc")
+ return s.getlength()
+ res = self.interpret(func, [])
+ assert res == 4
+
+ def test_unicode_getlength(self):
+ def func():
+ s = UnicodeBuilder()
+ s.append(u"a")
+ s.append(u"abc")
+ return s.getlength()
+ res = self.interpret(func, [])
+ assert res == 4
+
class TestLLtype(BaseTestStringBuilder, LLRtypeMixin):
pass
class TestOOtype(BaseTestStringBuilder, OORtypeMixin):
- pass
+ def test_string_getlength(self):
+ py.test.skip("getlength(): not implemented on ootype")
+ def test_unicode_getlength(self):
+ py.test.skip("getlength(): not implemented on ootype")
Modified: pypy/branch/fast-forward/pypy/rpython/test/test_rclass.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/test/test_rclass.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/test/test_rclass.py Fri Oct 22 23:09:43 2010
@@ -3,7 +3,7 @@
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.lltypesystem.lltype import *
from pypy.rpython.ootypesystem import ootype
-from pypy.rlib.rarithmetic import intmask
+from pypy.rlib.rarithmetic import intmask, r_longlong
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.objspace.flow.model import summary
@@ -319,6 +319,17 @@
res = self.interpret(f, [])
assert res == 42
+ def test_staticmethod2(self):
+ class A(object):
+ f = staticmethod(lambda x, y: x*y)
+ class B(A):
+ f = staticmethod(lambda x, y: x+y)
+ def f():
+ b = B()
+ return b.f(6, 7)
+ res = self.interpret(f, [])
+ assert res == 13
+
def test_is(self):
class A: pass
class B(A): pass
@@ -1001,6 +1012,40 @@
res = self.interpret(f, [5])
assert res == 0
+ def test_order_of_fields(self):
+ class A(object):
+ pass
+ def f(n):
+ a = A()
+ a.as_int = n
+ a.as_char = chr(n)
+ a.as_unichar = unichr(n)
+ a.as_double = n + 0.5
+ a.as_bool = bool(n)
+ a.as_void = None
+ a.as_longlong = r_longlong(n)
+ a.as_reference = A()
+ return a
+
+ res = self.interpret(f, [5])
+ names = list(typeOf(res).TO._names)
+ i = names.index('inst_as_int')
+ c = names.index('inst_as_char')
+ u = names.index('inst_as_unichar')
+ d = names.index('inst_as_double')
+ b = names.index('inst_as_bool')
+ v = names.index('inst_as_void')
+ l = names.index('inst_as_longlong')
+ r = names.index('inst_as_reference')
+ assert v == 1 # void fields are first
+ assert sorted([c, b]) == [7, 8]
+ if sys.maxint == 2147483647:
+ assert sorted([u, i, r]) == [4, 5, 6] # 32-bit types
+ assert sorted([d, l]) == [2, 3] # 64-bit types
+ else:
+ assert sorted([u]) == [6] # 32-bit types
+ assert sorted([i, r, d, l]) == [2, 3, 4, 5] # 64-bit types
+
class TestOOtype(BaseTestRclass, OORtypeMixin):
Modified: pypy/branch/fast-forward/pypy/rpython/test/test_rptr.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rpython/test/test_rptr.py (original)
+++ pypy/branch/fast-forward/pypy/rpython/test/test_rptr.py Fri Oct 22 23:09:43 2010
@@ -212,10 +212,31 @@
S = Struct('S', ('x', Signed))
def fn(n):
- p = malloc(S, flavor='whatever')
+ p = malloc(S, flavor='raw')
p.x = n
result = p.x
- free(p, flavor='whatever')
+ free(p, flavor='raw')
+ return n
+
+ res = interpret(fn, [23])
+ assert res == 23
+
+ S = Struct('S', ('x', Signed))
+ def fn(n):
+ p = malloc(S, flavor='raw', track_allocation=False)
+ p.x = n
+ result = p.x
+ return n
+
+ res = interpret(fn, [23])
+ assert res == 23
+
+ S = Struct('S', ('x', Signed))
+ def fn(n):
+ p = malloc(S, flavor='raw', track_allocation=False)
+ p.x = n
+ result = p.x
+ free(p, flavor='raw', track_allocation=False)
return n
res = interpret(fn, [23])
Modified: pypy/branch/fast-forward/pypy/translator/c/funcgen.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/funcgen.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/funcgen.py Fri Oct 22 23:09:43 2010
@@ -427,7 +427,7 @@
r = self.expr(op.result)
return 'OP_CALL_ARGS((%s), %s);' % (', '.join(args), r)
- def generic_call(self, FUNC, fnexpr, args_v, v_result):
+ def generic_call(self, FUNC, fnexpr, args_v, v_result, targets=None):
args = []
assert len(args_v) == len(FUNC.TO.ARGS)
for v, ARGTYPE in zip(args_v, FUNC.TO.ARGS):
@@ -444,17 +444,26 @@
# skip assignment of 'void' return value
r = self.expr(v_result)
line = '%s = %s' % (r, line)
+ if targets:
+ for graph in targets:
+ if getattr(graph, 'inhibit_tail_call', False):
+ line += '\nPYPY_INHIBIT_TAIL_CALL();'
+ break
return line
def OP_DIRECT_CALL(self, op):
fn = op.args[0]
+ try:
+ targets = [fn.value._obj.graph]
+ except AttributeError:
+ targets = None
return self.generic_call(fn.concretetype, self.expr(fn),
- op.args[1:], op.result)
+ op.args[1:], op.result, targets)
def OP_INDIRECT_CALL(self, op):
fn = op.args[0]
return self.generic_call(fn.concretetype, self.expr(fn),
- op.args[1:-1], op.result)
+ op.args[1:-1], op.result, op.args[-1].value)
def OP_ADR_CALL(self, op):
ARGTYPES = [v.concretetype for v in op.args[1:]]
Modified: pypy/branch/fast-forward/pypy/translator/c/gcc/instruction.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/gcc/instruction.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/gcc/instruction.py Fri Oct 22 23:09:43 2010
@@ -82,6 +82,11 @@
def all_sources_of(self, localvar):
return [localvar]
+class InsnCondJump(Insn): # only for debugging; not used internally
+ _args_ = ['label']
+ def __init__(self, label):
+ self.label = label
+
class Label(Insn):
_args_ = ['label', 'lineno']
def __init__(self, label, lineno):
@@ -170,9 +175,12 @@
self.delta = -7 # use an odd value as marker
class InsnStop(Insn):
- pass
+ _args_ = ['reason']
+ def __init__(self, reason='?'):
+ self.reason = reason
class InsnRet(InsnStop):
+ _args_ = []
framesize = 0
def __init__(self, registers):
self.registers = registers
Modified: pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf/track5.s
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf/track5.s (original)
+++ pypy/branch/fast-forward/pypy/translator/c/gcc/test/elf/track5.s Fri Oct 22 23:09:43 2010
@@ -44,7 +44,7 @@
addl %eax, %ebx
jmp .L1221
.L1227:
- call RPyAbort
+ ;;call RPyAbort
cmpl 12(%esi), %ebx
jb .L1229
addl $20, %esp
Modified: pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py Fri Oct 22 23:09:43 2010
@@ -6,7 +6,7 @@
from pypy.translator.c.gcc.instruction import InsnFunctionStart, InsnStop
from pypy.translator.c.gcc.instruction import InsnSetLocal, InsnCopyLocal
from pypy.translator.c.gcc.instruction import InsnPrologue, InsnEpilogue
-from pypy.translator.c.gcc.instruction import InsnGCROOT
+from pypy.translator.c.gcc.instruction import InsnGCROOT, InsnCondJump
from pypy.translator.c.gcc.instruction import InsnStackAdjust
from pypy.translator.c.gcc.instruction import InsnCannotFollowEsp
from pypy.translator.c.gcc.instruction import LocalVar, somenewvalue
@@ -46,6 +46,7 @@
self.findlabels()
self.parse_instructions()
try:
+ self.trim_unreachable_instructions()
self.find_noncollecting_calls()
if not self.list_collecting_call_insns():
return []
@@ -122,19 +123,36 @@
assert label not in self.labels, "duplicate label: %s" % label
self.labels[label] = Label(label, lineno)
+ def trim_unreachable_instructions(self):
+ reached = set([self.insns[0]])
+ prevlen = 0
+ while len(reached) > prevlen:
+ prevlen = len(reached)
+ for insn in self.insns:
+ if insn not in reached:
+ for previnsn in insn.previous_insns:
+ if previnsn in reached:
+ # this instruction is reachable too
+ reached.add(insn)
+ break
+ # now kill all unreachable instructions
+ i = 0
+ while i < len(self.insns):
+ if self.insns[i] in reached:
+ i += 1
+ else:
+ del self.insns[i]
+
def find_noncollecting_calls(self):
- cannot_collect = self.CANNOT_COLLECT.copy()
+ cannot_collect = {}
for line in self.lines:
match = self.r_gcnocollect_marker.search(line)
if match:
name = match.group(1)
cannot_collect[name] = True
#
- if self.format in ('darwin', 'mingw32', 'msvc'):
- self.cannot_collect = dict.fromkeys(
- ['_' + name for name in cannot_collect])
- else:
- self.cannot_collect = cannot_collect
+ self.cannot_collect = dict.fromkeys(
+ [self.function_names_prefix + name for name in cannot_collect])
def append_instruction(self, insn):
# Add the instruction to the list, and link it to the previous one.
@@ -410,7 +428,8 @@
return result
# ____________________________________________________________
- CANNOT_COLLECT = { # some of the most used functions that cannot collect
+ BASE_FUNCTIONS_NOT_RETURNING = {
+ 'abort': None,
'pypy_debug_catch_fatal_exception': None,
'RPyAbort': None,
'RPyAssertFailed': None,
@@ -644,7 +663,7 @@
if label != '0':
self.register_jump_to(label)
tablelin += 1
- return InsnStop()
+ return InsnStop("jump table")
if self.r_unaryinsn_star.match(line):
# that looks like an indirect tail-call.
# tail-calls are equivalent to RET for us
@@ -658,7 +677,7 @@
assert not target.startswith('.')
# tail-calls are equivalent to RET for us
return InsnRet(self.CALLEE_SAVE_REGISTERS)
- return InsnStop()
+ return InsnStop("jump")
def register_jump_to(self, label):
if not isinstance(self.insns[-1], InsnStop):
@@ -682,7 +701,7 @@
else:
label = match.group(1)
self.register_jump_to(label)
- return []
+ return [InsnCondJump(label)]
visit_jmpl = visit_jmp
visit_je = conditional_jump
@@ -754,7 +773,7 @@
target, = sources
if target in self.FUNCTIONS_NOT_RETURNING:
- return [InsnStop(), InsnCannotFollowEsp()]
+ return [InsnStop(target)]
if self.format == 'mingw32' and target == '__alloca':
# in functions with large stack requirements, windows
# needs a call to _alloca(), to turn reserved pages
@@ -885,7 +904,7 @@
# statically known pointer to a register
# %eax -> %rax
- new_line = re.sub(r"%e(ax|bx|cx|dx|di|si)$", r"%r\1", line)
+ new_line = re.sub(r"%e(ax|bx|cx|dx|di|si|bp)$", r"%r\1", line)
# %r10d -> %r10
new_line = re.sub(r"%r(\d+)d$", r"%r\1", new_line)
return func(self, new_line)
@@ -951,6 +970,7 @@
class ElfFunctionGcRootTracker32(FunctionGcRootTracker32):
format = 'elf'
+ function_names_prefix = ''
ESP = '%esp'
EBP = '%ebp'
@@ -984,13 +1004,14 @@
r_bottom_marker = re.compile(r"\t/[*] GC_STACK_BOTTOM [*]/")
FUNCTIONS_NOT_RETURNING = {
- 'abort': None,
'_exit': None,
'__assert_fail': None,
'___assert_rtn': None,
'L___assert_rtn$stub': None,
'L___eprintf$stub': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING[_name] = None
def __init__(self, lines, filetag=0):
match = self.r_functionstart.match(lines[0])
@@ -1010,6 +1031,8 @@
class ElfFunctionGcRootTracker64(FunctionGcRootTracker64):
format = 'elf64'
+ function_names_prefix = ''
+
ESP = '%rsp'
EBP = '%rbp'
EAX = '%rax'
@@ -1042,13 +1065,14 @@
r_bottom_marker = re.compile(r"\t/[*] GC_STACK_BOTTOM [*]/")
FUNCTIONS_NOT_RETURNING = {
- 'abort': None,
'_exit': None,
'__assert_fail': None,
'___assert_rtn': None,
'L___assert_rtn$stub': None,
'L___eprintf$stub': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING[_name] = None
def __init__(self, lines, filetag=0):
match = self.r_functionstart.match(lines[0])
@@ -1068,6 +1092,7 @@
class DarwinFunctionGcRootTracker(ElfFunctionGcRootTracker32):
format = 'darwin'
+ function_names_prefix = '_'
r_functionstart = re.compile(r"_(\w+):\s*$")
OFFSET_LABELS = 0
@@ -1079,15 +1104,19 @@
class Mingw32FunctionGcRootTracker(DarwinFunctionGcRootTracker):
format = 'mingw32'
+ function_names_prefix = '_'
FUNCTIONS_NOT_RETURNING = {
- '_abort': None,
'_exit': None,
'__assert': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING['_' + _name] = None
class MsvcFunctionGcRootTracker(FunctionGcRootTracker32):
format = 'msvc'
+ function_names_prefix = '_'
+
ESP = 'esp'
EBP = 'ebp'
EAX = 'eax'
@@ -1127,7 +1156,6 @@
r_bottom_marker = re.compile(r"; .+\tpypy_asm_stack_bottom\(\);")
FUNCTIONS_NOT_RETURNING = {
- '_abort': None,
'__exit': None,
'__assert': None,
'__wassert': None,
@@ -1136,6 +1164,8 @@
'DWORD PTR __imp__abort': None,
'DWORD PTR __imp___wassert': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING['_' + _name] = None
@classmethod
def init_regexp(cls):
@@ -1543,9 +1573,7 @@
assert self.seen_main
def _globalname(name, disp=""):
- if self.format in ('darwin', 'mingw32', 'msvc'):
- name = '_' + name
- return name
+ return tracker_cls.function_names_prefix + name
def _variant(**kwargs):
txt = kwargs[self.format]
Modified: pypy/branch/fast-forward/pypy/translator/c/src/g_include.h
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/src/g_include.h (original)
+++ pypy/branch/fast-forward/pypy/translator/c/src/g_include.h Fri Oct 22 23:09:43 2010
@@ -53,6 +53,7 @@
# include "src/rtyper.h"
# include "src/debug_print.h"
# include "src/debug_traceback.h"
+# include "src/debug_alloc.h"
#ifndef AVR
# include "src/ll_os.h"
# include "src/ll_strtod.h"
Modified: pypy/branch/fast-forward/pypy/translator/c/src/main.h
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/src/main.h (original)
+++ pypy/branch/fast-forward/pypy/translator/c/src/main.h Fri Oct 22 23:09:43 2010
@@ -53,10 +53,16 @@
}
exitcode = STANDALONE_ENTRY_POINT(list);
+
+#ifdef RPY_ASSERT
+ pypy_debug_alloc_results();
+#endif
+
if (RPyExceptionOccurred()) {
/* print the RPython traceback */
pypy_debug_catch_fatal_exception();
}
+
return exitcode;
memory_out:
Modified: pypy/branch/fast-forward/pypy/translator/c/src/signals.h
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/src/signals.h (original)
+++ pypy/branch/fast-forward/pypy/translator/c/src/signals.h Fri Oct 22 23:09:43 2010
@@ -6,20 +6,6 @@
#include
-#ifndef LONG_MAX
-#if SIZEOF_LONG == 4
-#define LONG_MAX 0X7FFFFFFFL
-#elif SIZEOF_LONG == 8
-#define LONG_MAX 0X7FFFFFFFFFFFFFFFL
-#else
-#error "could not set LONG_MAX in pyport.h"
-#endif
-#endif
-
-#ifndef LONG_MIN
-#define LONG_MIN (-LONG_MAX-1)
-#endif
-
#include
#ifdef MS_WINDOWS
@@ -28,10 +14,6 @@
#include
-#ifndef SIG_ERR
-#define SIG_ERR ((PyOS_sighandler_t)(-1))
-#endif
-
#if defined(PYOS_OS2) && !defined(PYCC_GCC)
#define NSIG 12
#include
@@ -65,11 +47,11 @@
/* utility to poll for signals that arrived */
int pypysig_poll(void); /* => signum or -1 */
-/* When a signal is received, the high bit of pypysig_occurred is set.
- After all signals are processed by pypysig_poll(), the high bit is
+/* When a signal is received, the bit 30 of pypysig_occurred is set.
+ After all signals are processed by pypysig_poll(), the bit 30 is
cleared again. The variable is exposed and RPython code is free to
use the other bits in any way. */
-#define PENDING_SIGNAL_BIT (LONG_MIN) /* high bit */
+#define PENDING_SIGNAL_BIT (1 << 30)
/* This is a struct for the JIT. See interp_signal.py. */
struct pypysig_long_struct {
long value;
Modified: pypy/branch/fast-forward/pypy/translator/c/src/stack.h
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/src/stack.h (original)
+++ pypy/branch/fast-forward/pypy/translator/c/src/stack.h Fri Oct 22 23:09:43 2010
@@ -33,6 +33,12 @@
&& LL_stack_too_big_slowpath());
}
+#ifdef __GNUC__
+# define PYPY_INHIBIT_TAIL_CALL() asm("/* inhibit_tail_call */")
+#else
+# define PYPY_INHIBIT_TAIL_CALL() /* add hints for other compilers here */
+#endif
+
#ifndef PYPY_NOT_MAIN_FILE
#include
Modified: pypy/branch/fast-forward/pypy/translator/c/test/test_genc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/test/test_genc.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/test/test_genc.py Fri Oct 22 23:09:43 2010
@@ -426,6 +426,7 @@
if py.test.config.option.view:
t.view()
assert ' BarStruct ' in t.driver.cbuilder.c_source_filename.read()
+ free(foo, flavor="raw")
def test_recursive_llhelper():
from pypy.rpython.annlowlevel import llhelper
@@ -473,7 +474,27 @@
return f(s)
a_f = A(f, "f")
a_g = A(g, "g")
- t = lltype.malloc(STRUCT, flavor="raw")
+ t = lltype.malloc(STRUCT, flavor="raw", immortal=True)
t.bar = llhelper(FTPTR, a_f.make_func())
fn = compile(chooser, [bool])
assert fn(True)
+
+def test_inhibit_tail_call():
+ from pypy.rpython.lltypesystem import lltype
+ def foobar_fn(n):
+ return 42
+ foobar_fn._dont_inline_ = True
+ def main(n):
+ return foobar_fn(n)
+ #
+ t = Translation(main, [int], backend="c")
+ t.rtype()
+ t.context._graphof(foobar_fn).inhibit_tail_call = True
+ t.source_c()
+ lines = t.driver.cbuilder.c_source_filename.readlines()
+ for i, line in enumerate(lines):
+ if '= pypy_g_foobar_fn' in line:
+ break
+ else:
+ assert 0, "the call was not found in the C source"
+ assert 'PYPY_INHIBIT_TAIL_CALL();' in lines[i+1]
Modified: pypy/branch/fast-forward/pypy/translator/c/test/test_lltyped.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/test/test_lltyped.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/test/test_lltyped.py Fri Oct 22 23:09:43 2010
@@ -401,6 +401,7 @@
for i in range(n):
p = malloc(S, flavor='raw', zero=True)
if p.x != 0 or p.y != 0:
+ free(p, flavor='raw')
return -1
p.x = i
p.y = i
@@ -418,14 +419,16 @@
def f(n):
for length in range(n-1, -1, -1):
p = malloc(S, length, flavor='raw', zero=True)
- if p.x != 0:
- return -1
- p.x = n
- for j in range(length):
- if p.y[j] != 0:
- return -3
- p.y[j] = n^j
- free(p, flavor='raw')
+ try:
+ if p.x != 0:
+ return -1
+ p.x = n
+ for j in range(length):
+ if p.y[j] != 0:
+ return -3
+ p.y[j] = n^j
+ finally:
+ free(p, flavor='raw')
return 42
fn = self.getcompiled(f, [int])
@@ -655,7 +658,7 @@
def test_prebuilt_ll2ctypes_array(self):
from pypy.rpython.lltypesystem import rffi, ll2ctypes
A = rffi.CArray(Char)
- a = malloc(A, 6, flavor='raw')
+ a = malloc(A, 6, flavor='raw', immortal=True)
a[0] = 'a'
a[1] = 'b'
a[2] = 'c'
@@ -676,7 +679,7 @@
def test_ll2ctypes_array_from_c(self):
from pypy.rpython.lltypesystem import rffi, ll2ctypes
A = rffi.CArray(Char)
- a = malloc(A, 6, flavor='raw')
+ a = malloc(A, 6, flavor='raw', immortal=True)
a[0] = 'a'
a[1] = 'b'
a[2] = 'c'
Modified: pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py Fri Oct 22 23:09:43 2010
@@ -624,13 +624,13 @@
os.unlink(self.filename)
def define_callback_with_collect(cls):
- from pypy.rlib.libffi import ffi_type_pointer, cast_type_to_ffitype,\
+ from pypy.rlib.clibffi import ffi_type_pointer, cast_type_to_ffitype,\
CDLL, ffi_type_void, CallbackFuncPtr, ffi_type_sint
from pypy.rpython.lltypesystem import rffi, ll2ctypes
import gc
ffi_size_t = cast_type_to_ffitype(rffi.SIZE_T)
- from pypy.rlib.libffi import get_libc_name
+ from pypy.rlib.clibffi import get_libc_name
def callback(ll_args, ll_res, stuff):
gc.collect()
Modified: pypy/branch/fast-forward/pypy/translator/c/test/test_standalone.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/test/test_standalone.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/test/test_standalone.py Fri Oct 22 23:09:43 2010
@@ -16,11 +16,16 @@
class StandaloneTests(object):
config = None
- def compile(self, entry_point, debug=True, shared=False):
+ def compile(self, entry_point, debug=True, shared=False,
+ stackcheck=False):
t = TranslationContext(self.config)
t.buildannotator().build_types(entry_point, [s_list_of_strings])
t.buildrtyper().specialize()
+ if stackcheck:
+ from pypy.translator.transform import insert_ll_stackcheck
+ insert_ll_stackcheck(t)
+
t.config.translation.shared = shared
cbuilder = CStandaloneBuilder(t, entry_point, t.config)
@@ -630,6 +635,22 @@
else:
os.environ['CC'] = old_cc
+ def test_inhibit_tail_call(self):
+ # the point is to check that the f()->f() recursion stops
+ from pypy.rlib.rstackovf import StackOverflow
+ def f(n):
+ if n <= 0:
+ return 42
+ return f(n+1)
+ def entry_point(argv):
+ try:
+ return f(1)
+ except StackOverflow:
+ print 'hi!'
+ return 0
+ t, cbuilder = self.compile(entry_point, stackcheck=True)
+ out = cbuilder.cmdexec("")
+ assert out.strip() == "hi!"
class TestMaemo(TestStandalone):
def setup_class(cls):
Modified: pypy/branch/fast-forward/pypy/translator/c/test/test_typed.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/test/test_typed.py (original)
+++ pypy/branch/fast-forward/pypy/translator/c/test/test_typed.py Fri Oct 22 23:09:43 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
import autopath
import sys
import math
@@ -823,3 +824,41 @@
while int(x + frac) >= -sys.maxint-1:
x -= 1
assert f(x + frac) == -666
+
+ def test_context_manager(self):
+ state = []
+ class C:
+ def __init__(self, name):
+ self.name = name
+ def __enter__(self):
+ state.append('acquire')
+ return self
+ def __exit__(self, typ, value, tb):
+ if typ is not None:
+ if value is None:
+ raise RuntimeError('test failed')
+ state.append('raised')
+ else:
+ if value is not None:
+ raise RuntimeError('test failed')
+ state.append('release')
+
+ def func(n):
+ del state[:]
+ try:
+ with C('hello') as c:
+ state.append(c.name)
+ if n == 1:
+ raise ValueError
+ elif n == 2:
+ raise TypeError
+ except (ValueError, TypeError):
+ pass
+ return ', '.join(state)
+ f = self.getcompiled(func, [int])
+ res = f(0)
+ assert res == 'acquire, hello, release'
+ res = f(1)
+ assert res == 'acquire, hello, raised, release'
+ res = f(2)
+ assert res == 'acquire, hello, raised, release'
Modified: pypy/branch/fast-forward/pypy/translator/goal/ann_override.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/goal/ann_override.py (original)
+++ pypy/branch/fast-forward/pypy/translator/goal/ann_override.py Fri Oct 22 23:09:43 2010
@@ -27,21 +27,6 @@
pol.pypytypes = {}
pol.single_space = single_space
- #def override__wrap_exception_cls(pol, space, x):
- # import pypy.objspace.std.typeobject as typeobject
- # clsdef = getbookkeeper().getuniqueclassdef(typeobject.W_TypeObject)
- # return annmodel.SomeInstance(clsdef, can_be_None=True)
- #
- #def override__fake_object(pol, space, x):
- # from pypy.interpreter import typedef
- # clsdef = getbookkeeper().getuniqueclassdef(typedef.W_Root)
- # return annmodel.SomeInstance(clsdef)
- #
- #def override__cpy_compile(pol, self, source, filename, mode, flags):
- # from pypy.interpreter import pycode
- # clsdef = getbookkeeper().getuniqueclassdef(pycode.PyCode)
- # return annmodel.SomeInstance(clsdef)
-
def specialize__wrap(pol, funcdesc, args_s):
from pypy.interpreter.baseobjspace import Wrappable
from pypy.annotation.classdef import ClassDef
Modified: pypy/branch/fast-forward/pypy/translator/goal/app_main.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/goal/app_main.py (original)
+++ pypy/branch/fast-forward/pypy/translator/goal/app_main.py Fri Oct 22 23:09:43 2010
@@ -306,7 +306,7 @@
break
elif arg == '-u':
options["unbuffered"] = True
- elif arg == '-O':
+ elif arg == '-O' or arg == '-OO':
pass
elif arg == '--version' or arg == '-V':
print "Python", sys.version
@@ -451,9 +451,12 @@
python_startup = os.getenv('PYTHONSTARTUP')
if python_startup:
try:
- startup = open(python_startup).read()
- except IOError:
- pass
+ f = open(python_startup)
+ startup = f.read()
+ f.close()
+ except IOError, e:
+ print >> sys.stderr, "Could not open PYTHONSTARTUP"
+ print >> sys.stderr, "IOError:", e
else:
def run_it():
co_python_startup = compile(startup,
Modified: pypy/branch/fast-forward/pypy/translator/jvm/test/test_class.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/jvm/test/test_class.py (original)
+++ pypy/branch/fast-forward/pypy/translator/jvm/test/test_class.py Fri Oct 22 23:09:43 2010
@@ -1,6 +1,6 @@
import py
from pypy.translator.jvm.test.runtest import JvmTest
-from pypy.translator.oosupport.test_template.class_ import BaseTestClass, BaseTestSpecialcase
+from pypy.translator.oosupport.test_template.class_ import BaseTestClass
class TestJvmClass(JvmTest, BaseTestClass):
def test_overridden_classattr_as_defaults(self):
@@ -26,6 +26,3 @@
def test_specialize_methods(self):
py.test.skip('ABSTRACT METHOD FIX: RE-TEST AFTER MERGE')
-
-class TestJvmSpecialCase(JvmTest, BaseTestSpecialcase):
- pass
Modified: pypy/branch/fast-forward/pypy/translator/oosupport/test_template/class_.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/oosupport/test_template/class_.py (original)
+++ pypy/branch/fast-forward/pypy/translator/oosupport/test_template/class_.py Fri Oct 22 23:09:43 2010
@@ -1,6 +1,5 @@
import py
from pypy.rpython.test import test_rclass
-from pypy.rpython.test.test_rspecialcase import BaseTestRspecialcase
class BaseTestClass(test_rclass.TestOOtype):
def test_abstract_method(self):
@@ -66,6 +65,3 @@
def test_cast_object_mix_null(self):
py.test.skip('cannot return ootype.NULL from translated functions')
-
-class BaseTestSpecialcase(BaseTestRspecialcase):
- pass
Modified: pypy/branch/fast-forward/pypy/translator/platform/darwin.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/platform/darwin.py (original)
+++ pypy/branch/fast-forward/pypy/translator/platform/darwin.py Fri Oct 22 23:09:43 2010
@@ -14,7 +14,10 @@
def __init__(self, cc=None):
if cc is None:
- cc = 'gcc'
+ try:
+ cc = os.environ['CC']
+ except KeyError:
+ cc = 'gcc'
self.cc = cc
def _args_for_shared(self, args):
Modified: pypy/branch/fast-forward/pypy/translator/transform.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/transform.py (original)
+++ pypy/branch/fast-forward/pypy/translator/transform.py Fri Oct 22 23:09:43 2010
@@ -221,15 +221,19 @@
stack_check_ptr_const = Constant(stack_check_ptr, lltype.typeOf(stack_check_ptr))
edges = set()
insert_in = set()
+ block2graph = {}
for caller in translator.graphs:
for block, callee in find_calls_from(translator, caller):
if getattr(getattr(callee, 'func', None),
'insert_stack_check_here', False):
insert_in.add(callee.startblock)
+ block2graph[callee.startblock] = callee
continue
if block is not caller.startblock:
edges.add((caller.startblock, block))
+ block2graph[caller.startblock] = caller
edges.add((block, callee.startblock))
+ block2graph[block] = caller
edgelist = [Edge(block1, block2) for (block1, block2) in edges]
edgedict = make_edge_dict(edgelist)
@@ -241,6 +245,10 @@
v.concretetype = lltype.Void
unwind_op = SpaceOperation('direct_call', [stack_check_ptr_const], v)
block.operations.insert(0, unwind_op)
+ # prevents cycles of tail calls from occurring -- such cycles would
+ # not consume any stack, so would turn into potentially infinite loops
+ graph = block2graph[block]
+ graph.inhibit_tail_call = True
return len(insert_in)
From afa at codespeak.net Fri Oct 22 23:54:13 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 22 Oct 2010 23:54:13 +0200 (CEST)
Subject: [pypy-svn] r78230 - in pypy/branch/fast-forward/pypy: interpreter
module/__builtin__ rlib
Message-ID: <20101022215413.6F9D8282BDC@codespeak.net>
Author: afa
Date: Fri Oct 22 23:54:08 2010
New Revision: 78230
Modified:
pypy/branch/fast-forward/pypy/interpreter/pyopcode.py
pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py
pypy/branch/fast-forward/pypy/rlib/clibffi.py
Log:
Fixes after the merge
Modified: pypy/branch/fast-forward/pypy/interpreter/pyopcode.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyopcode.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyopcode.py Fri Oct 22 23:54:08 2010
@@ -933,7 +933,6 @@
raise NotImplementedError("WITH_CLEANUP for CPython <= 2.4")
unroller = self.space.interpclass_w(w_unroller)
- w_exit = self.popvalue()
is_app_exc = (unroller is not None and
isinstance(unroller, SApplicationException))
if is_app_exc:
@@ -1375,7 +1374,8 @@
block_classes = {'SETUP_LOOP': LoopBlock,
'SETUP_EXCEPT': ExceptBlock,
'SETUP_FINALLY': FinallyBlock,
- 'SETUP_WITH': WithBlock}
+ 'SETUP_WITH': WithBlock,
+ }
### helpers written at the application-level ###
# Some of these functions are expected to be generally useful if other
Modified: pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py (original)
+++ pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py Fri Oct 22 23:54:08 2010
@@ -2,7 +2,7 @@
from pypy.interpreter.error import OperationError, operationerrfmt
from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped, applevel
from pypy.interpreter.gateway import interp2app, ObjSpace
-from pypy.interpreter.typedef import TypeDef
+from pypy.interpreter.typedef import TypeDef, make_weakref_descr
from pypy.interpreter.argument import Arguments
from pypy.interpreter.baseobjspace import Wrappable
from pypy.interpreter.typedef import GetSetProperty, descr_get_dict
@@ -706,7 +706,7 @@
space.call_function(w_func)
def descr_exit(self, space, w_type, w_value, w_tb):
- w_func = self.getattr(space, space.wrap('__exit__'), False)
+ w_func = self.getattr(space, '__exit__', False)
if w_func is not None:
return space.call_function(w_func, w_type, w_value, w_tb)
Modified: pypy/branch/fast-forward/pypy/rlib/clibffi.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/clibffi.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/clibffi.py Fri Oct 22 23:54:08 2010
@@ -148,7 +148,7 @@
# ffi_type_slong and ffi_type_ulong are omitted because
# their meaning changes too much from one libffi version to
# another. DON'T USE THEM! use cast_type_to_ffitype().
- 'float', 'pointer', 'void',
+ 'float', 'longdouble', 'pointer', 'void',
# by size
'sint8', 'uint8', 'sint16', 'uint16', 'sint32', 'uint32',
'sint64', 'uint64']
@@ -171,14 +171,16 @@
def _signed_type_for(TYPE):
sz = rffi.sizeof(TYPE)
- if sz == 2: return ffi_type_sint16
+ if sz == 1: return ffi_type_sint8
+ elif sz == 2: return ffi_type_sint16
elif sz == 4: return ffi_type_sint32
elif sz == 8: return ffi_type_sint64
else: raise ValueError("unsupported type size for %r" % (TYPE,))
def _unsigned_type_for(TYPE):
sz = rffi.sizeof(TYPE)
- if sz == 2: return ffi_type_uint16
+ if sz == 1: return ffi_type_uint8
+ elif sz == 2: return ffi_type_uint16
elif sz == 4: return ffi_type_uint32
elif sz == 8: return ffi_type_uint64
else: raise ValueError("unsupported type size for %r" % (TYPE,))
@@ -186,6 +188,7 @@
TYPE_MAP = {
rffi.DOUBLE : ffi_type_double,
rffi.FLOAT : ffi_type_float,
+ rffi.LONGDOUBLE : ffi_type_longdouble,
rffi.UCHAR : ffi_type_uchar,
rffi.CHAR : ffi_type_schar,
rffi.SHORT : ffi_type_sshort,
@@ -200,6 +203,7 @@
rffi.LONGLONG : _signed_type_for(rffi.LONGLONG),
lltype.Void : ffi_type_void,
lltype.UniChar : _unsigned_type_for(lltype.UniChar),
+ lltype.Bool : _unsigned_type_for(lltype.Bool),
}
def external(name, args, result, **kwds):
From afa at codespeak.net Sat Oct 23 00:08:36 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Sat, 23 Oct 2010 00:08:36 +0200 (CEST)
Subject: [pypy-svn] r78231 - pypy/branch/fast-forward/pypy/module/__builtin__
Message-ID: <20101022220836.0E553282BE8@codespeak.net>
Author: afa
Date: Sat Oct 23 00:08:34 2010
New Revision: 78231
Modified:
pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py
Log:
More fixes
Modified: pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py (original)
+++ pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py Sat Oct 23 00:08:34 2010
@@ -604,7 +604,7 @@
return w_ret
def descr_int(self, space):
- w_func = self.getattr(space, space.wrap('__int__'), False)
+ w_func = self.getattr(space, '__int__', False)
if w_func is not None:
return space.call_function(w_func)
@@ -619,7 +619,7 @@
space.wrap("__trunc__ returned non-Integral"))
def descr_long(self, space):
- w_func = self.getattr(space, space.wrap('__long__'), False)
+ w_func = self.getattr(space, '__long__', False)
if w_func is not None:
return space.call_function(w_func)
return self.descr_int(space)
From afa at codespeak.net Sat Oct 23 11:09:52 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Sat, 23 Oct 2010 11:09:52 +0200 (CEST)
Subject: [pypy-svn] r78232 - pypy/branch/fast-forward/pypy/rlib
Message-ID: <20101023090952.C762F282B9C@codespeak.net>
Author: afa
Date: Sat Oct 23 11:09:50 2010
New Revision: 78232
Modified:
pypy/branch/fast-forward/pypy/rlib/clibffi.py
Log:
Fix after merge
Modified: pypy/branch/fast-forward/pypy/rlib/clibffi.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/clibffi.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/clibffi.py Sat Oct 23 11:09:50 2010
@@ -376,6 +376,8 @@
FUNCFLAG_STDCALL = 0
FUNCFLAG_CDECL = 1 # for WINAPI calls
FUNCFLAG_PYTHONAPI = 4
+FUNCFLAG_USE_ERRNO = 8
+FUNCFLAG_USE_LASTERROR = 16
class AbstractFuncPtr(object):
ll_cif = lltype.nullptr(FFI_CIFP.TO)
From afa at codespeak.net Sat Oct 23 16:59:32 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Sat, 23 Oct 2010 16:59:32 +0200 (CEST)
Subject: [pypy-svn] r78233 -
pypy/branch/fast-forward/lib-python/modified-2.7.0/test
Message-ID: <20101023145932.13AC7282B9C@codespeak.net>
Author: afa
Date: Sat Oct 23 16:59:30 2010
New Revision: 78233
Added:
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_dumbdbm.py
- copied, changed from r78232, pypy/branch/fast-forward/lib-python/2.7.0/test/test_dumbdbm.py
Log:
Close the file in dumbdbm test
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_dumbdbm.py (from r78232, pypy/branch/fast-forward/lib-python/2.7.0/test/test_dumbdbm.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/test/test_dumbdbm.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_dumbdbm.py Sat Oct 23 16:59:30 2010
@@ -107,9 +107,11 @@
f.close()
# Mangle the file by adding \r before each newline
- data = open(_fname + '.dir').read()
+ with open(_fname + '.dir') as f:
+ data = f.read()
data = data.replace('\n', '\r\n')
- open(_fname + '.dir', 'wb').write(data)
+ with open(_fname + '.dir', 'wb') as f:
+ f.write(data)
f = dumbdbm.open(_fname)
self.assertEqual(f['1'], 'hello')
From afa at codespeak.net Sat Oct 23 17:37:33 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Sat, 23 Oct 2010 17:37:33 +0200 (CEST)
Subject: [pypy-svn] r78234 - in
pypy/branch/fast-forward/pypy/interpreter/pyparser: . test
Message-ID: <20101023153733.B4BDE36E0C3@codespeak.net>
Author: afa
Date: Sat Oct 23 17:37:29 2010
New Revision: 78234
Modified:
pypy/branch/fast-forward/pypy/interpreter/pyparser/pyparse.py
pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py
Log:
while compiling, decoding errors should be turned into SyntaxErrors
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/pyparse.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/pyparse.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/pyparse.py Sat Oct 23 17:37:29 2010
@@ -125,6 +125,11 @@
if space.is_w(e.w_type, space.w_LookupError):
raise error.SyntaxError("Unknown encoding: %s" % enc,
filename=compile_info.filename)
+ # Transform unicode errors into SyntaxError
+ if space.is_w(e.w_type, space.w_UnicodeDecodeError):
+ e.normalize_exception(space)
+ w_message = space.str(e.get_w_value(space))
+ raise error.SyntaxError(space.str_w(w_message))
raise
flags = compile_info.flags
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py Sat Oct 23 17:37:29 2010
@@ -56,6 +56,10 @@
input = "# coding: not-here"
exc = py.test.raises(SyntaxError, self.parse, input).value
assert exc.msg == "Unknown encoding: not-here"
+ input = u"# coding: ascii\n\xe2".encode('utf-8')
+ exc = py.test.raises(SyntaxError, self.parse, input).value
+ assert exc.msg == ("'ascii' codec can't decode byte 0xc3 "
+ "in position 16: ordinal not in range(128)")
def test_syntax_error(self):
parse = self.parse
From agaynor at codespeak.net Sat Oct 23 20:05:13 2010
From: agaynor at codespeak.net (agaynor at codespeak.net)
Date: Sat, 23 Oct 2010 20:05:13 +0200 (CEST)
Subject: [pypy-svn] r78235 - pypy/branch/fast-forward/pypy/doc/config
Message-ID: <20101023180513.163B6282B9C@codespeak.net>
Author: agaynor
Date: Sat Oct 23 20:05:10 2010
New Revision: 78235
Added:
pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._hashlib.txt
Log:
Document the _hashlib module.
Added: pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._hashlib.txt
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._hashlib.txt Sat Oct 23 20:05:10 2010
@@ -0,0 +1,2 @@
+Use the '_hashlib' module.
+Used by the 'hashlib' standard lib module, and indirectly by the various cryptographic libs. This module is expected to be working and is included by default.
From afa at codespeak.net Sat Oct 23 22:15:54 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Sat, 23 Oct 2010 22:15:54 +0200 (CEST)
Subject: [pypy-svn] r78236 - in pypy/branch/fast-forward/pypy: objspace/std
objspace/std/test rlib rlib/test
Message-ID: <20101023201554.3CE2A36E0CD@codespeak.net>
Author: afa
Date: Sat Oct 23 22:15:50 2010
New Revision: 78236
Modified:
pypy/branch/fast-forward/pypy/objspace/std/longtype.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py
pypy/branch/fast-forward/pypy/rlib/rbigint.py
pypy/branch/fast-forward/pypy/rlib/test/test_rbigint.py
Log:
Add long.bit_length()
Modified: pypy/branch/fast-forward/pypy/objspace/std/longtype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/longtype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/longtype.py Sat Oct 23 22:15:50 2010
@@ -91,6 +91,13 @@
def descr_get_imag(space, w_obj):
return space.newlong(0)
+def bit_length(space, w_obj):
+ try:
+ return space.wrap(w_obj.num.bit_length())
+ except OverflowError:
+ raise OperationError(space.w_OverflowError,
+ space.wrap("too many digits in integer"))
+
# ____________________________________________________________
long_typedef = StdTypeDef("long",
@@ -106,5 +113,6 @@
denominator = typedef.GetSetProperty(descr_get_denominator),
real = typedef.GetSetProperty(descr_get_real),
imag = typedef.GetSetProperty(descr_get_imag),
+ bit_length = gateway.interp2app(bit_length),
)
long_typedef.registermethods(globals())
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py Sat Oct 23 22:15:50 2010
@@ -269,3 +269,7 @@
pass
assert type(L(7).conjugate()) is long
+
+ def test_bit_length(self):
+ assert 8L.bit_length() == 4
+ assert (-1<<40).bit_length() == 41
Modified: pypy/branch/fast-forward/pypy/rlib/rbigint.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rbigint.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rbigint.py Sat Oct 23 22:15:50 2010
@@ -558,6 +558,23 @@
if self._numdigits() == 1 and self.digits[0] == 0:
self.sign = 0
+ def bit_length(self):
+ i = self._numdigits()
+ if i == 1 and self.digits[0] == 0:
+ return 0
+ msd = self.digits[i - 1]
+ msd_bits = 0
+ if msd >= 32:
+ msd_bits += 6
+ msd >>= 6
+ msd_bits += [
+ 0, 1, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4,
+ 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5
+ ][msd]
+ # yes, this can overflow: a huge number which fits 3 gigabytes of
+ # memory has around 24 gigabits!
+ bits = ovfcheck((i-1) * SHIFT + msd_bits)
+ return bits
def __repr__(self):
return "" % (self.digits, self.sign, self.str())
Modified: pypy/branch/fast-forward/pypy/rlib/test/test_rbigint.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/test/test_rbigint.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/test/test_rbigint.py Sat Oct 23 22:15:50 2010
@@ -325,6 +325,16 @@
b = rbigint.fromlong(-1<<3000)
assert a.mul(b).tolong() == (-1<<10000)*(-1<<3000)
+ def test_bit_length(self):
+ assert rbigint.fromlong(0).bit_length() == 0
+ assert rbigint.fromlong(1).bit_length() == 1
+ assert rbigint.fromlong(2).bit_length() == 2
+ assert rbigint.fromlong(3).bit_length() == 2
+ assert rbigint.fromlong(4).bit_length() == 3
+ assert rbigint.fromlong(-3).bit_length() == 2
+ assert rbigint.fromlong(-4).bit_length() == 3
+ assert rbigint.fromlong(1<<40).bit_length() == 41
+
class TestInternalFunctions(object):
def test__inplace_divrem1(self):
# signs are not handled in the helpers!
From afa at codespeak.net Sat Oct 23 22:32:29 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Sat, 23 Oct 2010 22:32:29 +0200 (CEST)
Subject: [pypy-svn] r78237 - pypy/branch/fast-forward/lib-python
Message-ID: <20101023203229.86909282B9C@codespeak.net>
Author: afa
Date: Sat Oct 23 22:32:28 2010
New Revision: 78237
Modified:
pypy/branch/fast-forward/lib-python/TODO
Log:
More TODO items
Modified: pypy/branch/fast-forward/lib-python/TODO
==============================================================================
--- pypy/branch/fast-forward/lib-python/TODO (original)
+++ pypy/branch/fast-forward/lib-python/TODO Sat Oct 23 22:32:28 2010
@@ -28,6 +28,10 @@
- missing functions in itertools: combinations, product, compress...
+- in test_os.py, fix posix.setregid(-1, -1), posix.setreuid(-1, -1). This
+ proably requires to use the git_t typedef instead of rffi.INT.
+
+
Medium tasks
------------
@@ -36,6 +40,8 @@
- missing builtin: memoryview
+- signal.set_wakeup_fd()
+
Longer tasks
------------
From antocuni at codespeak.net Sun Oct 24 12:05:30 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Sun, 24 Oct 2010 12:05:30 +0200 (CEST)
Subject: [pypy-svn] r78238 - pypy/extradoc/talk
Message-ID: <20101024100530.28289282B9C@codespeak.net>
Author: antocuni
Date: Sun Oct 24 12:05:27 2010
New Revision: 78238
Modified:
pypy/extradoc/talk/bibtex.bib
Log:
update
Modified: pypy/extradoc/talk/bibtex.bib
==============================================================================
--- pypy/extradoc/talk/bibtex.bib (original)
+++ pypy/extradoc/talk/bibtex.bib Sun Oct 24 12:05:27 2010
@@ -25,7 +25,8 @@
title = {High performance implementation of Python for CLI/.NET with JIT compiler generation for dynamic languages},
school = {{DISI}, Universit\'a di Genova},
author = {Antonio Cuni},
- year = {2010}
+ year = {2010},
+ note = {Technical Report {DISI-TH-2010-05}}
},
@inproceedings{ancona_rpython:dls_2007,
From arigo at codespeak.net Sun Oct 24 13:23:58 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 24 Oct 2010 13:23:58 +0200 (CEST)
Subject: [pypy-svn] r78239 - pypy/trunk/pypy/rlib/rsre
Message-ID: <20101024112358.41B20282B9C@codespeak.net>
Author: arigo
Date: Sun Oct 24 13:23:56 2010
New Revision: 78239
Modified:
pypy/trunk/pypy/rlib/rsre/rsre_jit.py
Log:
Reactivate JITing of regexps. It seems to give a speed-up at
least on spambayes.
Modified: pypy/trunk/pypy/rlib/rsre/rsre_jit.py
==============================================================================
--- pypy/trunk/pypy/rlib/rsre/rsre_jit.py (original)
+++ pypy/trunk/pypy/rlib/rsre/rsre_jit.py Sun Oct 24 13:23:56 2010
@@ -2,7 +2,7 @@
class RSreJitDriver(JitDriver):
- active = False # XXX temporary?
+ active = True
def __init__(self, name, debugprint, **kwds):
JitDriver.__init__(self, **kwds)
From arigo at codespeak.net Sun Oct 24 13:27:26 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 24 Oct 2010 13:27:26 +0200 (CEST)
Subject: [pypy-svn] r78240 - pypy/trunk/pypy/jit/codewriter/test
Message-ID: <20101024112726.9FB7536E0C3@codespeak.net>
Author: arigo
Date: Sun Oct 24 13:27:25 2010
New Revision: 78240
Modified:
pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py
Log:
Fix the test.
Modified: pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py (original)
+++ pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py Sun Oct 24 13:27:25 2010
@@ -691,6 +691,7 @@
class FakeJitDriverSD:
index = 42
class jitdriver:
+ active = True
greens = ['green1', 'green2', 'voidgreen3']
reds = ['red1', 'red2', 'voidred3']
jd = FakeJitDriverSD()
From arigo at codespeak.net Sun Oct 24 13:32:45 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 24 Oct 2010 13:32:45 +0200 (CEST)
Subject: [pypy-svn] r78241 - pypy/trunk/pypy/jit/tool
Message-ID: <20101024113245.8C2E1282B9C@codespeak.net>
Author: arigo
Date: Sun Oct 24 13:32:44 2010
New Revision: 78241
Added:
pypy/trunk/pypy/jit/tool/pypytrace.vim
Log:
A vim syntax highlighing too, just because.
Added: pypy/trunk/pypy/jit/tool/pypytrace.vim
==============================================================================
--- (empty file)
+++ pypy/trunk/pypy/jit/tool/pypytrace.vim Sun Oct 24 13:32:44 2010
@@ -0,0 +1,30 @@
+" Language : PyPy JIT traces
+" Maintainer : Armin Rigo
+
+if exists("b:current_syntax")
+ finish
+endif
+
+syn case ignore
+
+syn match pypyNumber '\<[0-9.]\+\>'
+syn match pypyConstPtr '\'
+syn region pypyDescr start=/descr= end=/>/ contains=pypyDescrField
+syn match pypyDescrField '[.]\w\+ ' contained
+syn match pypyOpNameStart '^' nextgroup=pypyOpName
+syn match pypyOpNameEqual ' = ' nextgroup=pypyOpName
+syn match pypyOpName '\l\l\w\+' contained
+syn match pypyFailArgs '[[].*[]]'
+syn match pypyLoopArgs '^[[].*'
+syn match pypyLoopStart '^#.*'
+syn match pypyDebugMergePoint '^debug_merge_point(.\+)'
+
+hi def link pypyLoopStart Structure
+"hi def link pypyLoopArgs PreProc
+hi def link pypyFailArgs String
+hi def link pypyOpName Statement
+hi def link pypyDebugMergePoint Comment
+hi def link pypyConstPtr Constant
+hi def link pypyNumber Number
+hi def link pypyDescr String
+hi def link pypyDescrField Label
From arigo at codespeak.net Sun Oct 24 13:33:43 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 24 Oct 2010 13:33:43 +0200 (CEST)
Subject: [pypy-svn] r78242 - pypy/branch/leak-finder-more
Message-ID: <20101024113343.D2BB0282B9C@codespeak.net>
Author: arigo
Date: Sun Oct 24 13:33:42 2010
New Revision: 78242
Added:
pypy/branch/leak-finder-more/
- copied from r78241, pypy/trunk/
Log:
Enable leakfinder also in app-level tests.
From arigo at codespeak.net Sun Oct 24 13:35:08 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 24 Oct 2010 13:35:08 +0200 (CEST)
Subject: [pypy-svn] r78243 - pypy/branch/leak-finder-more/pypy
Message-ID: <20101024113508.7D78336E0C3@codespeak.net>
Author: arigo
Date: Sun Oct 24 13:35:07 2010
New Revision: 78243
Modified:
pypy/branch/leak-finder-more/pypy/conftest.py
Log:
Do it.
Modified: pypy/branch/leak-finder-more/pypy/conftest.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/conftest.py (original)
+++ pypy/branch/leak-finder-more/pypy/conftest.py Sun Oct 24 13:35:07 2010
@@ -327,6 +327,18 @@
class PyPyTestFunction(py.test.collect.Function):
# All PyPy test items catch and display OperationErrors specially.
#
+ def runtest(self):
+ leakfinder.start_tracking_allocations()
+ try:
+ self._runtest()
+ finally:
+ if leakfinder.TRACK_ALLOCATIONS:
+ leaks = leakfinder.stop_tracking_allocations(False)
+ else:
+ leaks = None # stop_tracking_allocations() already called
+ if leaks: # check for leaks, but only if the test passed so far
+ raise leakfinder.MallocMismatch(leaks)
+
def execute_appex(self, space, target, *args):
try:
target(*args)
@@ -353,16 +365,9 @@
def _keywords(self):
return super(IntTestFunction, self)._keywords() + ['interplevel']
- def runtest(self):
+ def _runtest(self):
try:
- leakfinder.start_tracking_allocations()
- try:
- super(IntTestFunction, self).runtest()
- finally:
- if leakfinder.TRACK_ALLOCATIONS:
- leaks = leakfinder.stop_tracking_allocations(False)
- else:
- leaks = None # stop_tracking_allocations() already called
+ super(IntTestFunction, self).runtest()
except OperationError, e:
check_keyboard_interrupt(e)
raise
@@ -381,8 +386,6 @@
_pygame_imported = True
assert option.view, ("should not invoke Pygame "
"if conftest.option.view is False")
- if leaks: # check for leaks, but only if the test passed so far
- raise leakfinder.MallocMismatch(leaks)
class AppTestFunction(PyPyTestFunction):
def _prunetraceback(self, traceback):
@@ -395,7 +398,7 @@
def _keywords(self):
return ['applevel'] + super(AppTestFunction, self)._keywords()
- def runtest(self):
+ def _runtest(self):
target = self.obj
if option.runappdirect:
return target()
@@ -427,7 +430,7 @@
space.setattr(w_instance, space.wrap(name[2:]),
getattr(instance, name))
- def runtest(self):
+ def _runtest(self):
target = self.obj
if option.runappdirect:
return target()
From arigo at codespeak.net Sun Oct 24 14:20:52 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 24 Oct 2010 14:20:52 +0200 (CEST)
Subject: [pypy-svn] r78244 - pypy/branch/leak-finder-more/pypy
Message-ID: <20101024122052.AD55A282B9C@codespeak.net>
Author: arigo
Date: Sun Oct 24 14:20:50 2010
New Revision: 78244
Modified:
pypy/branch/leak-finder-more/pypy/conftest.py
Log:
Bah.
Modified: pypy/branch/leak-finder-more/pypy/conftest.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/conftest.py (original)
+++ pypy/branch/leak-finder-more/pypy/conftest.py Sun Oct 24 14:20:50 2010
@@ -328,16 +328,29 @@
# All PyPy test items catch and display OperationErrors specially.
#
def runtest(self):
- leakfinder.start_tracking_allocations()
+ self.runtest_open()
try:
- self._runtest()
+ self.runtest_perform()
finally:
- if leakfinder.TRACK_ALLOCATIONS:
- leaks = leakfinder.stop_tracking_allocations(False)
- else:
- leaks = None # stop_tracking_allocations() already called
- if leaks: # check for leaks, but only if the test passed so far
- raise leakfinder.MallocMismatch(leaks)
+ self.runtest_close()
+ self.runtest_finish()
+
+ def runtest_open(self):
+ leakfinder.start_tracking_allocations()
+
+ def runtest_perform(self):
+ super(PyPyTestFunction, self).runtest()
+
+ def runtest_close(self):
+ if leakfinder.TRACK_ALLOCATIONS:
+ self._pypytest_leaks = leakfinder.stop_tracking_allocations(False)
+ else: # stop_tracking_allocations() already called
+ self._pypytest_leaks = None
+
+ def runtest_finish(self):
+ # check for leaks, but only if the test passed so far
+ if self._pypytest_leaks:
+ raise leakfinder.MallocMismatch(self._pypytest_leaks)
def execute_appex(self, space, target, *args):
try:
@@ -365,9 +378,9 @@
def _keywords(self):
return super(IntTestFunction, self)._keywords() + ['interplevel']
- def _runtest(self):
+ def runtest_perform(self):
try:
- super(IntTestFunction, self).runtest()
+ super(IntTestFunction, self).runtest_perform()
except OperationError, e:
check_keyboard_interrupt(e)
raise
@@ -380,12 +393,15 @@
py.test.skip('%s: %s' % (e.__class__.__name__, e))
cls = cls.__bases__[0]
raise
+
+ def runtest_finish(self):
if 'pygame' in sys.modules:
global _pygame_imported
if not _pygame_imported:
_pygame_imported = True
assert option.view, ("should not invoke Pygame "
"if conftest.option.view is False")
+ super(IntTestFunction, self).runtest_finish()
class AppTestFunction(PyPyTestFunction):
def _prunetraceback(self, traceback):
@@ -398,7 +414,7 @@
def _keywords(self):
return ['applevel'] + super(AppTestFunction, self)._keywords()
- def _runtest(self):
+ def runtest_perform(self):
target = self.obj
if option.runappdirect:
return target()
@@ -430,7 +446,7 @@
space.setattr(w_instance, space.wrap(name[2:]),
getattr(instance, name))
- def _runtest(self):
+ def runtest_perform(self):
target = self.obj
if option.runappdirect:
return target()
From arigo at codespeak.net Sun Oct 24 14:21:52 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 24 Oct 2010 14:21:52 +0200 (CEST)
Subject: [pypy-svn] r78245 - pypy/trunk/pypy/annotation/test
Message-ID: <20101024122152.8AE48282B9C@codespeak.net>
Author: arigo
Date: Sun Oct 24 14:21:51 2010
New Revision: 78245
Modified:
pypy/trunk/pypy/annotation/test/test_annrpython.py
Log:
Let it run on python 2.5.
Modified: pypy/trunk/pypy/annotation/test/test_annrpython.py
==============================================================================
--- pypy/trunk/pypy/annotation/test/test_annrpython.py (original)
+++ pypy/trunk/pypy/annotation/test/test_annrpython.py Sun Oct 24 14:21:51 2010
@@ -1,4 +1,4 @@
-
+from __future__ import with_statement
import autopath
import py.test
import sys
From arigo at codespeak.net Sun Oct 24 14:27:23 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 24 Oct 2010 14:27:23 +0200 (CEST)
Subject: [pypy-svn] r78246 - in pypy/trunk/pypy/module/_ffi: . test
Message-ID: <20101024122723.ACFB3282B9C@codespeak.net>
Author: arigo
Date: Sun Oct 24 14:27:22 2010
New Revision: 78246
Modified:
pypy/trunk/pypy/module/_ffi/ (props changed)
pypy/trunk/pypy/module/_ffi/test/ (props changed)
Log:
fixeol
From arigo at codespeak.net Sun Oct 24 14:35:45 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 24 Oct 2010 14:35:45 +0200 (CEST)
Subject: [pypy-svn] r78247 - in pypy/trunk/pypy/module: _winreg bz2
Message-ID: <20101024123545.BB30F36E0C3@codespeak.net>
Author: arigo
Date: Sun Oct 24 14:35:44 2010
New Revision: 78247
Modified:
pypy/trunk/pypy/module/_winreg/interp_winreg.py
pypy/trunk/pypy/module/bz2/interp_bz2.py
Log:
More with_statements.
Modified: pypy/trunk/pypy/module/_winreg/interp_winreg.py
==============================================================================
--- pypy/trunk/pypy/module/_winreg/interp_winreg.py (original)
+++ pypy/trunk/pypy/module/_winreg/interp_winreg.py Sun Oct 24 14:35:44 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
from pypy.interpreter.baseobjspace import Wrappable
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
from pypy.interpreter.gateway import interp2app
Modified: pypy/trunk/pypy/module/bz2/interp_bz2.py
==============================================================================
--- pypy/trunk/pypy/module/bz2/interp_bz2.py (original)
+++ pypy/trunk/pypy/module/bz2/interp_bz2.py Sun Oct 24 14:35:44 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
from pypy.rpython.tool import rffi_platform as platform
from pypy.rpython.lltypesystem import rffi
from pypy.rpython.lltypesystem import lltype
From david at codespeak.net Sun Oct 24 14:37:18 2010
From: david at codespeak.net (david at codespeak.net)
Date: Sun, 24 Oct 2010 14:37:18 +0200 (CEST)
Subject: [pypy-svn] r78248 - in
pypy/branch/arm-backend/pypy/jit/backend/arm: . test
Message-ID: <20101024123718.671FE282B9C@codespeak.net>
Author: david
Date: Sun Oct 24 14:37:16 2010
New Revision: 78248
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py
Log:
Refactor guards and finish res op to leave using common code.
Start implementig bridges based on refactored code.
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Sun Oct 24 14:37:16 2010
@@ -1,11 +1,14 @@
-from pypy.jit.backend.arm.codebuilder import ARMv7Builder
-from pypy.jit.backend.arm import registers as r
from pypy.jit.backend.arm import conditions as c
-from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity
+from pypy.jit.backend.arm import registers as r
+from pypy.jit.backend.arm.arch import WORD, FUNC_ALIGN
+from pypy.jit.backend.arm.codebuilder import ARMv7Builder, ARMv7InMemoryBuilder
from pypy.jit.backend.arm.regalloc import ARMRegisterManager
+from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity
+from pypy.jit.metainterp.history import ConstInt, Box, BasicFailDescr
from pypy.jit.metainterp.resoperation import rop
-from pypy.jit.metainterp.history import ConstInt, Box
-from pypy.rpython.lltypesystem import lltype
+from pypy.rlib import rgc
+from pypy.rpython.annlowlevel import llhelper
+from pypy.rpython.lltypesystem import lltype, rffi, llmemory
# XXX Move to llsupport
from pypy.jit.backend.x86.support import values_array
@@ -17,15 +20,132 @@
self.cpu = cpu
self.input_arg_boxes_int = values_array(lltype.Signed, failargs_limit) # merge with fail_boxes_int later
self.fail_boxes_int = values_array(lltype.Signed, failargs_limit)
+ self._debug_asm = True
+
+ self._gen_exit_path()
+ self.align()
+ self.mc._start_addr = self.mc.curraddr()
+
+
+ def setup_failure_recovery(self):
+
+ @rgc.no_collect
+ def failure_recovery_func(mem_loc, stackloc):
+ """mem_loc is a structure in memory describing where the values for
+ the failargs are stored. stacklock is the address of the stack
+ section where the registers were saved."""
+ enc = rffi.cast(rffi.CCHARP, mem_loc)
+ stack = rffi.cast(rffi.CCHARP, stackloc)
+ return self.decode_registers_and_descr(enc, stack)
+
+ self.failure_recovery_func = failure_recovery_func
+
+ @rgc.no_collect
+ def decode_registers_and_descr(self, enc, stack):
+ """Decode locations encoded in memory at enc and write the values to
+ the failboxes.
+ Registers are saved on the stack
+ XXX Rest to follow"""
+ i = -1
+ while(True):
+ i += 1
+ r = enc[i]
+ if r == '\xFF':
+ break
+ reg = ord(enc[i])
+ self.fail_boxes_int.setitem(i, self.decode32(stack, reg*WORD))
+ assert enc[i] == '\xFF'
+ descr = self.decode32(enc, i+1)
+ return descr
+
+ def decode32(self, mem, index):
+ highval = ord(mem[index+3])
+ if highval >= 128:
+ highval -= 256
+ return (ord(mem[index])
+ | ord(mem[index+1]) << 8
+ | ord(mem[index+2]) << 16
+ | highval << 24)
+
+ def encode32(self, mem, i, n):
+ mem[i] = chr(n & 0xFF)
+ mem[i+1] = chr((n >> 8) & 0xFF)
+ mem[i+2] = chr((n >> 16) & 0xFF)
+ mem[i+3] = chr((n >> 24) & 0xFF)
+
+ def _gen_exit_path(self):
+ self.setup_failure_recovery()
+ functype = lltype.Ptr(lltype.FuncType([lltype.Signed, lltype.Signed], lltype.Signed))
+ decode_registers_addr = llhelper(functype, self.failure_recovery_func)
+ self.mc.PUSH(range(12)) # registers r0 .. r11
+ self.mc.MOV_rr(r.r0, r.lr) # move mem block address, to r0 to pass as
+ # parameter to next procedure call
+ self.mc.MOV_rr(r.r1, r.sp) # pass the current stack pointer as second param
+ self.mc.gen_load_int(r.r2, rffi.cast(lltype.Signed, decode_registers_addr))
+ self.mc.gen_load_int(r.lr, self.mc.curraddr()+self.mc.size_of_gen_load_int+WORD)
+ self.mc.MOV_rr(r.pc, r.r2)
+ self.mc.MOV_rr(r.ip, r.r0)
+ self.mc.LDM(r.sp, range(12), w=1) # XXX Replace with POP instr. someday
+
+ self.mc.MOV_rr(r.r0, r.ip)
+
+ self.gen_func_epilog()
+
+ def _gen_path_to_exit_path(self, op, args, regalloc, fcond=c.AL):
+ box = Box()
+ reg = regalloc.try_allocate_reg(box)
+ # XXX free this memory
+ mem = lltype.malloc(rffi.CArray(lltype.Char), len(args)+5, flavor='raw')
+ for i in range(len(args)):
+ curreg = regalloc.try_allocate_reg(args[i])
+ mem[i] = chr(curreg)
+ i = len(args)
+ mem[i] = chr(0xFF)
+ memaddr = rffi.cast(lltype.Signed, mem)
+
+
+ n = self.cpu.get_fail_descr_number(op.getdescr())
+ self.encode32(mem, i+1, n)
+ self.mc.gen_load_int(r.lr, memaddr, cond=fcond)
+ self.mc.gen_load_int(reg, self.mc.baseaddr(), cond=fcond)
+ self.mc.MOV_rr(r.pc, reg, cond=fcond)
+
+ op.getdescr()._arm_guard_reg = reg
+ return memaddr
+
+ def align(self):
+ while(self.mc.curraddr() % FUNC_ALIGN != 0):
+ self.mc.writechar(chr(0))
+
+ def gen_func_epilog(self,cond=c.AL):
+ self.mc.LDM(r.sp, r.callee_restored_registers, cond=cond, w=1)
+
+ def gen_func_prolog(self):
+ self.mc.PUSH(r.callee_saved_registers)
+
+ def gen_bootstrap_code(self, inputargs, regalloc):
+ for i in range(len(inputargs)):
+ reg = regalloc.try_allocate_reg(inputargs[i])
+ addr = self.fail_boxes_int.get_addr_for_num(i)
+ self.mc.gen_load_int(reg, addr)
+ self.mc.LDR_ri(reg, reg)
+
+ def gen_bridge_bootstrap_code(self, inputargs, regalloc):
+ for i in range(len(inputargs)):
+ reg = regalloc.try_allocate_reg(inputargs[i])
+ addr = self.fail_boxes_int.get_addr_for_num(i)
+ self.mc.gen_load_int(reg, addr)
+ self.mc.LDR_ri(reg, reg)
+ # cpu interface
def assemble_loop(self, inputargs, operations, looptoken):
- assert len(inputargs) == 1
longevity = compute_vars_longevity(inputargs, operations)
regalloc = ARMRegisterManager(longevity, assembler=self.mc)
+ loop_start=self.mc.curraddr()
self.gen_func_prolog()
self.gen_bootstrap_code(inputargs, regalloc)
loop_head=self.mc.curraddr()
- looptoken._arm_bootstrap_code = self.mc.baseaddr()
+ looptoken._arm_bootstrap_code = loop_start
looptoken._arm_loop_code = loop_head
looptoken._temp_inputargs = inputargs#XXX remove
fcond=c.AL
@@ -33,12 +153,42 @@
opnum = op.getopnum()
fcond = self.operations[opnum](self, op, regalloc, fcond)
self.gen_func_epilog()
- f = open('loop.asm', 'wb')
- for i in range(self.mc._pos):
- f.write(self.mc._data[i])
- f.close()
+ if self._debug_asm:
+ self._dump_trace('loop.asm')
print 'Done assembling'
+ def assemble_bridge(self, faildescr, inputargs, operations):
+ # XXX need to restore args here
+ longevity = compute_vars_longevity(inputargs, operations)
+ regalloc = ARMRegisterManager(longevity, assembler=self.mc)
+ bridge_head = self.mc.curraddr()
+ self.gen_bridge_bootstrap_code(inputargs, regalloc)
+ fcond = c.AL
+ for op in operations:
+ opnum = op.getopnum()
+ fcond = self.operations[opnum](self, op, regalloc, fcond)
+ self.gen_func_epilog()
+ print 'Done building bridges'
+ self.patch_trace(faildescr, bridge_head)
+ print 'Done patching trace'
+ if self._debug_asm:
+ self._dump_trace('bridge.asm')
+
+
+ def _dump_trace(self, name):
+ self.mc._dump_trace(name)
+
+ def patch_trace(self, faildescr, bridge_addr):
+ # XXX make sure there is enough space at patch target
+ fcond = faildescr._arm_guard_cond
+ b = ARMv7InMemoryBuilder(faildescr._arm_guard_code, faildescr._arm_guard_code+100)
+ reg = faildescr._arm_guard_reg
+ b.gen_load_int(reg, bridge_addr, fcond)
+ b.MOV_rr(r.pc, reg, cond=fcond)
+
+
+ # Resoperations
+
def emit_op_jump(self, op, regalloc, fcond):
tmp = Box()
tmpreg = regalloc.try_allocate_reg(tmp)
@@ -49,13 +199,13 @@
# XXX only if every value is in a register
self.mc.MOV_rr(inpreg, reg)
loop_code = op.getdescr()._arm_loop_code
- self.gen_load_int(tmpreg, loop_code)
+ self.mc.gen_load_int(tmpreg, loop_code)
self.mc.MOV_rr(r.pc, tmpreg)
regalloc.possibly_free_var(tmpreg)
return fcond
def emit_op_finish(self, op, regalloc, fcond):
- self.gen_write_back(op, op.getarglist(), regalloc, fcond)
+ self._gen_path_to_exit_path(op, op.getarglist(), regalloc, fcond)
return fcond
def emit_op_int_le(self, op, regalloc, fcond):
@@ -74,46 +224,14 @@
def emit_op_guard_true(self, op, regalloc, fcond):
assert fcond == c.GT
- self.gen_write_back(op, op.getfailargs(), regalloc, fcond)
- self.gen_func_epilog(cond=fcond)
+ descr = op.getdescr()
+ assert isinstance(descr, BasicFailDescr)
+ memaddr = self._gen_path_to_exit_path(op, op.getfailargs(), regalloc, fcond)
+ descr._failure_recovery_code = memaddr
+ descr._arm_guard_code = self.mc.curraddr()
+ descr._arm_guard_cond = fcond
return c.AL
- def gen_write_back(self, op, args, regalloc, fcond):
- temp = Box()
- temp_reg = regalloc.try_allocate_reg(temp)
- for i in range(len(args)):
- reg = regalloc.try_allocate_reg(args[i])
- addr = self.fail_boxes_int.get_addr_for_num(i)
- self.gen_load_int(temp_reg, addr, cond=fcond)
- self.mc.STR_ri(reg, temp_reg, cond=fcond)
-
- regalloc.possibly_free_var(temp_reg)
- n = self.cpu.get_fail_descr_number(op.getdescr())
- self.mc.MOV_ri(r.r0, n, cond=fcond)
-
- def gen_func_epilog(self,cond=c.AL):
- self.mc.LDM(r.sp, r.callee_restored_registers, cond=cond)
-
- def gen_func_prolog(self):
- self.mc.PUSH(r.callee_saved_registers)
-
- def gen_bootstrap_code(self, inputargs, regalloc):
- for i in range(len(inputargs)):
- reg = regalloc.try_allocate_reg(inputargs[i])
- addr = self.input_arg_boxes_int.get_addr_for_num(i)
- self.gen_load_int(reg, addr)
- self.mc.LDR_ri(reg, reg)
-
- def gen_load_int(self, reg, value, cond=c.AL):
- assert reg != r.ip, 'ip is used to load int'
- self.mc.MOV_ri(reg, (value & 0xFF), cond=cond)
-
- for offset in range(8, 25, 8):
- self.mc.MOV_ri(r.ip, (value >> offset) & 0xFF, cond=cond)
- self.mc.ORR_rr(reg, reg, r.ip, offset, cond=cond)
-
-
-
def make_operation_list():
def notimplemented(self, op, regalloc, fcond):
raise NotImplementedError
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Sun Oct 24 14:37:16 2010
@@ -1,22 +1,29 @@
-import conditions as cond
-import registers as reg
-from pypy.rlib.rmmap import alloc
-from pypy.rpython.lltypesystem import lltype, rffi
+from pypy.jit.backend.arm import conditions as cond
+from pypy.jit.backend.arm import registers as reg
+from pypy.jit.backend.arm.arch import WORD
from pypy.jit.backend.arm.instruction_builder import define_instructions
-class ARMv7Builder(object):
+from pypy.rlib.rmmap import alloc, PTR
+from pypy.rpython.lltypesystem import lltype, rffi
- def __init__(self):
- self._data = alloc(1024)
+class AbstractARMv7Builder(object):
+ def _init(self, data, map_size):
+ self._data = data
+ self._size = map_size
self._pos = 0
+ def _dump_trace(self, name):
+ f = open('output/%s' % name, 'wb')
+ for i in range(self._pos):
+ f.write(self._data[i])
+ f.close()
+
def PUSH(self, regs, cond=cond.AL):
assert reg.sp not in regs
instr = self._encode_reg_list(cond << 28 | 0x92D << 16, regs)
self.write32(instr)
- def LDM(self, rn, regs, cond=cond.AL):
- w = 0
+ def LDM(self, rn, regs, w=0, cond=cond.AL):
instr = cond << 28 | 0x89 << 20 | w << 21 | (rn & 0xFF) << 16
instr = self._encode_reg_list(instr, regs)
self.write32(instr)
@@ -58,4 +65,30 @@
def curraddr(self):
return self.baseaddr() + self._pos
-define_instructions(ARMv7Builder)
+ size_of_gen_load_int = 7 * WORD
+ def gen_load_int(self, r, value, cond=cond.AL):
+ assert r != reg.ip, 'ip is used to load int'
+ self.MOV_ri(r, (value & 0xFF), cond=cond)
+
+ for offset in range(8, 25, 8):
+ t = (value >> offset) & 0xFF
+ #if t == 0:
+ # continue
+ self.MOV_ri(reg.ip, t, cond=cond)
+ self.ORR_rr(r, r, reg.ip, offset, cond=cond)
+
+class ARMv7InMemoryBuilder(AbstractARMv7Builder):
+ def __init__(self, start, end):
+ map_size = end - start
+ data = rffi.cast(PTR, start)
+ self._init(data, map_size)
+
+class ARMv7Builder(AbstractARMv7Builder):
+
+ def __init__(self):
+ map_size = 1024
+ data = alloc(1024)
+ self._pos = 0
+ self._init(data, map_size)
+
+define_instructions(AbstractARMv7Builder)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py Sun Oct 24 14:37:16 2010
@@ -56,3 +56,18 @@
supervisor_and_coproc = {
'MCR': {'op1': 0x20, 'op': 1, 'rn':0, 'coproc':0},
}
+
+block_data = {
+ 'STMDA': {'op': 0x0},
+ 'LDMDA': {'op': 0x1},
+ 'STMIA': {'op': 0x8},
+ 'LDMDB': {'op': 0x11},
+ 'STMIB': {'op': 0x18},
+ 'LDMIB': {'op': 0x19},
+ 'STM': {'op': 0x4},
+ 'LDM': {'op': 0x5},
+}
+branch = {
+ 'B': {'op': 0x20},
+ 'BL': {'op': 0x30},
+}
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Sun Oct 24 14:37:16 2010
@@ -18,6 +18,9 @@
def compile_loop(self, inputargs, operations, looptoken):
self.assembler.assemble_loop(inputargs, operations, looptoken)
+ def compile_bridge(self, faildescr, inputargs, operations):
+ self.assembler.assemble_bridge(faildescr, inputargs, operations)
+
def set_future_value_int(self, index, intvalue):
self.assembler.input_arg_boxes_int.setitem(index, intvalue)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py Sun Oct 24 14:37:16 2010
@@ -10,7 +10,7 @@
def run_asm(asm):
BOOTSTRAP_TP = lltype.FuncType([], lltype.Signed)
- addr = asm.mc.baseaddr()
+ addr = asm.mc._start_addr
assert addr % 8 == 0
func = rffi.cast(lltype.Ptr(BOOTSTRAP_TP), addr)
return func()
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py Sun Oct 24 14:37:16 2010
@@ -1,9 +1,14 @@
-from pypy.jit.backend.arm import registers as r
from pypy.jit.backend.arm import conditions as c
+from pypy.jit.backend.arm import registers as r
+from pypy.jit.backend.arm.arch import WORD
from pypy.jit.backend.arm.assembler import AssemblerARM
+from pypy.jit.backend.arm.codebuilder import ARMv7InMemoryBuilder
from pypy.jit.backend.arm.test.support import skip_unless_arm, run_asm
from pypy.jit.metainterp.resoperation import rop
+from pypy.rpython.annlowlevel import llhelper
+from pypy.rpython.lltypesystem import lltype, rffi, llmemory
+
skip_unless_arm()
class TestRunningAssembler():
@@ -16,19 +21,19 @@
def test_load_small_int_to_reg(self):
self.a.gen_func_prolog()
- self.a.gen_load_int(r.r0, 123)
+ self.a.mc.gen_load_int(r.r0, 123)
self.a.gen_func_epilog()
assert run_asm(self.a) == 123
def test_load_medium_int_to_reg(self):
self.a.gen_func_prolog()
- self.a.gen_load_int(r.r0, 0xBBD7)
+ self.a.mc.gen_load_int(r.r0, 0xBBD7)
self.a.gen_func_epilog()
assert run_asm(self.a) == 48087
def test_load_int_to_reg(self):
self.a.gen_func_prolog()
- self.a.gen_load_int(r.r0, 0xFFFFFF85)
+ self.a.mc.gen_load_int(r.r0, 0xFFFFFF85)
self.a.gen_func_epilog()
assert run_asm(self.a) == -123
@@ -43,14 +48,14 @@
def test_sub(self):
self.a.gen_func_prolog()
- self.a.gen_load_int(r.r1, 123456)
+ self.a.mc.gen_load_int(r.r1, 123456)
self.a.mc.SUB_ri(r.r0, r.r1, 123)
self.a.gen_func_epilog()
assert run_asm(self.a) == 123333
def test_cmp(self):
self.a.gen_func_prolog()
- self.a.gen_load_int(r.r1, 22)
+ self.a.mc.gen_load_int(r.r1, 22)
self.a.mc.CMP(r.r1, 123)
self.a.mc.MOV_ri(r.r0, 1, c.LE)
self.a.mc.MOV_ri(r.r0, 0, c.GT)
@@ -59,7 +64,7 @@
def test_int_le_false(self):
self.a.gen_func_prolog()
- self.a.gen_load_int(r.r1, 2222)
+ self.a.mc.gen_load_int(r.r1, 2222)
self.a.mc.CMP(r.r1, 123)
self.a.mc.MOV_ri(r.r0, 1, c.LE)
self.a.mc.MOV_ri(r.r0, 0, c.GT)
@@ -73,7 +78,7 @@
self.a.mc.CMP(r.r1, 0) # z=0, z=1
self.a.mc.MOV_ri(r.r1, 0, cond=c.NE)
self.a.mc.MOV_ri(r.r1, 7, cond=c.EQ)
- self.a.gen_load_int(r.r4, loop_head, cond=c.NE)
+ self.a.mc.gen_load_int(r.r4, loop_head, cond=c.NE)
self.a.mc.MOV_rr(r.pc, r.r4, cond=c.NE)
self.a.mc.MOV_rr(r.r0, r.r1)
self.a.gen_func_epilog()
@@ -85,10 +90,25 @@
loop_head = self.a.mc.curraddr()
self.a.mc.ADD_ri(r.r1, r.r1, 1)
self.a.mc.CMP(r.r1, 9)
- self.a.gen_load_int(r.r4, loop_head, cond=c.NE)
+ self.a.mc.gen_load_int(r.r4, loop_head, cond=c.NE)
self.a.mc.MOV_rr(r.pc, r.r4, cond=c.NE)
self.a.mc.MOV_rr(r.r0, r.r1)
self.a.gen_func_epilog()
assert run_asm(self.a) == 9
+ def test_call_python_func(self):
+ functype = lltype.Ptr(lltype.FuncType([lltype.Signed], lltype.Signed))
+ call_addr = rffi.cast(lltype.Signed, llhelper(functype, callme))
+ self.a.gen_func_prolog()
+ self.a.mc.MOV_ri(r.r0, 123)
+ self.a.mc.gen_load_int(r.r1, call_addr)
+ self.a.mc.gen_load_int(r.lr, self.a.mc.curraddr()+self.a.mc.size_of_gen_load_int+WORD)
+ self.a.mc.MOV_rr(r.pc, r.r1)
+ self.a.gen_func_epilog()
+ assert run_asm(self.a) == 133
+
+def callme(inp):
+ i = inp + 10
+ return i
+
From david at codespeak.net Sun Oct 24 14:47:01 2010
From: david at codespeak.net (david at codespeak.net)
Date: Sun, 24 Oct 2010 14:47:01 +0200 (CEST)
Subject: [pypy-svn] r78249 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101024124701.09BE836E0C3@codespeak.net>
Author: david
Date: Sun Oct 24 14:47:00 2010
New Revision: 78249
Added:
pypy/branch/arm-backend/pypy/jit/backend/arm/arch.py
Log:
Arch constants
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/arch.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/arch.py Sun Oct 24 14:47:00 2010
@@ -0,0 +1,2 @@
+FUNC_ALIGN=8
+WORD=4
From antocuni at codespeak.net Sun Oct 24 17:54:50 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Sun, 24 Oct 2010 17:54:50 +0200 (CEST)
Subject: [pypy-svn] r78250 - pypy/extradoc/talk
Message-ID: <20101024155450.EB96F282B9C@codespeak.net>
Author: antocuni
Date: Sun Oct 24 17:54:45 2010
New Revision: 78250
Modified:
pypy/extradoc/talk/bibtex.bib
Log:
s/\t/' '
Modified: pypy/extradoc/talk/bibtex.bib
==============================================================================
--- pypy/extradoc/talk/bibtex.bib (original)
+++ pypy/extradoc/talk/bibtex.bib Sun Oct 24 17:54:45 2010
@@ -26,7 +26,7 @@
school = {{DISI}, Universit\'a di Genova},
author = {Antonio Cuni},
year = {2010},
- note = {Technical Report {DISI-TH-2010-05}}
+ note = {Technical Report {DISI-TH-2010-05}}
},
@inproceedings{ancona_rpython:dls_2007,
From afa at codespeak.net Sun Oct 24 20:07:08 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Sun, 24 Oct 2010 20:07:08 +0200 (CEST)
Subject: [pypy-svn] r78251 - in pypy/branch/fast-forward/pypy:
annotation/test jit/codewriter/test jit/tool module/_ffi
module/_ffi/test module/_winreg module/bz2 rlib/rsre
Message-ID: <20101024180708.AFCAC282B9C@codespeak.net>
Author: afa
Date: Sun Oct 24 20:07:06 2010
New Revision: 78251
Added:
pypy/branch/fast-forward/pypy/jit/tool/pypytrace.vim
- copied unchanged from r78250, pypy/trunk/pypy/jit/tool/pypytrace.vim
Modified:
pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py
pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py
pypy/branch/fast-forward/pypy/module/_ffi/ (props changed)
pypy/branch/fast-forward/pypy/module/_ffi/test/ (props changed)
pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py
pypy/branch/fast-forward/pypy/module/bz2/interp_bz2.py
pypy/branch/fast-forward/pypy/rlib/rsre/rsre_jit.py
Log:
Merge from trunk
78228:78250
Modified: pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py
==============================================================================
--- pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py (original)
+++ pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py Sun Oct 24 20:07:06 2010
@@ -1,4 +1,4 @@
-
+from __future__ import with_statement
import autopath
import py.test
import sys
Modified: pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py
==============================================================================
--- pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py (original)
+++ pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py Sun Oct 24 20:07:06 2010
@@ -691,6 +691,7 @@
class FakeJitDriverSD:
index = 42
class jitdriver:
+ active = True
greens = ['green1', 'green2', 'voidgreen3']
reds = ['red1', 'red2', 'voidred3']
jd = FakeJitDriverSD()
Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py (original)
+++ pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py Sun Oct 24 20:07:06 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
from pypy.interpreter.baseobjspace import Wrappable
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
from pypy.interpreter.gateway import interp2app
Modified: pypy/branch/fast-forward/pypy/module/bz2/interp_bz2.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/bz2/interp_bz2.py (original)
+++ pypy/branch/fast-forward/pypy/module/bz2/interp_bz2.py Sun Oct 24 20:07:06 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
from pypy.rpython.tool import rffi_platform as platform
from pypy.rpython.lltypesystem import rffi
from pypy.rpython.lltypesystem import lltype
Modified: pypy/branch/fast-forward/pypy/rlib/rsre/rsre_jit.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rsre/rsre_jit.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rsre/rsre_jit.py Sun Oct 24 20:07:06 2010
@@ -2,7 +2,7 @@
class RSreJitDriver(JitDriver):
- active = False # XXX temporary?
+ active = True
def __init__(self, name, debugprint, **kwds):
JitDriver.__init__(self, **kwds)
From cfbolz at codespeak.net Mon Oct 25 11:06:04 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon, 25 Oct 2010 11:06:04 +0200 (CEST)
Subject: [pypy-svn] r78252 - pypy/extradoc/sprintinfo/ddorf2010
Message-ID: <20101025090604.B333F282C04@codespeak.net>
Author: cfbolz
Date: Mon Oct 25 11:06:01 2010
New Revision: 78252
Added:
pypy/extradoc/sprintinfo/ddorf2010/planning.txt (contents, props changed)
Log:
(all) planning for today
Added: pypy/extradoc/sprintinfo/ddorf2010/planning.txt
==============================================================================
--- (empty file)
+++ pypy/extradoc/sprintinfo/ddorf2010/planning.txt Mon Oct 25 11:06:01 2010
@@ -0,0 +1,27 @@
+people present:
+ Anto
+ David
+ Carl Friedrich
+ Ronny
+ Armin
+ Lukas
+
+
+topics:
+
+- get mmap to work on Beagleboard (David, Armin)
+- bridges in the ARM backend (David, Armin)
+- hg conversion (Ronny, Anto)
+- memory benchmarks (Carl Friedrich, Lukas)
+- buildbot improvements, locks, email on failing tests
+- run pypy-c-nojit in the benchmarks
+- make it possible to run benchmark on branches
+- understand problems of mapdict
+- rewrite ctypes to use _ffi
+- can classes with non-type metaclasses use typeversion?
+
+
+scheduled discussions:
+
+- hg migration planning
+- understand h?kan's branch
From david at codespeak.net Mon Oct 25 13:46:24 2010
From: david at codespeak.net (david at codespeak.net)
Date: Mon, 25 Oct 2010 13:46:24 +0200 (CEST)
Subject: [pypy-svn] r78253 -
pypy/branch/arm-backend/pypy/rpython/lltypesystem
Message-ID: <20101025114624.33CE4282C04@codespeak.net>
Author: david
Date: Mon Oct 25 13:46:22 2010
New Revision: 78253
Modified:
pypy/branch/arm-backend/pypy/rpython/lltypesystem/ll2ctypes.py
Log:
(arigo, david) Hack for mmap to use mmap64
Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/ll2ctypes.py
==============================================================================
--- pypy/branch/arm-backend/pypy/rpython/lltypesystem/ll2ctypes.py (original)
+++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/ll2ctypes.py Mon Oct 25 13:46:22 2010
@@ -528,7 +528,7 @@
return _items
_items.append(nextitem)
i += 1
-
+
items = property(getitems)
class _array_of_known_length(_array_of_unknown_length):
@@ -892,7 +892,7 @@
return clibname+'.dll'
else:
return ctypes.util.find_library('c')
-
+
libc_name = get_libc_name() # Make sure the name is determined during import, not at runtime
# XXX is this always correct???
standard_c_lib = ctypes.CDLL(get_libc_name(), **load_library_kwargs)
@@ -938,9 +938,12 @@
return lib[elem]
except AttributeError:
pass
-
+
old_eci = funcptr._obj.compilation_info
funcname = funcptr._obj._name
+ #XXX Fix this, hack for ARM
+ if funcname == 'mmap':
+ funcname = 'mmap64'
if hasattr(old_eci, '_with_ctypes'):
eci = old_eci._with_ctypes
else:
From afa at codespeak.net Mon Oct 25 14:08:12 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 25 Oct 2010 14:08:12 +0200 (CEST)
Subject: [pypy-svn] r78254 - pypy/branch/fast-forward/pypy/objspace/std
Message-ID: <20101025120812.19CFA282C06@codespeak.net>
Author: afa
Date: Mon Oct 25 14:08:11 2010
New Revision: 78254
Modified:
pypy/branch/fast-forward/pypy/objspace/std/longtype.py
Log:
Fix translation
Modified: pypy/branch/fast-forward/pypy/objspace/std/longtype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/longtype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/longtype.py Mon Oct 25 14:08:11 2010
@@ -92,8 +92,9 @@
return space.newlong(0)
def bit_length(space, w_obj):
+ bigint = space.bigint_w(w_obj)
try:
- return space.wrap(w_obj.num.bit_length())
+ return space.wrap(bigint.bit_length())
except OverflowError:
raise OperationError(space.w_OverflowError,
space.wrap("too many digits in integer"))
From afa at codespeak.net Mon Oct 25 14:13:51 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 25 Oct 2010 14:13:51 +0200 (CEST)
Subject: [pypy-svn] r78256 - in pypy/branch/fast-forward/pypy/objspace/flow:
. test
Message-ID: <20101025121351.8551B282C06@codespeak.net>
Author: afa
Date: Mon Oct 25 14:13:50 2010
New Revision: 78256
Modified:
pypy/branch/fast-forward/pypy/objspace/flow/flowcontext.py
pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py
Log:
Another R to RPython: __exit__ must not return True to swallow exceptions.
This fixes constructs like::
with c:
res = fn()
return res
otherwise the flow space finds a path where no exception is raised,
but 'res' is not initialized.
Modified: pypy/branch/fast-forward/pypy/objspace/flow/flowcontext.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/flow/flowcontext.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/flow/flowcontext.py Mon Oct 25 14:13:50 2010
@@ -445,5 +445,8 @@
# Replace it with an object which will break translation when used
# (except maybe with 'exc_typ is None')
w_typ = self.space.wrap(self.space)
- return self.space.call_function(w_func, w_typ, w_val, w_tb)
+ self.space.call_function(w_func, w_typ, w_val, w_tb)
+ # Return None so that the flow space statically knows that we didn't
+ # swallow the exception
+ return self.space.w_None
Modified: pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py Mon Oct 25 14:13:50 2010
@@ -841,12 +841,13 @@
def g(): pass
def f(c, x):
with x:
- g()
+ res = g()
+ return res
graph = self.codetest(f)
assert self.all_operations(graph) == {
'getattr': 2, # __enter__ and __exit__
'simple_call': 4, # __enter__, g and 2 possible calls to __exit__
- 'is_true': 1} # check the result of __exit__()
+ }
def monkey_patch_code(self, code, stacksize, flags, codestring, names, varnames):
c = code
From afa at codespeak.net Mon Oct 25 14:17:00 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 25 Oct 2010 14:17:00 +0200 (CEST)
Subject: [pypy-svn] r78257 - in pypy/branch/fast-forward/pypy/module: _io
_io/test thread
Message-ID: <20101025121700.A88CB282C06@codespeak.net>
Author: afa
Date: Mon Oct 25 14:16:59 2010
New Revision: 78257
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
pypy/branch/fast-forward/pypy/module/thread/os_lock.py
Log:
Implement BufferedReader.read()
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Mon Oct 25 14:16:59 2010
@@ -1,10 +1,12 @@
+from __future__ import with_statement
from pypy.interpreter.typedef import (
TypeDef, generic_new_descr)
from pypy.interpreter.gateway import interp2app, unwrap_spec
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
from pypy.interpreter.error import OperationError
from pypy.rpython.lltypesystem import lltype, rffi
-from pypy.module._io.interp_iobase import W_IOBase
+from pypy.rlib.rstring import StringBuilder
+from pypy.module._io.interp_iobase import W_IOBase, convert_size
from pypy.module._io.interp_io import DEFAULT_BUFFER_SIZE
from pypy.module.thread.os_lock import Lock
@@ -14,6 +16,9 @@
self.buffer = lltype.nullptr(rffi.CCHARP.TO)
self.lock = None
+ self.readable = False
+ self.writable = False
+
def _init(self, space):
if self.buffer_size <= 0:
raise OperationError(space.w_ValueError, space.wrap(
@@ -44,9 +49,24 @@
self.abs_pos = pos
return pos
+ def _readahead(self):
+ if self.readable and self.read_end != -1:
+ return self.read_end - self.pos
+ return 0
+
+ def _unsupportedoperation(self, space, message):
+ w_exc = space.getattr(space.getbuiltinmodule('_io'),
+ space.wrap('UnsupportedOperation'))
+ raise OperationError(w_exc, space.wrap(message))
+
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def read_w(self, space, w_size=None):
+ self._unsupportedoperation(space, "read")
+
W_BufferedIOBase.typedef = TypeDef(
'_BufferedIOBase', W_IOBase.typedef,
__new__ = generic_new_descr(W_BufferedIOBase),
+ read = interp2app(W_BufferedIOBase.read_w),
)
class W_BufferedReader(W_BufferedIOBase):
@@ -63,7 +83,6 @@
self.raw = raw
self.buffer_size = buffer_size
self.readable = True
- self.writable = False
self._init(space)
self._reset_buf()
@@ -71,10 +90,108 @@
def _reset_buf(self):
self.read_end = -1
+ def _closed(self, space):
+ return self.raw._closed(space)
+
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def read_w(self, space, w_size=None):
+ self._check_closed(space, "read of closed file")
+ size = convert_size(space, w_size)
+
+ if size < 0:
+ # read until the end of stream
+ with self.lock:
+ res = self._read_all(space)
+ else:
+ res = self._read_fast(size)
+ if res is None:
+ with self.lock:
+ res = self._read_generic(space, size)
+ return space.wrap(res)
+
+ def _read_all(self, space):
+ builder = StringBuilder()
+ # First copy what we have in the current buffer
+ current_size = self._readahead()
+ data = None
+ if current_size:
+ data = rffi.charpsize2str(rffi.ptradd(self.buffer, self.pos),
+ current_size)
+ builder.append(data)
+ self._reset_buf()
+ # We're going past the buffer's bounds, flush it
+ if self.writable:
+ self._writer_flush_unlocked(restore_pos=True)
+
+ while True:
+ # Read until EOF or until read() would block
+ w_data = space.call_method(self.raw, "read")
+ if space.is_w(w_data, space.w_None):
+ break
+ data = space.str_w(w_data)
+ size = len(data)
+ if size == 0:
+ break
+ builder.append(data)
+ current_size += size
+ if self.abs_pos != -1:
+ self.abs_pos += size
+ return builder.build()
+
+ def _read_generic(self, space, n):
+ """Generic read function: read from the stream until enough bytes are
+ read, or until an EOF occurs or until read() would block."""
+ current_size = self._readahead()
+ if n <= current_size:
+ return self._read_fast(n)
+
+ builder = StringBuilder(n)
+ remaining = n
+ written = 0
+ data = None
+ if current_size:
+ data = rffi.charpsize2str(rffi.ptradd(self.buffer, self.pos),
+ current_size)
+ builder.append(data)
+ self._reset_buf()
+
+ # XXX potential bug in CPython? The following is not enabled.
+ # We're going past the buffer's bounds, flush it
+ ## if self.writable:
+ ## self._writer_flush_unlocked(restore_pos=True)
+
+ while remaining > 0:
+ # Read until EOF or until read() would block
+ w_data = space.call_method(self.raw, "read", space.wrap(remaining))
+ if space.is_w(w_data, space.w_None):
+ break
+ data = space.str_w(w_data)
+ size = len(data)
+ if size == 0:
+ break
+ builder.append(data)
+ current_size += size
+ remaining -= size
+ if self.abs_pos != -1:
+ self.abs_pos += size
+ return builder.build()
+
+ def _read_fast(self, n):
+ """Read n bytes from the buffer if it can, otherwise return None.
+ This function is simple enough that it can run unlocked."""
+ current_size = self._readahead()
+ if n <= current_size:
+ res = rffi.charpsize2str(rffi.ptradd(self.buffer, self.pos), n)
+ self.pos += n
+ return res
+ return None
+
W_BufferedReader.typedef = TypeDef(
'BufferedReader', W_BufferedIOBase.typedef,
__new__ = generic_new_descr(W_BufferedReader),
__init__ = interp2app(W_BufferedReader.descr_init),
+
+ read = interp2app(W_BufferedReader.read_w),
)
class W_BufferedWriter(W_BufferedIOBase):
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Mon Oct 25 14:16:59 2010
@@ -1,4 +1,3 @@
-from pypy.module._io.interp_iobase import W_RawIOBase
from pypy.interpreter.typedef import (
TypeDef, interp_attrproperty, interp_attrproperty_w, GetSetProperty,
make_weakref_descr)
@@ -9,6 +8,7 @@
from pypy.rlib.rstring import StringBuilder
from os import O_RDONLY, O_WRONLY, O_RDWR, O_CREAT, O_TRUNC
import sys, os, stat, errno
+from pypy.module._io.interp_iobase import W_RawIOBase, convert_size
def interp_member_w(name, cls, doc=None):
"NOT_RPYTHON: initialization-time only"
@@ -93,12 +93,6 @@
return readable, writable, append, flags
-def convert_size(space, w_size):
- if space.is_w(w_size, space.w_None):
- return -1
- else:
- return space.int_w(w_size)
-
SMALLCHUNK = 8 * 1024
BIGCHUNK = 512 * 1024
@@ -211,10 +205,14 @@
def descr_get_mode(space, self):
return space.wrap(self._mode())
- def _check_closed(self, space):
+ def _closed(self, space):
+ return self.fd < 0
+
+ def _check_closed(self, space, message=None):
+ if message is None:
+ message = "I/O operation on closed file"
if self.fd < 0:
- raise OperationError(space.w_ValueError, space.wrap(
- "I/O operation on closed file"))
+ raise OperationError(space.w_ValueError, space.wrap(message))
def _close(self, space):
if self.fd < 0:
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Mon Oct 25 14:16:59 2010
@@ -47,11 +47,12 @@
# attribute as returned by whatever subclass.
return self.__IOBase_closed
- def _check_closed(self, space):
+ def _check_closed(self, space, message=None):
+ if message is None:
+ message = "I/O operation on closed file"
if self._closed(space):
raise OperationError(
- space.w_ValueError,
- space.wrap("I/O operation on closed file"))
+ space.w_ValueError, space.wrap(message))
def closed_get_w(space, self):
return space.newbool(self.__IOBase_closed)
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py Mon Oct 25 14:16:59 2010
@@ -9,7 +9,7 @@
self.buf = []
self.pos = 0
- def _check_closed(self, space):
+ def _check_closed(self, space, message=None):
pass
def _check_initialized(self):
pass
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Mon Oct 25 14:16:59 2010
@@ -8,7 +8,15 @@
tmpfile.write("a\nb\nc", mode='wb')
cls.w_tmpfile = cls.space.wrap(str(tmpfile))
- def test_simple(self):
+ def test_simple_read(self):
import _io
raw = _io.FileIO(self.tmpfile)
- _io.BufferedReader(raw)
+ f = _io.BufferedReader(raw)
+ assert f.read() == "a\nb\nc"
+ f.close()
+ #
+ raw.seek(0)
+ f = _io.BufferedReader(raw)
+ r = f.read(4)
+ assert r == "a\nb\n"
+ f.close()
Modified: pypy/branch/fast-forward/pypy/module/thread/os_lock.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/thread/os_lock.py (original)
+++ pypy/branch/fast-forward/pypy/module/thread/os_lock.py Mon Oct 25 14:16:59 2010
@@ -30,6 +30,7 @@
"A wrappable box around an interp-level lock object."
def __init__(self, space):
+ self.space = space
try:
self.lock = thread.allocate_lock()
except thread.error:
@@ -70,6 +71,13 @@
def descr__exit__(self, space, __args__):
self.descr_lock_release(space)
+ def __enter__(self):
+ self.descr_lock_acquire(self.space)
+ return self
+
+ def __exit__(self, *args):
+ self.descr_lock_release(self.space)
+
descr_acquire = interp2app(Lock.descr_lock_acquire,
unwrap_spec=['self', ObjSpace, int])
descr_release = interp2app(Lock.descr_lock_release,
From arigo at codespeak.net Mon Oct 25 14:42:03 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 25 Oct 2010 14:42:03 +0200 (CEST)
Subject: [pypy-svn] r78258 -
pypy/branch/leak-finder-more/pypy/module/_rawffi/test
Message-ID: <20101025124203.22A5736E363@codespeak.net>
Author: arigo
Date: Mon Oct 25 14:42:01 2010
New Revision: 78258
Modified:
pypy/branch/leak-finder-more/pypy/module/_rawffi/test/test_nested.py
Log:
Fix.
Modified: pypy/branch/leak-finder-more/pypy/module/_rawffi/test/test_nested.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/module/_rawffi/test/test_nested.py (original)
+++ pypy/branch/leak-finder-more/pypy/module/_rawffi/test/test_nested.py Mon Oct 25 14:42:01 2010
@@ -107,7 +107,6 @@
assert S.fieldoffset('x') == 0
assert S.fieldoffset('ar') == A5alignment
s = S()
- s = S()
s.x = 'G'
raises(TypeError, 's.ar')
assert s.fieldaddress('ar') == s.buffer + S.fieldoffset('ar')
From arigo at codespeak.net Mon Oct 25 14:43:04 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 25 Oct 2010 14:43:04 +0200 (CEST)
Subject: [pypy-svn] r78259 - pypy/branch/leak-finder-more/pypy/rlib
Message-ID: <20101025124304.40688282C06@codespeak.net>
Author: arigo
Date: Mon Oct 25 14:43:02 2010
New Revision: 78259
Modified:
pypy/branch/leak-finder-more/pypy/rlib/clibffi.py
Log:
Another 'track_allocation=False'.
Modified: pypy/branch/leak-finder-more/pypy/rlib/clibffi.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/rlib/clibffi.py (original)
+++ pypy/branch/leak-finder-more/pypy/rlib/clibffi.py Mon Oct 25 14:43:02 2010
@@ -432,7 +432,8 @@
flags=FUNCFLAG_CDECL):
AbstractFuncPtr.__init__(self, "callback", argtypes, restype, flags)
self.ll_closure = closureHeap.alloc()
- self.ll_userdata = lltype.malloc(USERDATA_P.TO, flavor='raw')
+ self.ll_userdata = lltype.malloc(USERDATA_P.TO, flavor='raw',
+ track_allocation=False)
self.ll_userdata.callback = rffi.llhelper(CALLBACK_TP, func)
self.ll_userdata.addarg = additional_arg
res = c_ffi_prep_closure(self.ll_closure, self.ll_cif,
@@ -447,7 +448,7 @@
closureHeap.free(self.ll_closure)
self.ll_closure = lltype.nullptr(FFI_CLOSUREP.TO)
if self.ll_userdata:
- lltype.free(self.ll_userdata, flavor='raw')
+ lltype.free(self.ll_userdata, flavor='raw', track_allocation=False)
self.ll_userdata = lltype.nullptr(USERDATA_P.TO)
class RawFuncPtr(AbstractFuncPtr):
From arigo at codespeak.net Mon Oct 25 14:44:43 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 25 Oct 2010 14:44:43 +0200 (CEST)
Subject: [pypy-svn] r78260 -
pypy/branch/leak-finder-more/pypy/module/_rawffi/test
Message-ID: <20101025124443.7512F36E363@codespeak.net>
Author: arigo
Date: Mon Oct 25 14:44:41 2010
New Revision: 78260
Modified:
pypy/branch/leak-finder-more/pypy/module/_rawffi/test/test__rawffi.py
Log:
Fix.
Modified: pypy/branch/leak-finder-more/pypy/module/_rawffi/test/test__rawffi.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/module/_rawffi/test/test__rawffi.py (original)
+++ pypy/branch/leak-finder-more/pypy/module/_rawffi/test/test__rawffi.py Mon Oct 25 14:44:41 2010
@@ -296,6 +296,7 @@
assert _rawffi.charp2string(res[0]) is None
arg1.free()
arg2.free()
+ a.free()
def test_raw_callable(self):
import _rawffi
From arigo at codespeak.net Mon Oct 25 14:47:33 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Mon, 25 Oct 2010 14:47:33 +0200 (CEST)
Subject: [pypy-svn] r78261 - pypy/branch/leak-finder-more/pypy/module/rctime
Message-ID: <20101025124733.7E788282C06@codespeak.net>
Author: arigo
Date: Mon Oct 25 14:47:31 2010
New Revision: 78261
Modified:
pypy/branch/leak-finder-more/pypy/module/rctime/interp_time.py
Log:
Fix.
Modified: pypy/branch/leak-finder-more/pypy/module/rctime/interp_time.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/module/rctime/interp_time.py (original)
+++ pypy/branch/leak-finder-more/pypy/module/rctime/interp_time.py Mon Oct 25 14:47:31 2010
@@ -69,7 +69,7 @@
CLOCKS_PER_SEC = cConfig.CLOCKS_PER_SEC
clock_t = cConfig.clock_t
tm = cConfig.tm
-glob_buf = lltype.malloc(tm, flavor='raw', zero=True)
+glob_buf = lltype.malloc(tm, flavor='raw', zero=True, immortal=True)
if cConfig.has_gettimeofday:
c_gettimeofday = external('gettimeofday', [rffi.VOIDP, rffi.VOIDP], rffi.INT)
From david at codespeak.net Mon Oct 25 16:41:21 2010
From: david at codespeak.net (david at codespeak.net)
Date: Mon, 25 Oct 2010 16:41:21 +0200 (CEST)
Subject: [pypy-svn] r78264 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101025144121.75D43282BE3@codespeak.net>
Author: david
Date: Mon Oct 25 16:41:19 2010
New Revision: 78264
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
Log:
(david, arigo) Add breakpoint instruction
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Mon Oct 25 16:41:19 2010
@@ -37,6 +37,9 @@
else:
raise NotImplentedError
+ def BKPT(self, cond=cond.AL):
+ self.write32(cond << 28 | 0x1200070)
+
def _encode_reg_list(self, instr, regs):
for reg in regs:
instr |= 0x1 << reg
@@ -87,7 +90,7 @@
def __init__(self):
map_size = 1024
- data = alloc(1024)
+ data = alloc(map_size)
self._pos = 0
self._init(data, map_size)
From david at codespeak.net Mon Oct 25 16:45:59 2010
From: david at codespeak.net (david at codespeak.net)
Date: Mon, 25 Oct 2010 16:45:59 +0200 (CEST)
Subject: [pypy-svn] r78267 -
pypy/branch/arm-backend/pypy/jit/backend/arm/tools
Message-ID: <20101025144559.82B2A282BE3@codespeak.net>
Author: david
Date: Mon Oct 25 16:45:58 2010
New Revision: 78267
Added:
pypy/branch/arm-backend/pypy/jit/backend/arm/tools/
pypy/branch/arm-backend/pypy/jit/backend/arm/tools/objdump.py (contents, props changed)
Log:
Wrapper for calling objdump on dumped memory
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/tools/objdump.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/tools/objdump.py Mon Oct 25 16:45:58 2010
@@ -0,0 +1,5 @@
+#!/usr/bin/env python
+import os
+import sys
+
+os.system('objdump -D --architecture=arm --target=binary %s' % sys.argv[1])
From david at codespeak.net Mon Oct 25 16:47:23 2010
From: david at codespeak.net (david at codespeak.net)
Date: Mon, 25 Oct 2010 16:47:23 +0200 (CEST)
Subject: [pypy-svn] r78268 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101025144723.0BCB9282BE3@codespeak.net>
Author: david
Date: Mon Oct 25 16:47:22 2010
New Revision: 78268
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py
Log:
(david, arigo) Encode and decode args and registers for jumps and bridges
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Mon Oct 25 16:47:22 2010
@@ -123,19 +123,15 @@
def gen_func_prolog(self):
self.mc.PUSH(r.callee_saved_registers)
- def gen_bootstrap_code(self, inputargs, regalloc):
- for i in range(len(inputargs)):
- reg = regalloc.try_allocate_reg(inputargs[i])
- addr = self.fail_boxes_int.get_addr_for_num(i)
- self.mc.gen_load_int(reg, addr)
- self.mc.LDR_ri(reg, reg)
-
- def gen_bridge_bootstrap_code(self, inputargs, regalloc):
+ def gen_bootstrap_code(self, inputargs, regalloc, looptoken):
+ regs = []
for i in range(len(inputargs)):
reg = regalloc.try_allocate_reg(inputargs[i])
addr = self.fail_boxes_int.get_addr_for_num(i)
self.mc.gen_load_int(reg, addr)
self.mc.LDR_ri(reg, reg)
+ regs.append(reg)
+ looptoken._arm_arglocs = regs
# cpu interface
def assemble_loop(self, inputargs, operations, looptoken):
@@ -143,7 +139,7 @@
regalloc = ARMRegisterManager(longevity, assembler=self.mc)
loop_start=self.mc.curraddr()
self.gen_func_prolog()
- self.gen_bootstrap_code(inputargs, regalloc)
+ self.gen_bootstrap_code(inputargs, regalloc, looptoken)
loop_head=self.mc.curraddr()
looptoken._arm_bootstrap_code = loop_start
looptoken._arm_loop_code = loop_head
@@ -158,11 +154,13 @@
print 'Done assembling'
def assemble_bridge(self, faildescr, inputargs, operations):
- # XXX need to restore args here
+ enc = rffi.cast(rffi.CCHARP, faildescr._failure_recovery_code)
longevity = compute_vars_longevity(inputargs, operations)
regalloc = ARMRegisterManager(longevity, assembler=self.mc)
+
+ regalloc.update_bindings(enc, inputargs)
bridge_head = self.mc.curraddr()
- self.gen_bridge_bootstrap_code(inputargs, regalloc)
+
fcond = c.AL
for op in operations:
opnum = op.getopnum()
@@ -188,14 +186,13 @@
# Resoperations
-
def emit_op_jump(self, op, regalloc, fcond):
tmp = Box()
tmpreg = regalloc.try_allocate_reg(tmp)
- inputargs = op.getdescr()._temp_inputargs
+ registers = op.getdescr()._arm_arglocs
for i in range(op.numargs()):
reg = regalloc.try_allocate_reg(op.getarg(i))
- inpreg = regalloc.try_allocate_reg(inputargs[i])
+ inpreg = registers[i]
# XXX only if every value is in a register
self.mc.MOV_rr(inpreg, reg)
loop_code = op.getdescr()._arm_loop_code
@@ -226,9 +223,9 @@
assert fcond == c.GT
descr = op.getdescr()
assert isinstance(descr, BasicFailDescr)
+ descr._arm_guard_code = self.mc.curraddr()
memaddr = self._gen_path_to_exit_path(op, op.getfailargs(), regalloc, fcond)
descr._failure_recovery_code = memaddr
- descr._arm_guard_code = self.mc.curraddr()
descr._arm_guard_cond = fcond
return c.AL
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Mon Oct 25 16:47:22 2010
@@ -11,6 +11,10 @@
def __init__(self, longevity, frame_manager=None, assembler=None):
RegisterManager.__init__(self, longevity, frame_manager, assembler)
+ def update_bindings(self, enc, inputargs):
+ for i in range(len(inputargs)):
+ self.try_allocate_reg(inputargs[i], ord(enc[i]))
+
class ARMFrameManager(FrameManager):
@staticmethod
def frame_pos(loc, type):
From fijall at gmail.com Mon Oct 25 16:50:50 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 25 Oct 2010 16:50:50 +0200
Subject: [pypy-svn] r78267 -
pypy/branch/arm-backend/pypy/jit/backend/arm/tools
In-Reply-To: <20101025144559.82B2A282BE3@codespeak.net>
References: <20101025144559.82B2A282BE3@codespeak.net>
Message-ID:
Hey.
Can we make this directory 'tool' instead of 'tools' like every other
tool directory?
Cheers,
fijal
On Mon, Oct 25, 2010 at 4:45 PM, wrote:
> Author: david
> Date: Mon Oct 25 16:45:58 2010
> New Revision: 78267
>
> Added:
> ? pypy/branch/arm-backend/pypy/jit/backend/arm/tools/
> ? pypy/branch/arm-backend/pypy/jit/backend/arm/tools/objdump.py ? (contents, props changed)
> Log:
> Wrapper for calling objdump on dumped memory
>
> Added: pypy/branch/arm-backend/pypy/jit/backend/arm/tools/objdump.py
> ==============================================================================
> --- (empty file)
> +++ pypy/branch/arm-backend/pypy/jit/backend/arm/tools/objdump.py ? ? ? Mon Oct 25 16:45:58 2010
> @@ -0,0 +1,5 @@
> +#!/usr/bin/env python
> +import os
> +import sys
> +
> +os.system('objdump -D --architecture=arm --target=binary %s' % sys.argv[1])
> _______________________________________________
> pypy-svn mailing list
> pypy-svn at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-svn
>
From david at codespeak.net Mon Oct 25 16:54:03 2010
From: david at codespeak.net (david at codespeak.net)
Date: Mon, 25 Oct 2010 16:54:03 +0200 (CEST)
Subject: [pypy-svn] r78269 - in
pypy/branch/arm-backend/pypy/jit/backend/arm: tool tools
Message-ID: <20101025145403.6F926282C06@codespeak.net>
Author: david
Date: Mon Oct 25 16:54:02 2010
New Revision: 78269
Added:
pypy/branch/arm-backend/pypy/jit/backend/arm/tool/
- copied from r78267, pypy/branch/arm-backend/pypy/jit/backend/arm/tools/
Removed:
pypy/branch/arm-backend/pypy/jit/backend/arm/tools/
Log:
tools -> tool
From david at codespeak.net Mon Oct 25 17:22:48 2010
From: david at codespeak.net (david at codespeak.net)
Date: Mon, 25 Oct 2010 17:22:48 +0200 (CEST)
Subject: [pypy-svn] r78270 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101025152248.4012B282C06@codespeak.net>
Author: david
Date: Mon Oct 25 17:22:46 2010
New Revision: 78270
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py
Log:
Unify input arg and fail boxes
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Mon Oct 25 17:22:46 2010
@@ -18,7 +18,6 @@
def __init__(self, cpu, failargs_limit=1000):
self.mc = ARMv7Builder()
self.cpu = cpu
- self.input_arg_boxes_int = values_array(lltype.Signed, failargs_limit) # merge with fail_boxes_int later
self.fail_boxes_int = values_array(lltype.Signed, failargs_limit)
self._debug_asm = True
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Mon Oct 25 17:22:46 2010
@@ -22,7 +22,7 @@
self.assembler.assemble_bridge(faildescr, inputargs, operations)
def set_future_value_int(self, index, intvalue):
- self.assembler.input_arg_boxes_int.setitem(index, intvalue)
+ self.assembler.fail_boxes_int.setitem(index, intvalue)
def get_latest_value_int(self, index):
return self.assembler.fail_boxes_int.getitem(index)
From afa at codespeak.net Mon Oct 25 17:28:24 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 25 Oct 2010 17:28:24 +0200 (CEST)
Subject: [pypy-svn] r78271 - in pypy/branch/fast-forward/pypy: module/_io
module/_io/test rlib
Message-ID: <20101025152824.553C5282C06@codespeak.net>
Author: afa
Date: Mon Oct 25 17:28:21 2010
New Revision: 78271
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
pypy/branch/fast-forward/pypy/rlib/streamio.py
Log:
Add BufferedReader.seek(),
and fix buffer management.
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Mon Oct 25 17:28:21 2010
@@ -3,17 +3,23 @@
TypeDef, generic_new_descr)
from pypy.interpreter.gateway import interp2app, unwrap_spec
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
-from pypy.interpreter.error import OperationError
+from pypy.interpreter.error import OperationError, operationerrfmt
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.rlib.rstring import StringBuilder
+from pypy.rlib.rarithmetic import r_longlong
from pypy.module._io.interp_iobase import W_IOBase, convert_size
from pypy.module._io.interp_io import DEFAULT_BUFFER_SIZE
from pypy.module.thread.os_lock import Lock
+class BlockingIOError(Exception):
+ pass
+
class W_BufferedIOBase(W_IOBase):
def __init__(self, space):
W_IOBase.__init__(self, space)
self.buffer = lltype.nullptr(rffi.CCHARP.TO)
+ self.pos = 0 # Current logical position in the buffer
+ self.raw_pos = 0 # Position of the raw stream in the buffer.
self.lock = None
self.readable = False
@@ -54,6 +60,13 @@
return self.read_end - self.pos
return 0
+ def _raw_offset(self):
+ if self.raw_pos and (
+ (self.readable and self.read_end != -1) or
+ (self.writable and self.write_end != -1)):
+ return self.raw_pos - self.pos
+ return 0
+
def _unsupportedoperation(self, space, message):
w_exc = space.getattr(space.getbuiltinmodule('_io'),
space.wrap('UnsupportedOperation'))
@@ -63,10 +76,55 @@
def read_w(self, space, w_size=None):
self._unsupportedoperation(space, "read")
+ @unwrap_spec('self', ObjSpace, r_longlong, int)
+ def seek_w(self, space, pos, whence=0):
+ if whence not in (0, 1, 2):
+ raise operationerrfmt(space.w_ValueError,
+ "whence must be between 0 and 2, not %d", whence)
+ self._check_closed(space, "seek of closed file")
+ if whence != 2 and self.readable:
+ # Check if seeking leaves us inside the current buffer, so as to
+ # return quickly if possible. Also, we needn't take the lock in
+ # this fast path.
+ current = self._raw_tell(space)
+ available = self._readahead()
+ if available > 0:
+ if whence == 0:
+ offset = pos - (current - self._raw_offset())
+ else:
+ offset = pos
+ if -self.pos <= offset <= available:
+ self.pos += offset
+ return space.wrap(current - available + offset)
+
+ # Fallback: invoke raw seek() method and clear buffer
+ with self.lock:
+ if self.writable:
+ self._writer_flush_unlocked(restore_pos=False)
+ self._writer_reset_buf()
+
+ if whence == 1:
+ pos -= self._raw_offset()
+ n = self._raw_seek(space, pos, whence)
+ self.raw_pos = -1
+ if self.readable:
+ self._reader_reset_buf()
+ return space.wrap(n)
+
+ def _raw_seek(self, space, pos, whence):
+ w_pos = space.call_method(self.raw, "seek",
+ space.wrap(pos), space.wrap(whence))
+ pos = space.r_longlong_w(w_pos)
+ if pos < 0:
+ raise OperationError(space.w_IOError, space.wrap(
+ "Raw stream returned invalid position"))
+ return pos
+
W_BufferedIOBase.typedef = TypeDef(
'_BufferedIOBase', W_IOBase.typedef,
__new__ = generic_new_descr(W_BufferedIOBase),
read = interp2app(W_BufferedIOBase.read_w),
+ seek = interp2app(W_BufferedIOBase.seek_w),
)
class W_BufferedReader(W_BufferedIOBase):
@@ -85,9 +143,9 @@
self.readable = True
self._init(space)
- self._reset_buf()
+ self._reader_reset_buf()
- def _reset_buf(self):
+ def _reader_reset_buf(self):
self.read_end = -1
def _closed(self, space):
@@ -110,6 +168,7 @@
return space.wrap(res)
def _read_all(self, space):
+ "Read all the file, don't update the cache"
builder = StringBuilder()
# First copy what we have in the current buffer
current_size = self._readahead()
@@ -118,7 +177,7 @@
data = rffi.charpsize2str(rffi.ptradd(self.buffer, self.pos),
current_size)
builder.append(data)
- self._reset_buf()
+ self._reader_reset_buf()
# We're going past the buffer's bounds, flush it
if self.writable:
self._writer_flush_unlocked(restore_pos=True)
@@ -138,6 +197,28 @@
self.abs_pos += size
return builder.build()
+ def _raw_read(self, space, n):
+ w_data = space.call_method(self.raw, "read", space.wrap(n))
+ if space.is_w(w_data, space.w_None):
+ raise BlockingIOError()
+ data = space.str_w(w_data)
+ if self.abs_pos != -1:
+ self.abs_pos += len(data)
+ return data
+
+ def _fill_buffer(self, space):
+ start = self.read_end
+ if start == -1:
+ start = 0
+ length = self.buffer_size - start
+ data = self._raw_read(space, length)
+ size = len(data)
+ if size > 0:
+ for i in range(start, start + size):
+ self.buffer[i] = data[i]
+ self.read_end = self.raw_pos = start + size
+ return size
+
def _read_generic(self, space, n):
"""Generic read function: read from the stream until enough bytes are
read, or until an EOF occurs or until read() would block."""
@@ -153,27 +234,56 @@
data = rffi.charpsize2str(rffi.ptradd(self.buffer, self.pos),
current_size)
builder.append(data)
- self._reset_buf()
+ self._reader_reset_buf()
# XXX potential bug in CPython? The following is not enabled.
# We're going past the buffer's bounds, flush it
## if self.writable:
## self._writer_flush_unlocked(restore_pos=True)
+ # Read whole blocks, and don't buffer them
while remaining > 0:
- # Read until EOF or until read() would block
- w_data = space.call_method(self.raw, "read", space.wrap(remaining))
- if space.is_w(w_data, space.w_None):
+ r = self.buffer_size * (remaining // self.buffer_size)
+ if r == 0:
break
- data = space.str_w(w_data)
+ try:
+ data = self._raw_read(space, r)
+ except BlockingIOError:
+ if written == 0:
+ return None
+ data = ""
size = len(data)
if size == 0:
- break
- builder.append(data)
- current_size += size
+ return builder.build()
remaining -= size
- if self.abs_pos != -1:
- self.abs_pos += size
+ written += size
+
+ self.pos = 0
+ self.raw_pos = 0
+ self.read_end = 0
+
+ while remaining > 0 and self.read_end < self.buffer_size:
+ # Read until EOF or until read() would block
+ try:
+ size = self._fill_buffer(space)
+ except BlockingIOError:
+ if written == 0:
+ return None
+ break
+
+ if remaining > 0:
+ if size > remaining:
+ size = remaining
+ # XXX inefficient
+ l = []
+ for i in range(self.pos,self.pos + size):
+ l.append(self.buffer[i])
+ data = ''.join(l)
+ builder.append(data)
+
+ written += size
+ self.pos += size
+ remaining -= size
return builder.build()
def _read_fast(self, n):
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Mon Oct 25 17:28:21 2010
@@ -20,3 +20,14 @@
r = f.read(4)
assert r == "a\nb\n"
f.close()
+
+ def test_seek(self):
+ import _io
+ raw = _io.FileIO(self.tmpfile)
+ f = _io.BufferedReader(raw)
+ assert f.read() == "a\nb\nc"
+ f.seek(0)
+ assert f.read() == "a\nb\nc"
+ f.seek(-2, 2)
+ assert f.read() == "\nc"
+ f.close()
Modified: pypy/branch/fast-forward/pypy/rlib/streamio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/streamio.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/streamio.py Mon Oct 25 17:28:21 2010
@@ -200,6 +200,7 @@
raise WindowsError(rwin32.GetLastError(),
"Could not truncate file")
finally:
+ # we restore the file pointer position in any case
os.lseek(fd, curpos, 0)
From david at codespeak.net Mon Oct 25 17:41:25 2010
From: david at codespeak.net (david at codespeak.net)
Date: Mon, 25 Oct 2010 17:41:25 +0200 (CEST)
Subject: [pypy-svn] r78272 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101025154125.3F4C8282C08@codespeak.net>
Author: david
Date: Mon Oct 25 17:41:23 2010
New Revision: 78272
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py
Log:
(david, arigo) handle holes in bridges
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Mon Oct 25 17:41:23 2010
@@ -49,6 +49,8 @@
while(True):
i += 1
r = enc[i]
+ if r == '\xFE':
+ continue
if r == '\xFF':
break
reg = ord(enc[i])
@@ -96,8 +98,12 @@
# XXX free this memory
mem = lltype.malloc(rffi.CArray(lltype.Char), len(args)+5, flavor='raw')
for i in range(len(args)):
- curreg = regalloc.try_allocate_reg(args[i])
- mem[i] = chr(curreg)
+ if args[i]:
+ curreg = regalloc.try_allocate_reg(args[i])
+ mem[i] = chr(curreg)
+ else:
+ mem[i] = '\xFE'
+
i = len(args)
mem[i] = chr(0xFF)
memaddr = rffi.cast(lltype.Signed, mem)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Mon Oct 25 17:41:23 2010
@@ -12,8 +12,12 @@
RegisterManager.__init__(self, longevity, frame_manager, assembler)
def update_bindings(self, enc, inputargs):
+ j = 0
for i in range(len(inputargs)):
- self.try_allocate_reg(inputargs[i], ord(enc[i]))
+ while enc[j] == '\xFE':
+ j += 1
+ self.try_allocate_reg(inputargs[i], ord(enc[j]))
+ j += 1
class ARMFrameManager(FrameManager):
@staticmethod
From david at codespeak.net Mon Oct 25 17:47:26 2010
From: david at codespeak.net (david at codespeak.net)
Date: Mon, 25 Oct 2010 17:47:26 +0200 (CEST)
Subject: [pypy-svn] r78273 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101025154726.33D0A282C08@codespeak.net>
Author: david
Date: Mon Oct 25 17:47:24 2010
New Revision: 78273
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py
Log:
(david, arigo) implement get_latest_value_count
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Mon Oct 25 17:47:24 2010
@@ -57,6 +57,7 @@
self.fail_boxes_int.setitem(i, self.decode32(stack, reg*WORD))
assert enc[i] == '\xFF'
descr = self.decode32(enc, i+1)
+ self.fail_boxes_count = i
return descr
def decode32(self, mem, index):
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Mon Oct 25 17:47:24 2010
@@ -27,6 +27,13 @@
def get_latest_value_int(self, index):
return self.assembler.fail_boxes_int.getitem(index)
+ def get_latest_value_count(self):
+ return self.assembler.fail_boxes_count
+
+ def clear_latest_values(self, count):
+ # XXX TODO
+ pass
+
def execute_token(self, executable_token):
addr = executable_token._arm_bootstrap_code
assert addr % 8 == 0
From afa at codespeak.net Mon Oct 25 18:21:01 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 25 Oct 2010 18:21:01 +0200 (CEST)
Subject: [pypy-svn] r78274 - pypy/branch/leak-finder-more/pypy/module/cpyext
Message-ID: <20101025162101.14732282BE7@codespeak.net>
Author: afa
Date: Mon Oct 25 18:20:59 2010
New Revision: 78274
Modified:
pypy/branch/leak-finder-more/pypy/module/cpyext/api.py
Log:
Fix memory leak when dlopen() fails.
Modified: pypy/branch/leak-finder-more/pypy/module/cpyext/api.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/module/cpyext/api.py (original)
+++ pypy/branch/leak-finder-more/pypy/module/cpyext/api.py Mon Oct 25 18:20:59 2010
@@ -908,8 +908,10 @@
from pypy.rlib import rdynload
try:
ll_libname = rffi.str2charp(path)
- dll = rdynload.dlopen(ll_libname)
- lltype.free(ll_libname, flavor='raw')
+ try:
+ dll = rdynload.dlopen(ll_libname)
+ finally:
+ lltype.free(ll_libname, flavor='raw')
except rdynload.DLOpenError, e:
raise operationerrfmt(
space.w_ImportError,
From afa at codespeak.net Mon Oct 25 18:22:29 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 25 Oct 2010 18:22:29 +0200 (CEST)
Subject: [pypy-svn] r78275 - pypy/branch/leak-finder-more/pypy/module/cpyext
Message-ID: <20101025162229.8630B282BE7@codespeak.net>
Author: afa
Date: Mon Oct 25 18:22:28 2010
New Revision: 78275
Modified:
pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py
Log:
This function is a @specialize.memo() and will be called only once.
Mark allocations as immortal.
Modified: pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py (original)
+++ pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py Mon Oct 25 18:22:28 2010
@@ -193,10 +193,12 @@
if state.new_method_def:
return state.new_method_def
from pypy.module.cpyext.modsupport import PyMethodDef
- ptr = lltype.malloc(PyMethodDef, flavor="raw", zero=True)
- ptr.c_ml_name = rffi.str2charp("__new__")
+ ptr = lltype.malloc(PyMethodDef, flavor="raw", zero=True,
+ immortal=True)
+ ptr.c_ml_name = rffi.str2charp_immortal("__new__")
rffi.setintfield(ptr, 'c_ml_flags', METH_VARARGS | METH_KEYWORDS)
- ptr.c_ml_doc = rffi.str2charp("T.__new__(S, ...) -> a new object with type S, a subtype of T")
+ ptr.c_ml_doc = rffi.str2charp_immortal(
+ "T.__new__(S, ...) -> a new object with type S, a subtype of T")
state.new_method_def = ptr
return ptr
From cfbolz at codespeak.net Mon Oct 25 18:46:34 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Mon, 25 Oct 2010 18:46:34 +0200 (CEST)
Subject: [pypy-svn] r78276 - in pypy/trunk/pypy/objspace/std: . test
Message-ID: <20101025164634.CA26736E393@codespeak.net>
Author: cfbolz
Date: Mon Oct 25 18:46:32 2010
New Revision: 78276
Modified:
pypy/trunk/pypy/objspace/std/test/test_typeobject.py
pypy/trunk/pypy/objspace/std/typeobject.py
Log:
(cfbolz, arigo): you learn something new every day. in cpython you can add slots
that exist as methods as well and it "works": the slot is mostly ignored.
Modified: pypy/trunk/pypy/objspace/std/test/test_typeobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/test/test_typeobject.py (original)
+++ pypy/trunk/pypy/objspace/std/test/test_typeobject.py Mon Oct 25 18:46:32 2010
@@ -1085,4 +1085,12 @@
assert b == 1
-
+ def test_slots_with_method_in_class(self):
+ # this works in cpython...
+ class A(object):
+ __slots__ = ["f"]
+ def f(self, x):
+ return x + 1
+ a = A()
+ assert a.f(1) == 2
+
Modified: pypy/trunk/pypy/objspace/std/typeobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/typeobject.py (original)
+++ pypy/trunk/pypy/objspace/std/typeobject.py Mon Oct 25 18:46:32 2010
@@ -561,9 +561,11 @@
slot_name = _mangle(slot_name, w_self.name)
# Force interning of slot names.
slot_name = space.str_w(space.new_interned_str(slot_name))
- member = Member(w_self.nslots, slot_name, w_self)
- w_self.dict_w[slot_name] = space.wrap(member)
- w_self.nslots += 1
+ if slot_name not in w_self.dict_w:
+ # in cpython it is ignored less, but we probably don't care
+ member = Member(w_self.nslots, slot_name, w_self)
+ w_self.dict_w[slot_name] = space.wrap(member)
+ w_self.nslots += 1
def create_dict_slot(w_self):
if not w_self.hasdict:
From hakanardo at codespeak.net Mon Oct 25 21:39:00 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Mon, 25 Oct 2010 21:39:00 +0200 (CEST)
Subject: [pypy-svn] r78278 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101025193900.BD000282BDA@codespeak.net>
Author: hakanardo
Date: Mon Oct 25 21:38:58 2010
New Revision: 78278
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualizable.py
Log:
Some tests ajusted to reflect the new optimization
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py Mon Oct 25 21:38:58 2010
@@ -98,7 +98,7 @@
return res
res = self.meta_interp(f, [4, -1])
assert res == 145
- self.check_loops(int_add = 1)
+ self.check_loops(int_add = 1, everywhere=True)
def test_oosend_base(self):
myjitdriver = JitDriver(greens = [], reds = ['x', 'y', 'w'])
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py Mon Oct 25 21:38:58 2010
@@ -316,10 +316,9 @@
self.check_tree_loop_count(2) # the loop and the entry path
# we get:
- # ENTER - compile the new loop
- # ENTER - compile the entry bridge
+ # ENTER - compile the new loop and entry bridge
# ENTER - compile the leaving path
- self.check_enter_count(3)
+ self.check_enter_count(2)
class VirtualMiscTests:
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualizable.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualizable.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualizable.py Mon Oct 25 21:38:58 2010
@@ -197,7 +197,7 @@
return xy.inst_x
res = self.meta_interp(f, [20])
assert res == 134
- self.check_loops(getfield_gc=1, setfield_gc=1)
+ self.check_loops(getfield_gc=1, setfield_gc=1, everywhere=True)
# ------------------------------
@@ -1124,7 +1124,7 @@
res = self.meta_interp(f, [10])
assert res == 55
- self.check_loops(new_with_vtable=0, ptr_eq=1)
+ self.check_loops(new_with_vtable=0, ptr_eq=1, everywhere=True)
def test_virtual_child_frame_with_arrays(self):
myjitdriver = JitDriver(greens = [], reds = ['frame'],
From agaynor at codespeak.net Mon Oct 25 22:56:22 2010
From: agaynor at codespeak.net (agaynor at codespeak.net)
Date: Mon, 25 Oct 2010 22:56:22 +0200 (CEST)
Subject: [pypy-svn] r78279 - in pypy/trunk/pypy: interpreter objspace/test
Message-ID: <20101025205622.8F29B282BDA@codespeak.net>
Author: agaynor
Date: Mon Oct 25 22:56:18 2010
New Revision: 78279
Modified:
pypy/trunk/pypy/interpreter/function.py
pypy/trunk/pypy/objspace/test/test_descroperation.py
Log:
Fixed instantiating subclasses of classmethod. This was found via Django, thanks to Amaury for help with the patch.
Modified: pypy/trunk/pypy/interpreter/function.py
==============================================================================
--- pypy/trunk/pypy/interpreter/function.py (original)
+++ pypy/trunk/pypy/interpreter/function.py Mon Oct 25 22:56:18 2010
@@ -589,12 +589,14 @@
w_klass = space.type(w_obj)
return space.wrap(Method(space, self.w_function, w_klass, space.w_None))
- def descr_classmethod__new__(space, w_type, w_function):
+ def descr_classmethod__new__(space, w_subtype, w_function):
if not space.is_true(space.callable(w_function)):
typename = space.type(w_function).getname(space, '?')
raise operationerrfmt(space.w_TypeError,
"'%s' object is not callable", typename)
- return space.wrap(ClassMethod(w_function))
+ instance = space.allocate_instance(ClassMethod, w_subtype)
+ instance.__init__(w_function)
+ return space.wrap(instance)
class FunctionWithFixedCode(Function):
can_change_code = False
Modified: pypy/trunk/pypy/objspace/test/test_descroperation.py
==============================================================================
--- pypy/trunk/pypy/objspace/test/test_descroperation.py (original)
+++ pypy/trunk/pypy/objspace/test/test_descroperation.py Mon Oct 25 22:56:18 2010
@@ -400,7 +400,7 @@
class D(object):
def __init__(self, a):
self.a = a
-
+
assert A(1) == B(1)
assert B(1) == A(1)
assert A(1) == C(1)
@@ -457,6 +457,20 @@
else:
assert False, "did not raise"
+ def test_attribute_error(self):
+ class classmethodonly(classmethod):
+ def __get__(self, instance, type):
+ if instance is not None:
+ raise AttributeError("Must be called on a class, not an instance.")
+ return super(classmethodonly, self).__get__(instance, type)
+
+ class A(object):
+ @classmethodonly
+ def a(cls):
+ return 3
+
+ raises(AttributeError, lambda: A().a)
+
class AppTestWithBuiltinShortcut(AppTest_Descroperation):
OPTIONS = {'objspace.std.builtinshortcut': True}
From afa at codespeak.net Mon Oct 25 23:32:35 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Mon, 25 Oct 2010 23:32:35 +0200 (CEST)
Subject: [pypy-svn] r78280 - in pypy/branch/leak-finder-more/pypy:
module/cpyext rpython/lltypesystem
Message-ID: <20101025213235.81F6F282C08@codespeak.net>
Author: afa
Date: Mon Oct 25 23:32:32 2010
New Revision: 78280
Modified:
pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py
pypy/branch/leak-finder-more/pypy/rpython/lltypesystem/llarena.py
pypy/branch/leak-finder-more/pypy/rpython/lltypesystem/lltype.py
pypy/branch/leak-finder-more/pypy/rpython/lltypesystem/rffi.py
Log:
lltype.render_immortal() can mark an allocation as immortal
after the call to lltype.malloc()
Use it to kill rffi.str2charp_immortal
Modified: pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py (original)
+++ pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py Mon Oct 25 23:32:32 2010
@@ -195,10 +195,12 @@
from pypy.module.cpyext.modsupport import PyMethodDef
ptr = lltype.malloc(PyMethodDef, flavor="raw", zero=True,
immortal=True)
- ptr.c_ml_name = rffi.str2charp_immortal("__new__")
+ ptr.c_ml_name = rffi.str2charp("__new__")
+ lltype.render_immortal(ptr.c_ml_name)
rffi.setintfield(ptr, 'c_ml_flags', METH_VARARGS | METH_KEYWORDS)
- ptr.c_ml_doc = rffi.str2charp_immortal(
+ ptr.c_ml_doc = rffi.str2charp(
"T.__new__(S, ...) -> a new object with type S, a subtype of T")
+ lltype.render_immortal(ptr.c_ml_doc)
state.new_method_def = ptr
return ptr
Modified: pypy/branch/leak-finder-more/pypy/rpython/lltypesystem/llarena.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/rpython/lltypesystem/llarena.py (original)
+++ pypy/branch/leak-finder-more/pypy/rpython/lltypesystem/llarena.py Mon Oct 25 23:32:32 2010
@@ -440,7 +440,8 @@
[rffi.INT],
rffi.INT,
sandboxsafe=True, _nowrapper=True)
- _dev_zero = rffi.str2charp_immortal('/dev/zero') # prebuilt
+ _dev_zero = rffi.str2charp('/dev/zero') # prebuilt
+ lltype.render_immortal(_dev_zero)
def clear_large_memory_chunk(baseaddr, size):
# on some Unixy platforms, reading from /dev/zero is the fastest way
Modified: pypy/branch/leak-finder-more/pypy/rpython/lltypesystem/lltype.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/rpython/lltypesystem/lltype.py (original)
+++ pypy/branch/leak-finder-more/pypy/rpython/lltypesystem/lltype.py Mon Oct 25 23:32:32 2010
@@ -1868,6 +1868,13 @@
leakfinder.remember_free(p._obj0)
p._obj0._free()
+def render_immortal(p, track_allocation=True):
+ T = typeOf(p)
+ if not isinstance(T, Ptr) or p._togckind() != 'raw':
+ raise TypeError, "free(): only for pointers to non-gc containers"
+ if track_allocation:
+ leakfinder.remember_free(p._obj0)
+
def _make_scoped_allocator(T):
class ScopedAlloc:
def __init__(self, n=None, zero=False):
Modified: pypy/branch/leak-finder-more/pypy/rpython/lltypesystem/rffi.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/rpython/lltypesystem/rffi.py (original)
+++ pypy/branch/leak-finder-more/pypy/rpython/lltypesystem/rffi.py Mon Oct 25 23:32:32 2010
@@ -607,15 +607,6 @@
return array
str2charp._annenforceargs_ = [strtype]
- def str2charp_immortal(s):
- "NOT_RPYTHON"
- array = lltype.malloc(TYPEP.TO, len(s) + 1, flavor='raw',
- immortal=True)
- for i in range(len(s)):
- array[i] = s[i]
- array[len(s)] = lastchar
- return array
-
def free_charp(cp):
lltype.free(cp, flavor='raw')
@@ -734,19 +725,19 @@
l = [cp[i] for i in range(size)]
return emptystr.join(l)
- return (str2charp, str2charp_immortal, free_charp, charp2str,
+ return (str2charp, free_charp, charp2str,
get_nonmovingbuffer, free_nonmovingbuffer,
alloc_buffer, str_from_buffer, keep_buffer_alive_until_here,
charp2strn, charpsize2str,
)
-(str2charp, str2charp_immortal, free_charp, charp2str,
+(str2charp, free_charp, charp2str,
get_nonmovingbuffer, free_nonmovingbuffer,
alloc_buffer, str_from_buffer, keep_buffer_alive_until_here,
charp2strn, charpsize2str,
) = make_string_mappings(str)
-(unicode2wcharp, unicode2wcharp_immortal, free_wcharp, wcharp2unicode,
+(unicode2wcharp, free_wcharp, wcharp2unicode,
get_nonmoving_unicodebuffer, free_nonmoving_unicodebuffer,
alloc_unicodebuffer, unicode_from_buffer, keep_unicodebuffer_alive_until_here,
wcharp2unicoden, wcharpsize2unicode,
From afa at codespeak.net Tue Oct 26 09:30:03 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 26 Oct 2010 09:30:03 +0200 (CEST)
Subject: [pypy-svn] r78281 - in pypy/branch/leak-finder-more/pypy:
annotation module/cpyext
Message-ID: <20101026073003.A6009282C14@codespeak.net>
Author: afa
Date: Tue Oct 26 09:29:56 2010
New Revision: 78281
Modified:
pypy/branch/leak-finder-more/pypy/annotation/builtin.py
pypy/branch/leak-finder-more/pypy/module/cpyext/cdatetime.py
pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py
Log:
Reduce the number of reported leaks in the cpyext module,
and try to fix translation
Modified: pypy/branch/leak-finder-more/pypy/annotation/builtin.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/annotation/builtin.py (original)
+++ pypy/branch/leak-finder-more/pypy/annotation/builtin.py Tue Oct 26 09:29:56 2010
@@ -453,6 +453,9 @@
#p = lltype.malloc(T, flavor=s_flavor.const)
#lltype.free(p, flavor=s_flavor.const)
+def render_immortal(s_p, s_track_allocation=None):
+ assert s_track_allocation is None or s_track_allocation.is_constant()
+
def typeOf(s_val):
lltype = annotation_to_lltype(s_val, info="in typeOf(): ")
return immutablevalue(lltype)
@@ -520,6 +523,7 @@
BUILTIN_ANALYZERS[lltype.malloc] = malloc
BUILTIN_ANALYZERS[lltype.free] = free
+BUILTIN_ANALYZERS[lltype.render_immortal] = render_immortal
BUILTIN_ANALYZERS[lltype.typeOf] = typeOf
BUILTIN_ANALYZERS[lltype.cast_primitive] = cast_primitive
BUILTIN_ANALYZERS[lltype.nullptr] = nullptr
Modified: pypy/branch/leak-finder-more/pypy/module/cpyext/cdatetime.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/module/cpyext/cdatetime.py (original)
+++ pypy/branch/leak-finder-more/pypy/module/cpyext/cdatetime.py Tue Oct 26 09:29:56 2010
@@ -4,7 +4,7 @@
from pypy.module.cpyext.api import (
cpython_api, CANNOT_FAIL, cpython_struct, PyObjectFields)
from pypy.module.cpyext.import_ import PyImport_Import
-from pypy.module.cpyext.typeobject import PyTypeObjectPtr
+from pypy.module.cpyext.typeobject import PyTypeObjectPtr, render_immortal
from pypy.module.cpyext.state import State
from pypy.interpreter.error import OperationError
from pypy.tool.sourcetools import func_renamer
@@ -22,25 +22,34 @@
@cpython_api([], lltype.Ptr(PyDateTime_CAPI),
error=lltype.nullptr(PyDateTime_CAPI))
def _PyDateTime_Import(space):
- datetimeAPI = lltype.malloc(PyDateTime_CAPI, flavor='raw')
+ datetimeAPI = lltype.malloc(PyDateTime_CAPI, flavor='raw',
+ track_allocation=False)
if not we_are_translated():
datetimeAPI_dealloc(space)
space.fromcache(State).datetimeAPI = datetimeAPI
w_datetime = PyImport_Import(space, space.wrap("datetime"))
+
w_type = space.getattr(w_datetime, space.wrap("date"))
datetimeAPI.c_DateType = rffi.cast(
PyTypeObjectPtr, make_ref(space, w_type))
+ render_immortal(datetimeAPI.c_DateType, w_type)
+
w_type = space.getattr(w_datetime, space.wrap("datetime"))
datetimeAPI.c_DateTimeType = rffi.cast(
PyTypeObjectPtr, make_ref(space, w_type))
+ render_immortal(datetimeAPI.c_DateTimeType, w_type)
+
w_type = space.getattr(w_datetime, space.wrap("time"))
datetimeAPI.c_TimeType = rffi.cast(
PyTypeObjectPtr, make_ref(space, w_type))
+ render_immortal(datetimeAPI.c_TimeType, w_type)
+
w_type = space.getattr(w_datetime, space.wrap("timedelta"))
datetimeAPI.c_DeltaType = rffi.cast(
PyTypeObjectPtr, make_ref(space, w_type))
+ render_immortal(datetimeAPI.c_DeltaType, w_type)
return datetimeAPI
Modified: pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py (original)
+++ pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py Tue Oct 26 09:29:56 2010
@@ -538,12 +538,23 @@
w_obj.ready()
finish_type_2(space, py_type, w_obj)
+ render_immortal(py_type, w_obj)
state = space.fromcache(RefcountState)
state.non_heaptypes_w.append(w_obj)
return w_obj
+def render_immortal(py_type, w_obj):
+ lltype.render_immortal(py_type.c_tp_bases)
+ lltype.render_immortal(py_type.c_tp_mro)
+
+ assert isinstance(w_obj, W_TypeObject)
+ if w_obj.is_cpytype():
+ lltype.render_immortal(py_type.c_tp_dict)
+ else:
+ lltype.render_immortal(py_type.c_tp_name)
+
def finish_type_1(space, pto):
"""
Sets up tp_bases, necessary before creating the interpreter type.
From cfbolz at codespeak.net Tue Oct 26 10:27:09 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Tue, 26 Oct 2010 10:27:09 +0200 (CEST)
Subject: [pypy-svn] r78282 - pypy/extradoc/sprintinfo/ddorf2010
Message-ID: <20101026082709.35A47282C16@codespeak.net>
Author: cfbolz
Date: Tue Oct 26 10:27:05 2010
New Revision: 78282
Modified:
pypy/extradoc/sprintinfo/ddorf2010/planning.txt
Log:
(all) planning for today
Modified: pypy/extradoc/sprintinfo/ddorf2010/planning.txt
==============================================================================
--- pypy/extradoc/sprintinfo/ddorf2010/planning.txt (original)
+++ pypy/extradoc/sprintinfo/ddorf2010/planning.txt Tue Oct 26 10:27:05 2010
@@ -9,14 +9,15 @@
topics:
-- get mmap to work on Beagleboard (David, Armin)
-- bridges in the ARM backend (David, Armin)
-- hg conversion (Ronny, Anto)
-- memory benchmarks (Carl Friedrich, Lukas)
+- get mmap to work on Beagleboard DONE
+- bridges in the ARM backend SORT OF DONE
+- add spilling and location support (David, Armin around)
+- hg conversion WE HAVE A PLAN (Anto, Ronny)
+- memory benchmarks IN PROGRESS (Carl Friedrich, Lukas when he shows up)
- buildbot improvements, locks, email on failing tests
- run pypy-c-nojit in the benchmarks
- make it possible to run benchmark on branches
-- understand problems of mapdict
+- understand problems of mapdict (Carl Friedrich, Armin)
- rewrite ctypes to use _ffi
- can classes with non-type metaclasses use typeversion?
@@ -25,3 +26,4 @@
- hg migration planning
- understand h?kan's branch
+- JIT introduction (today after lunch)
From afa at codespeak.net Tue Oct 26 11:19:06 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 26 Oct 2010 11:19:06 +0200 (CEST)
Subject: [pypy-svn] r78283 - in pypy/branch/leak-finder-more/pypy: rpython
translator/c/test
Message-ID: <20101026091906.F1FB8282C18@codespeak.net>
Author: afa
Date: Tue Oct 26 11:19:05 2010
New Revision: 78283
Modified:
pypy/branch/leak-finder-more/pypy/rpython/rbuiltin.py
pypy/branch/leak-finder-more/pypy/translator/c/test/test_lltyped.py
Log:
when translated, lltype.render_immortal() generates a call to the "track_alloc_stop" operation.
Modified: pypy/branch/leak-finder-more/pypy/rpython/rbuiltin.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/rpython/rbuiltin.py (original)
+++ pypy/branch/leak-finder-more/pypy/rpython/rbuiltin.py Tue Oct 26 11:19:05 2010
@@ -386,6 +386,14 @@
hop.exception_cannot_occur()
hop.genop('free', vlist)
+def rtype_render_immortal(hop, i_track_allocation=None):
+ vlist = [hop.inputarg(hop.args_r[0], arg=0)]
+ v_track_allocation = parse_kwds(hop,
+ (i_track_allocation, None))
+ hop.exception_cannot_occur()
+ if i_track_allocation is None or v_track_allocation.value:
+ hop.genop('track_alloc_stop', vlist)
+
def rtype_const_result(hop):
hop.exception_cannot_occur()
return hop.inputconst(hop.r_result.lowleveltype, hop.s_result.const)
@@ -523,6 +531,7 @@
BUILTIN_TYPER[lltype.malloc] = rtype_malloc
BUILTIN_TYPER[lltype.free] = rtype_free
+BUILTIN_TYPER[lltype.render_immortal] = rtype_render_immortal
BUILTIN_TYPER[lltype.cast_primitive] = rtype_cast_primitive
BUILTIN_TYPER[lltype.cast_pointer] = rtype_cast_pointer
BUILTIN_TYPER[lltype.cast_opaque_ptr] = rtype_cast_opaque_ptr
Modified: pypy/branch/leak-finder-more/pypy/translator/c/test/test_lltyped.py
==============================================================================
--- pypy/branch/leak-finder-more/pypy/translator/c/test/test_lltyped.py (original)
+++ pypy/branch/leak-finder-more/pypy/translator/c/test/test_lltyped.py Tue Oct 26 11:19:05 2010
@@ -841,3 +841,17 @@
assert res == -98765432
res = fn(1)
assert res == -9999999
+
+ def test_render_immortal(self):
+ A = FixedSizeArray(Signed, 1)
+ a1 = malloc(A, flavor='raw')
+ render_immortal(a1)
+ a1[0] = 42
+ def llf():
+ a2 = malloc(A, flavor='raw')
+ render_immortal(a2)
+ a2[0] = 3
+ return a1[0] + a2[0]
+ fn = self.getcompiled(llf)
+ assert fn() == 45
+
From david at codespeak.net Tue Oct 26 11:36:43 2010
From: david at codespeak.net (david at codespeak.net)
Date: Tue, 26 Oct 2010 11:36:43 +0200 (CEST)
Subject: [pypy-svn] r78284 - in
pypy/branch/arm-backend/pypy/jit/backend/arm: . test
Message-ID: <20101026093643.BAAEF282C18@codespeak.net>
Author: david
Date: Tue Oct 26 11:36:42 2010
New Revision: 78284
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py
pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
Log:
Introduce locations for registers and replace register numbers with the locations
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Tue Oct 26 11:36:42 2010
@@ -80,16 +80,16 @@
functype = lltype.Ptr(lltype.FuncType([lltype.Signed, lltype.Signed], lltype.Signed))
decode_registers_addr = llhelper(functype, self.failure_recovery_func)
self.mc.PUSH(range(12)) # registers r0 .. r11
- self.mc.MOV_rr(r.r0, r.lr) # move mem block address, to r0 to pass as
+ self.mc.MOV_rr(r.r0.value, r.lr.value) # move mem block address, to r0 to pass as
# parameter to next procedure call
- self.mc.MOV_rr(r.r1, r.sp) # pass the current stack pointer as second param
- self.mc.gen_load_int(r.r2, rffi.cast(lltype.Signed, decode_registers_addr))
- self.mc.gen_load_int(r.lr, self.mc.curraddr()+self.mc.size_of_gen_load_int+WORD)
- self.mc.MOV_rr(r.pc, r.r2)
- self.mc.MOV_rr(r.ip, r.r0)
- self.mc.LDM(r.sp, range(12), w=1) # XXX Replace with POP instr. someday
+ self.mc.MOV_rr(r.r1.value, r.sp.value) # pass the current stack pointer as second param
+ self.mc.gen_load_int(r.r2.value, rffi.cast(lltype.Signed, decode_registers_addr))
+ self.mc.gen_load_int(r.lr.value, self.mc.curraddr()+self.mc.size_of_gen_load_int+WORD)
+ self.mc.MOV_rr(r.pc.value, r.r2.value)
+ self.mc.MOV_rr(r.ip.value, r.r0.value)
+ self.mc.LDM(r.sp.value, range(12), w=1) # XXX Replace with POP instr. someday
- self.mc.MOV_rr(r.r0, r.ip)
+ self.mc.MOV_rr(r.r0.value, r.ip.value)
self.gen_func_epilog()
@@ -101,7 +101,7 @@
for i in range(len(args)):
if args[i]:
curreg = regalloc.try_allocate_reg(args[i])
- mem[i] = chr(curreg)
+ mem[i] = chr(curreg.value)
else:
mem[i] = '\xFE'
@@ -112,11 +112,14 @@
n = self.cpu.get_fail_descr_number(op.getdescr())
self.encode32(mem, i+1, n)
- self.mc.gen_load_int(r.lr, memaddr, cond=fcond)
- self.mc.gen_load_int(reg, self.mc.baseaddr(), cond=fcond)
- self.mc.MOV_rr(r.pc, reg, cond=fcond)
-
- op.getdescr()._arm_guard_reg = reg
+ self.mc.gen_load_int(r.lr.value, memaddr, cond=fcond)
+ self.mc.gen_load_int(reg.value, self.mc.baseaddr(), cond=fcond)
+ self.mc.MOV_rr(r.pc.value, reg.value, cond=fcond)
+
+ # This register is used for patching when assembling a bridge
+ # guards going to be patched are allways conditional
+ if fcond != c.AL:
+ op.getdescr()._arm_guard_reg = reg
return memaddr
def align(self):
@@ -124,18 +127,18 @@
self.mc.writechar(chr(0))
def gen_func_epilog(self,cond=c.AL):
- self.mc.LDM(r.sp, r.callee_restored_registers, cond=cond, w=1)
+ self.mc.LDM(r.sp.value, [reg.value for reg in r.callee_restored_registers], cond=cond, w=1)
def gen_func_prolog(self):
- self.mc.PUSH(r.callee_saved_registers)
+ self.mc.PUSH([reg.value for reg in r.callee_saved_registers])
def gen_bootstrap_code(self, inputargs, regalloc, looptoken):
regs = []
for i in range(len(inputargs)):
reg = regalloc.try_allocate_reg(inputargs[i])
addr = self.fail_boxes_int.get_addr_for_num(i)
- self.mc.gen_load_int(reg, addr)
- self.mc.LDR_ri(reg, reg)
+ self.mc.gen_load_int(reg.value, addr)
+ self.mc.LDR_ri(reg.value, reg.value)
regs.append(reg)
looptoken._arm_arglocs = regs
@@ -187,8 +190,8 @@
fcond = faildescr._arm_guard_cond
b = ARMv7InMemoryBuilder(faildescr._arm_guard_code, faildescr._arm_guard_code+100)
reg = faildescr._arm_guard_reg
- b.gen_load_int(reg, bridge_addr, fcond)
- b.MOV_rr(r.pc, reg, cond=fcond)
+ b.gen_load_int(reg.value, bridge_addr, fcond)
+ b.MOV_rr(r.pc.value, reg.value, cond=fcond)
# Resoperations
@@ -200,10 +203,10 @@
reg = regalloc.try_allocate_reg(op.getarg(i))
inpreg = registers[i]
# XXX only if every value is in a register
- self.mc.MOV_rr(inpreg, reg)
+ self.mc.MOV_rr(inpreg.value, reg.value)
loop_code = op.getdescr()._arm_loop_code
- self.mc.gen_load_int(tmpreg, loop_code)
- self.mc.MOV_rr(r.pc, tmpreg)
+ self.mc.gen_load_int(tmpreg.value, loop_code)
+ self.mc.MOV_rr(r.pc.value, tmpreg.value)
regalloc.possibly_free_var(tmpreg)
return fcond
@@ -214,14 +217,14 @@
def emit_op_int_le(self, op, regalloc, fcond):
reg = regalloc.try_allocate_reg(op.getarg(0))
assert isinstance(op.getarg(1), ConstInt)
- self.mc.CMP(reg, op.getarg(1).getint())
+ self.mc.CMP(reg.value, op.getarg(1).getint())
return c.GT
def emit_op_int_add(self, op, regalloc, fcond):
reg = regalloc.try_allocate_reg(op.getarg(0))
res = regalloc.try_allocate_reg(op.result)
assert isinstance(op.getarg(1), ConstInt)
- self.mc.ADD_ri(res, reg, op.getarg(1).getint())
+ self.mc.ADD_ri(res.value, reg.value, op.getarg(1).getint())
regalloc.possibly_free_vars_for_op(op)
return fcond
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Tue Oct 26 11:36:42 2010
@@ -70,15 +70,18 @@
size_of_gen_load_int = 7 * WORD
def gen_load_int(self, r, value, cond=cond.AL):
- assert r != reg.ip, 'ip is used to load int'
+ """r is the register number, value is the value to be loaded to the
+ register"""
+ assert r != reg.ip.value, 'ip is used to load int'
self.MOV_ri(r, (value & 0xFF), cond=cond)
+ ip = reg.ip.value
for offset in range(8, 25, 8):
t = (value >> offset) & 0xFF
#if t == 0:
# continue
- self.MOV_ri(reg.ip, t, cond=cond)
- self.ORR_rr(r, r, reg.ip, offset, cond=cond)
+ self.MOV_ri(ip, t, cond=cond)
+ self.ORR_rr(r, r, ip, offset, cond=cond)
class ARMv7InMemoryBuilder(AbstractARMv7Builder):
def __init__(self, start, end):
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Tue Oct 26 11:36:42 2010
@@ -5,7 +5,7 @@
class ARMRegisterManager(RegisterManager):
all_regs = r.all_regs
box_types = None # or a list of acceptable types
- no_lower_byte_regs = r.all_regs
+ no_lower_byte_regs = all_regs
save_around_call_regs = all_regs
def __init__(self, longevity, frame_manager=None, assembler=None):
@@ -16,7 +16,7 @@
for i in range(len(inputargs)):
while enc[j] == '\xFE':
j += 1
- self.try_allocate_reg(inputargs[i], ord(enc[j]))
+ self.try_allocate_reg(inputargs[i], r.all_regs[ord(enc[j])])
j += 1
class ARMFrameManager(FrameManager):
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py Tue Oct 26 11:36:42 2010
@@ -1,12 +1,17 @@
-r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11, r12, r13, r14, r15 = range(16)
+from pypy.jit.backend.arm.locations import RegisterLocation
+
+registers = [RegisterLocation(i) for i in range(16)]
+r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11, r12, r13, r14, r15 = registers
+
# aliases for registers
-fp = 11
-ip = 12
-sp = 13
-lr = 14
-pc = 15
+fp = r11
+ip = r12
+sp = r13
+lr = r14
+pc = r15
+
+all_regs = registers[:12]
-all_regs = range(12)
callee_resp = [r4, r5, r6, r7, r8, r9, r10, r11]
callee_saved_registers = callee_resp+[lr]
callee_restored_registers = callee_resp+[pc]
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py Tue Oct 26 11:36:42 2010
@@ -21,78 +21,78 @@
def test_load_small_int_to_reg(self):
self.a.gen_func_prolog()
- self.a.mc.gen_load_int(r.r0, 123)
+ self.a.mc.gen_load_int(r.r0.value, 123)
self.a.gen_func_epilog()
assert run_asm(self.a) == 123
def test_load_medium_int_to_reg(self):
self.a.gen_func_prolog()
- self.a.mc.gen_load_int(r.r0, 0xBBD7)
+ self.a.mc.gen_load_int(r.r0.value, 0xBBD7)
self.a.gen_func_epilog()
assert run_asm(self.a) == 48087
def test_load_int_to_reg(self):
self.a.gen_func_prolog()
- self.a.mc.gen_load_int(r.r0, 0xFFFFFF85)
+ self.a.mc.gen_load_int(r.r0.value, 0xFFFFFF85)
self.a.gen_func_epilog()
assert run_asm(self.a) == -123
def test_or(self):
self.a.gen_func_prolog()
- self.a.mc.MOV_ri(r.r1, 8)
- self.a.mc.MOV_ri(r.r2, 8)
- self.a.mc.ORR_rr(r.r0, r.r1, r.r2, 4)
+ self.a.mc.MOV_ri(r.r1.value, 8)
+ self.a.mc.MOV_ri(r.r2.value, 8)
+ self.a.mc.ORR_rr(r.r0.value, r.r1.value, r.r2.value, 4)
self.a.gen_func_epilog()
assert run_asm(self.a) == 0x88
def test_sub(self):
self.a.gen_func_prolog()
- self.a.mc.gen_load_int(r.r1, 123456)
- self.a.mc.SUB_ri(r.r0, r.r1, 123)
+ self.a.mc.gen_load_int(r.r1.value, 123456)
+ self.a.mc.SUB_ri(r.r0.value, r.r1.value, 123)
self.a.gen_func_epilog()
assert run_asm(self.a) == 123333
def test_cmp(self):
self.a.gen_func_prolog()
- self.a.mc.gen_load_int(r.r1, 22)
- self.a.mc.CMP(r.r1, 123)
- self.a.mc.MOV_ri(r.r0, 1, c.LE)
- self.a.mc.MOV_ri(r.r0, 0, c.GT)
+ self.a.mc.gen_load_int(r.r1.value, 22)
+ self.a.mc.CMP(r.r1.value, 123)
+ self.a.mc.MOV_ri(r.r0.value, 1, c.LE)
+ self.a.mc.MOV_ri(r.r0.value, 0, c.GT)
self.a.gen_func_epilog()
assert run_asm(self.a) == 1
def test_int_le_false(self):
self.a.gen_func_prolog()
- self.a.mc.gen_load_int(r.r1, 2222)
- self.a.mc.CMP(r.r1, 123)
- self.a.mc.MOV_ri(r.r0, 1, c.LE)
- self.a.mc.MOV_ri(r.r0, 0, c.GT)
+ self.a.mc.gen_load_int(r.r1.value, 2222)
+ self.a.mc.CMP(r.r1.value, 123)
+ self.a.mc.MOV_ri(r.r0.value, 1, c.LE)
+ self.a.mc.MOV_ri(r.r0.value, 0, c.GT)
self.a.gen_func_epilog()
assert run_asm(self.a) == 0
def test_simple_jump(self):
self.a.gen_func_prolog()
- self.a.mc.MOV_ri(r.r1, 1)
+ self.a.mc.MOV_ri(r.r1.value, 1)
loop_head = self.a.mc.curraddr()
- self.a.mc.CMP(r.r1, 0) # z=0, z=1
- self.a.mc.MOV_ri(r.r1, 0, cond=c.NE)
- self.a.mc.MOV_ri(r.r1, 7, cond=c.EQ)
- self.a.mc.gen_load_int(r.r4, loop_head, cond=c.NE)
- self.a.mc.MOV_rr(r.pc, r.r4, cond=c.NE)
- self.a.mc.MOV_rr(r.r0, r.r1)
+ self.a.mc.CMP(r.r1.value, 0) # z=0, z=1
+ self.a.mc.MOV_ri(r.r1.value, 0, cond=c.NE)
+ self.a.mc.MOV_ri(r.r1.value, 7, cond=c.EQ)
+ self.a.mc.gen_load_int(r.r4.value, loop_head, cond=c.NE)
+ self.a.mc.MOV_rr(r.pc.value, r.r4.value, cond=c.NE)
+ self.a.mc.MOV_rr(r.r0.value, r.r1.value)
self.a.gen_func_epilog()
assert run_asm(self.a) == 7
def test_jump(self):
self.a.gen_func_prolog()
- self.a.mc.MOV_ri(r.r1, 1)
+ self.a.mc.MOV_ri(r.r1.value, 1)
loop_head = self.a.mc.curraddr()
- self.a.mc.ADD_ri(r.r1, r.r1, 1)
- self.a.mc.CMP(r.r1, 9)
- self.a.mc.gen_load_int(r.r4, loop_head, cond=c.NE)
- self.a.mc.MOV_rr(r.pc, r.r4, cond=c.NE)
- self.a.mc.MOV_rr(r.r0, r.r1)
+ self.a.mc.ADD_ri(r.r1.value, r.r1.value, 1)
+ self.a.mc.CMP(r.r1.value, 9)
+ self.a.mc.gen_load_int(r.r4.value, loop_head, cond=c.NE)
+ self.a.mc.MOV_rr(r.pc.value, r.r4.value, cond=c.NE)
+ self.a.mc.MOV_rr(r.r0.value, r.r1.value)
self.a.gen_func_epilog()
assert run_asm(self.a) == 9
@@ -101,10 +101,10 @@
functype = lltype.Ptr(lltype.FuncType([lltype.Signed], lltype.Signed))
call_addr = rffi.cast(lltype.Signed, llhelper(functype, callme))
self.a.gen_func_prolog()
- self.a.mc.MOV_ri(r.r0, 123)
- self.a.mc.gen_load_int(r.r1, call_addr)
- self.a.mc.gen_load_int(r.lr, self.a.mc.curraddr()+self.a.mc.size_of_gen_load_int+WORD)
- self.a.mc.MOV_rr(r.pc, r.r1)
+ self.a.mc.MOV_ri(r.r0.value, 123)
+ self.a.mc.gen_load_int(r.r1.value, call_addr)
+ self.a.mc.gen_load_int(r.lr.value, self.a.mc.curraddr()+self.a.mc.size_of_gen_load_int+WORD)
+ self.a.mc.MOV_rr(r.pc.value, r.r1.value)
self.a.gen_func_epilog()
assert run_asm(self.a) == 133
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Tue Oct 26 11:36:42 2010
@@ -25,97 +25,97 @@
self.cb = CodeBuilder()
def test_ldr(self):
- self.cb.LDR_ri(r.r0, r.r1)
+ self.cb.LDR_ri(r.r0.value, r.r1.value)
self.assert_equal('LDR r0, [r1]')
def test_ldr_neg(self):
- self.cb.LDR_ri(r.r3, r.fp, -16)
+ self.cb.LDR_ri(r.r3.value, r.fp.value, -16)
self.assert_equal('LDR r3, [fp, #-16]')
def test_add_ri(self):
- self.cb.ADD_ri(r.r0, r.r1, 1)
+ self.cb.ADD_ri(r.r0.value, r.r1.value, 1)
self.assert_equal('ADD r0, r1, #1')
def test_mov_rr(self):
- self.cb.MOV_rr(r.r7, r.r12)
+ self.cb.MOV_rr(r.r7.value, r.r12.value)
self.assert_equal('MOV r7, r12')
def test_mov_ri(self):
- self.cb.MOV_ri(r.r9, 123)
+ self.cb.MOV_ri(r.r9.value, 123)
self.assert_equal('MOV r9, #123')
def test_mov_ri2(self):
- self.cb.MOV_ri(r.r9, 255)
+ self.cb.MOV_ri(r.r9.value, 255)
self.assert_equal('MOV r9, #255')
def test_mov_ri_max(self):
py.test.skip('Check the actual largest thing')
- self.cb.MOV_ri(r.r9, 0xFFF)
+ self.cb.MOV_ri(r.r9.value, 0xFFF)
self.assert_equal('MOV r9, #4095')
def test_str_ri(self):
- self.cb.STR_ri(r.r9, r.r14)
+ self.cb.STR_ri(r.r9.value, r.r14.value)
self.assert_equal('STR r9, [r14]')
def test_str_ri_offset(self):
- self.cb.STR_ri(r.r9, r.r14, 23)
+ self.cb.STR_ri(r.r9.value, r.r14.value, 23)
self.assert_equal('STR r9, [r14, #23]')
def test_str_ri_offset(self):
- self.cb.STR_ri(r.r9, r.r14, -20)
+ self.cb.STR_ri(r.r9.value, r.r14.value, -20)
self.assert_equal('STR r9, [r14, #-20]')
def test_asr_ri(self):
- self.cb.ASR_ri(r.r7, r.r5, 24)
+ self.cb.ASR_ri(r.r7.value, r.r5.value, 24)
self.assert_equal('ASR r7, r5, #24')
def test_orr_rr_no_shift(self):
- self.cb.ORR_rr(r.r0, r.r7,r.r12)
+ self.cb.ORR_rr(r.r0.value, r.r7.value,r.r12.value)
self.assert_equal('ORR r0, r7, r12')
def test_orr_rr_lsl_8(self):
- self.cb.ORR_rr(r.r0, r.r7,r.r12, 8)
+ self.cb.ORR_rr(r.r0.value, r.r7.value,r.r12.value, 8)
self.assert_equal('ORR r0, r7, r12, lsl #8')
def test_push_one_reg(self):
- self.cb.PUSH([r.r1])
+ self.cb.PUSH([r.r1.value])
self.assert_equal('PUSH {r1}')
def test_push_multiple(self):
- self.cb.PUSH([r.r1, r.r3, r.r6, r.r8, r.pc])
+ self.cb.PUSH([reg.value for reg in [r.r1, r.r3, r.r6, r.r8, r.pc]])
self.assert_equal('PUSH {r1, r3, r6, r8, pc}')
def test_push_multiple2(self):
- self.cb.PUSH([r.fp, r.ip, r.lr, r.pc])
+ self.cb.PUSH([reg.value for reg in [r.fp, r.ip, r.lr, r.pc]])
self.assert_equal('PUSH {fp, ip, lr, pc}')
def test_ldm_one_reg(self):
- self.cb.LDM(r.sp, [r.fp])
+ self.cb.LDM(r.sp.value, [r.fp.value])
self.assert_equal('LDM sp, {fp}')
def test_ldm_multiple_reg(self):
- self.cb.LDM(r.sp, [r.fp, r.ip, r.lr])
+ self.cb.LDM(r.sp.value, [reg.value for reg in [r.fp, r.ip, r.lr]])
self.assert_equal('LDM sp, {fp, ip, lr}')
def test_ldm_multiple_reg2(self):
- self.cb.LDM(r.sp, [r.fp, r.sp, r.pc])
+ self.cb.LDM(r.sp.value, [reg.value for reg in [r.fp, r.sp, r.pc]])
self.assert_equal("LDM sp, {fp, sp, pc}")
def test_sub_ri(self):
- self.cb.SUB_ri(r.r2, r.r4, 123)
+ self.cb.SUB_ri(r.r2.value, r.r4.value, 123)
self.assert_equal('SUB r2, r4, #123')
def test_sub_ri2(self):
py.test.skip('XXX check the actual largest value')
- self.cb.SUB_ri(r.r3, r.r7, 0xFFF)
+ self.cb.SUB_ri(r.r3.value, r.r7.value, 0xFFF)
self.assert_equal('SUB r3, r7, #4095')
def test_cmp_ri(self):
- self.cb.CMP(r.r3, 123)
+ self.cb.CMP(r.r3.value, 123)
self.assert_equal('CMP r3, #123')
def test_mcr(self):
- self.cb.MCR(15, 0, r.r1, 7, 10,0)
+ self.cb.MCR(15, 0, r.r1.value, 7, 10,0)
self.assert_equal('MCR P15, 0, r1, c7, c10, 0')
@@ -149,31 +149,31 @@
if table['result'] and table['base']:
def f(self):
func = getattr(self.cb, name)
- func(r.r3, r.r7, 23)
+ func(r.r3.value, r.r7.value, 23)
self.assert_equal('%s r3, r7, #23' % name[:name.index('_')])
elif not table['base']:
def f(self):
func = getattr(self.cb, name)
- func(r.r3, 23)
+ func(r.r3.value, 23)
self.assert_equal('%s r3, #23' % name[:name.index('_')])
else:
def f(self):
func = getattr(self.cb, name)
- func(r.r3, 23)
+ func(r.r3.value, 23)
self.assert_equal('%s r3, #23' % name[:name.index('_')])
return f
def gen_test_imm_func(name, table):
def f(self):
func = getattr(self.cb, name)
- func(r.r3, r.r7, 23)
+ func(r.r3.value, r.r7.value, 23)
self.assert_equal('%s r3, [r7, #23]' % name[:name.index('_')])
return f
def gen_test_reg_func(name, table):
def f(self):
func = getattr(self.cb, name)
- func(r.r3, r.r7, r.r12)
+ func(r.r3.value, r.r7.value, r.r12.value)
self.assert_equal('%s r3, [r7, r12]' % name[:name.index('_')])
return f
@@ -181,18 +181,18 @@
if name[-2:] == 'ri':
def f(self):
func = getattr(self.cb, name)
- func(r.r3, r.r7, 12)
+ func(r.r3.value, r.r7.value, 12)
self.assert_equal('%s r3, r7, #12' % name[:name.index('_')])
elif table['base'] and table['result']:
def f(self):
func = getattr(self.cb, name)
- func(r.r3, r.r7, r.r12)
+ func(r.r3.value, r.r7.value, r.r12.value)
self.assert_equal('%s r3, r7, r12' % name[:name.index('_')])
else:
def f(self):
func = getattr(self.cb, name)
- func(r.r3, r.r7)
+ func(r.r3.value, r.r7.value)
self.assert_equal('%s r3, r7' % name[:name.index('_')])
return f
From arigo at codespeak.net Tue Oct 26 11:38:21 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 11:38:21 +0200 (CEST)
Subject: [pypy-svn] r78285 - pypy/branch/set-object-cleanup
Message-ID: <20101026093821.C9E4A282C18@codespeak.net>
Author: arigo
Date: Tue Oct 26 11:38:20 2010
New Revision: 78285
Added:
pypy/branch/set-object-cleanup/
- copied from r78284, pypy/trunk/
Log:
A branch in which to clean up a bit setobject.py.
From arigo at codespeak.net Tue Oct 26 11:39:01 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 11:39:01 +0200 (CEST)
Subject: [pypy-svn] r78286 - in pypy/trunk/pypy/tool/release: . test
Message-ID: <20101026093901.CFD1E282C18@codespeak.net>
Author: arigo
Date: Tue Oct 26 11:39:00 2010
New Revision: 78286
Modified:
pypy/trunk/pypy/tool/release/ (props changed)
pypy/trunk/pypy/tool/release/__init__.py (props changed)
pypy/trunk/pypy/tool/release/force-builds.py (props changed)
pypy/trunk/pypy/tool/release/make_release.py (props changed)
pypy/trunk/pypy/tool/release/test/ (props changed)
pypy/trunk/pypy/tool/release/test/__init__.py (props changed)
pypy/trunk/pypy/tool/release/test/test_make_release.py (props changed)
Log:
fixeol
From fijal at codespeak.net Tue Oct 26 13:21:05 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Tue, 26 Oct 2010 13:21:05 +0200 (CEST)
Subject: [pypy-svn] r78287 - pypy/trunk/pypy/config
Message-ID: <20101026112105.48208282C18@codespeak.net>
Author: fijal
Date: Tue Oct 26 13:21:03 2010
New Revision: 78287
Modified:
pypy/trunk/pypy/config/pypyoption.py
Log:
_ffi module also depends on libffi available on the system
Modified: pypy/trunk/pypy/config/pypyoption.py
==============================================================================
--- pypy/trunk/pypy/config/pypyoption.py (original)
+++ pypy/trunk/pypy/config/pypyoption.py Tue Oct 26 13:21:03 2010
@@ -76,6 +76,7 @@
# no _rawffi if importing pypy.rlib.clibffi raises ImportError
# or CompilationError
"_rawffi" : ["pypy.rlib.clibffi"],
+ "_ffi" : ["pypy.rlib.clibffi"],
"zlib" : ["pypy.rlib.rzlib"],
"bz2" : ["pypy.module.bz2.interp_bz2"],
From arigo at codespeak.net Tue Oct 26 13:41:10 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 13:41:10 +0200 (CEST)
Subject: [pypy-svn] r78288 - in
pypy/branch/set-object-cleanup/pypy/objspace/std: . test
Message-ID: <20101026114110.AAEC9282C18@codespeak.net>
Author: arigo
Date: Tue Oct 26 13:41:08 2010
New Revision: 78288
Modified:
pypy/branch/set-object-cleanup/pypy/objspace/std/frozensettype.py
pypy/branch/set-object-cleanup/pypy/objspace/std/setobject.py
pypy/branch/set-object-cleanup/pypy/objspace/std/settype.py
pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_setobject.py
Log:
Rewrite a bit sets. Tests pass, whatever that means.
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/frozensettype.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/frozensettype.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/frozensettype.py Tue Oct 26 13:41:08 2010
@@ -39,13 +39,13 @@
def descr__frozenset__new__(space, w_frozensettype,
w_iterable=gateway.NoneNotWrapped):
from pypy.objspace.std.setobject import W_FrozensetObject
- from pypy.objspace.std.setobject import _is_frozenset_exact
+ from pypy.objspace.std.setobject import make_setdata_from_w_iterable
if (space.is_w(w_frozensettype, space.w_frozenset) and
- _is_frozenset_exact(w_iterable)):
+ w_iterable is not None and type(w_iterable) is W_FrozensetObject):
return w_iterable
w_obj = space.allocate_instance(W_FrozensetObject, w_frozensettype)
- W_FrozensetObject.__init__(w_obj, space, None)
-
+ data = make_setdata_from_w_iterable(space, w_iterable)
+ W_FrozensetObject.__init__(w_obj, space, data)
return w_obj
frozenset_typedef = StdTypeDef("frozenset",
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/setobject.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/setobject.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/setobject.py Tue Oct 26 13:41:08 2010
@@ -21,11 +21,10 @@
return False
- def __init__(w_self, space, setdata=None):
- if setdata is None:
- w_self.setdata = r_dict(space.eq_w, space.hash_w)
- else:
- w_self.setdata = setdata.copy()
+ def __init__(w_self, space, setdata):
+ """Initialize the set by taking ownership of 'setdata'."""
+ assert setdata is not None
+ w_self.setdata = setdata
def __repr__(w_self):
"""representation for debugging purposes"""
@@ -33,6 +32,7 @@
return "<%s(%s)>" % (w_self.__class__.__name__, ', '.join(reprlist))
def _newobj(w_self, space, rdict_w=None):
+ """Make a new set or frozenset by taking ownership of 'rdict_w'."""
#return space.call(space.type(w_self),W_SetIterObject(rdict_w))
objtype = type(w_self)
if objtype is W_SetObject:
@@ -55,10 +55,7 @@
class W_FrozensetObject(W_BaseSetObject):
from pypy.objspace.std.frozensettype import frozenset_typedef as typedef
-
- def __init__(w_self, space, setdata):
- W_BaseSetObject.__init__(w_self, space, setdata)
- w_self.hash = -1
+ hash = 0
registerimplementation(W_BaseSetObject)
registerimplementation(W_SetObject)
@@ -109,8 +106,14 @@
# some helper functions
+def newset(space):
+ return r_dict(space.eq_w, space.hash_w)
+
def make_setdata_from_w_iterable(space, w_iterable=None):
- data = r_dict(space.eq_w, space.hash_w)
+ """Return a new r_dict with the content of w_iterable."""
+ if isinstance(w_iterable, W_BaseSetObject):
+ return w_iterable.setdata.copy()
+ data = newset(space)
if w_iterable is not None:
for w_item in space.listview(w_iterable):
data[w_item] = None
@@ -119,7 +122,7 @@
def _initialize_set(space, w_obj, w_iterable=None):
w_obj.setdata.clear()
if w_iterable is not None:
- w_obj.setdata.update(make_setdata_from_w_iterable(space, w_iterable))
+ w_obj.setdata = make_setdata_from_w_iterable(space, w_iterable)
def _convert_set_to_frozenset(space, w_obj):
if space.is_true(space.isinstance(w_obj, space.w_set)):
@@ -130,12 +133,6 @@
# helper functions for set operation on dicts
-def _is_frozenset_exact(w_obj):
- if (w_obj is not None) and (type(w_obj) is W_FrozensetObject):
- return True
- else:
- return False
-
def _is_eq(ld, rd):
if len(ld) != len(rd):
return False
@@ -144,66 +141,41 @@
return False
return True
-def _union_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
- else:
- ld = ldict.copy()
- ld.update(rdict)
- return ld, rdict
-
-def _difference_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
- else:
- ld = ldict.copy()
- del_list_w = []
+def _difference_dict(space, ld, rd):
+ result = newset(space)
for w_key in ld:
- if w_key in rdict:
- del_list_w.append(w_key)
- for w_key in del_list_w:
- del ld[w_key]
-
- return ld, rdict
-
-def _intersection_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
- else:
- ld = ldict.copy()
- del_list_w = []
- for w_key in ld:
- if w_key not in rdict:
- del_list_w.append(w_key)
-
- for w_key in del_list_w:
- del ld[w_key]
-
- return ld, rdict
-
+ if w_key not in rd:
+ result[w_key] = None
+ return result
-def _symmetric_difference_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
+def _difference_dict_update(space, ld, rd):
+ if ld is rd:
+ ld.clear() # for the case 'a.difference_update(a)'
else:
- ld = ldict.copy()
- del_list_w = []
- add_list_w = []
+ for w_key in rd:
+ try:
+ del ld[w_key]
+ except KeyError:
+ pass
+
+def _intersection_dict(space, ld, rd):
+ result = newset(space)
+ if len(ld) > len(rd):
+ ld, rd = rd, ld # loop over the smaller dict
for w_key in ld:
- if w_key in rdict:
- del_list_w.append(w_key)
+ if w_key in rd:
+ result[w_key] = None
+ return result
- for w_key in rdict:
+def _symmetric_difference_dict(space, ld, rd):
+ result = newset(space)
+ for w_key in ld:
+ if w_key not in rd:
+ result[w_key] = None
+ for w_key in rd:
if w_key not in ld:
- add_list_w.append(w_key)
-
- for w_key in del_list_w:
- del ld[w_key]
-
- for w_key in add_list_w:
- ld[w_key] = None
-
- return ld, rdict
+ result[w_key] = None
+ return result
def _issubset_dict(ldict, rdict):
if len(ldict) > len(rdict):
@@ -220,14 +192,13 @@
def set_update__Set_BaseSet(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _union_dict(ld, rd, True)
- return space.w_None
+ ld.update(rd)
def set_update__Set_ANY(space, w_left, w_other):
"""Update a set with the union of itself and another."""
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _union_dict(ld, rd, True)
- return space.w_None
+ ld = w_left.setdata
+ for w_item in space.listview(w_other):
+ ld[w_item] = None
def inplace_or__Set_Set(space, w_left, w_other):
set_update__Set_BaseSet(space, w_left, w_other)
@@ -241,25 +212,23 @@
This has no effect if the element is already present.
"""
w_left.setdata[w_other] = None
- return space.w_None
def set_copy__Set(space, w_set):
- return w_set._newobj(space,w_set.setdata)
+ return w_set._newobj(space, w_set.setdata.copy())
def frozenset_copy__Frozenset(space, w_left):
- if _is_frozenset_exact(w_left):
+ if type(w_left) is W_FrozensetObject:
return w_left
else:
- return set_copy__Set(space,w_left)
+ return set_copy__Set(space, w_left)
def set_clear__Set(space, w_left):
w_left.setdata.clear()
- return space.w_None
def set_difference__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _difference_dict(ld, rd, False)
+ new_ld = _difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
set_difference__Set_Frozenset = set_difference__Set_Set
@@ -272,7 +241,7 @@
def set_difference__Set_ANY(space, w_left, w_other):
ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _difference_dict(ld, rd, False)
+ new_ld = _difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
frozenset_difference__Frozenset_ANY = set_difference__Set_ANY
@@ -281,15 +250,17 @@
def set_difference_update__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _difference_dict(ld, rd, True)
- return space.w_None
+ _difference_dict_update(space, ld, rd)
set_difference_update__Set_Frozenset = set_difference_update__Set_Set
def set_difference_update__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _difference_dict(ld, rd, True)
- return space.w_None
+ ld = w_left.setdata
+ for w_key in space.listview(w_other):
+ try:
+ del ld[w_key]
+ except KeyError:
+ pass
def inplace_sub__Set_Set(space, w_left, w_other):
set_difference_update__Set_Set(space, w_left, w_other)
@@ -467,7 +438,7 @@
def hash__Frozenset(space, w_set):
multi = r_uint(1822399083) + r_uint(1822399083) + 1
- if w_set.hash != -1:
+ if w_set.hash != 0:
return space.wrap(w_set.hash)
hash = 1927868237
hash *= (len(w_set.setdata) + 1)
@@ -476,7 +447,7 @@
value = ((h ^ (h << 16) ^ 89869747) * multi)
hash = intmask(hash ^ value)
hash = hash * 69069 + 907133923
- if hash == -1:
+ if hash == 0:
hash = 590923713
hash = intmask(hash)
w_set.hash = hash
@@ -484,29 +455,31 @@
return space.wrap(hash)
def set_pop__Set(space, w_left):
- if len(w_left.setdata) == 0:
+ for w_key in w_left.setdata:
+ break
+ else:
raise OperationError(space.w_KeyError,
space.wrap('pop from an empty set'))
- w_keys = w_left.setdata.keys()
- w_value = w_keys[0]
- del w_left.setdata[w_value]
-
- return w_value
+ del w_left.setdata[w_key]
+ return w_key
def set_intersection__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _intersection_dict(ld, rd, False)
- return w_left._newobj(space,new_ld)
+ new_ld = _intersection_dict(space, ld, rd)
+ return w_left._newobj(space, new_ld)
set_intersection__Set_Frozenset = set_intersection__Set_Set
set_intersection__Frozenset_Frozenset = set_intersection__Set_Set
set_intersection__Frozenset_Set = set_intersection__Set_Set
def set_intersection__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _intersection_dict(ld, rd, False)
- return w_left._newobj(space,new_ld)
+ result = newset(space)
+ ld = w_left.setdata
+ for w_key in space.listview(w_other):
+ if w_key in ld:
+ result[w_key] = None
+ return w_left._newobj(space, result)
frozenset_intersection__Frozenset_ANY = set_intersection__Set_ANY
@@ -518,15 +491,18 @@
def set_intersection_update__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _intersection_dict(ld, rd, True)
- return space.w_None
+ new_ld = _intersection_dict(space, ld, rd)
+ w_left.setdata = new_ld
set_intersection_update__Set_Frozenset = set_intersection_update__Set_Set
def set_intersection_update__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _intersection_dict(ld, rd, True)
- return space.w_None
+ result = newset(space)
+ ld = w_left.setdata
+ for w_key in space.listview(w_other):
+ if w_key in ld:
+ result[w_key] = None
+ w_left.setdata = result
def inplace_and__Set_Set(space, w_left, w_other):
set_intersection_update__Set_Set(space, w_left, w_other)
@@ -537,7 +513,7 @@
def set_symmetric_difference__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _symmetric_difference_dict(ld, rd, False)
+ new_ld = _symmetric_difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
set_symmetric_difference__Set_Frozenset = set_symmetric_difference__Set_Set
@@ -553,7 +529,7 @@
def set_symmetric_difference__Set_ANY(space, w_left, w_other):
ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _symmetric_difference_dict(ld, rd, False)
+ new_ld = _symmetric_difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
frozenset_symmetric_difference__Frozenset_ANY = \
@@ -562,16 +538,16 @@
def set_symmetric_difference_update__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _symmetric_difference_dict(ld, rd, True)
- return space.w_None
+ new_ld = _symmetric_difference_dict(space, ld, rd)
+ w_left.setdata = new_ld
set_symmetric_difference_update__Set_Frozenset = \
set_symmetric_difference_update__Set_Set
def set_symmetric_difference_update__Set_ANY(space, w_left, w_other):
ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _symmetric_difference_dict(ld, rd, True)
- return space.w_None
+ new_ld = _symmetric_difference_dict(space, ld, rd)
+ w_left.setdata = new_ld
def inplace_xor__Set_Set(space, w_left, w_other):
set_symmetric_difference_update__Set_Set(space, w_left, w_other)
@@ -582,8 +558,9 @@
def set_union__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _union_dict(ld, rd, False)
- return w_left._newobj(space, new_ld)
+ result = ld.copy()
+ result.update(rd)
+ return w_left._newobj(space, result)
set_union__Set_Frozenset = set_union__Set_Set
set_union__Frozenset_Set = set_union__Set_Set
@@ -595,9 +572,11 @@
def set_union__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _union_dict(ld, rd, False)
- return w_left._newobj(space, new_ld)
+ ld = w_left.setdata()
+ result = ld.copy()
+ for w_key in space.viewlist(w_other):
+ result[w_key] = None
+ return w_left._newobj(space, result)
frozenset_union__Frozenset_ANY = set_union__Set_ANY
@@ -629,15 +608,6 @@
init_defaults)
_initialize_set(space, w_set, w_iterable)
-def init__Frozenset(space, w_set, __args__):
- w_iterable, = __args__.parse_obj(
- None, 'set',
- init_signature,
- init_defaults)
- if w_set.hash == -1:
- _initialize_set(space, w_set, w_iterable)
- hash__Frozenset(space, w_set)
-
app = gateway.applevel("""
def setrepr(currently_in_repr, s):
'The app-level part of repr().'
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/settype.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/settype.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/settype.py Tue Oct 26 13:41:08 2010
@@ -66,9 +66,9 @@
register_all(vars(), globals())
def descr__new__(space, w_settype, __args__):
- from pypy.objspace.std.setobject import W_SetObject
+ from pypy.objspace.std.setobject import W_SetObject, newset
w_obj = space.allocate_instance(W_SetObject, w_settype)
- W_SetObject.__init__(w_obj, space, None)
+ W_SetObject.__init__(w_obj, space, newset(space))
return w_obj
set_typedef = StdTypeDef("set",
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_setobject.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_setobject.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_setobject.py Tue Oct 26 13:41:08 2010
@@ -10,6 +10,7 @@
import py.test
from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject
from pypy.objspace.std.setobject import _initialize_set
+from pypy.objspace.std.setobject import newset, make_setdata_from_w_iterable
from pypy.objspace.std.setobject import set_intersection__Set_Set
from pypy.objspace.std.setobject import set_intersection__Set_ANY
from pypy.objspace.std.setobject import eq__Set_Set
@@ -28,12 +29,12 @@
self.false = self.space.w_False
def test_and(self):
- s = W_SetObject(self.space, None)
+ s = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, s, self.word)
- t0 = W_SetObject(self.space, None)
+ t0 = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, t0, self.otherword)
- t1 = W_FrozensetObject(self.space, None)
- _initialize_set(self.space, t1, self.otherword)
+ t1 = W_FrozensetObject(self.space,
+ make_setdata_from_w_iterable(self.space, self.otherword))
r0 = set_intersection__Set_Set(self.space, s, t0)
r1 = set_intersection__Set_Set(self.space, s, t1)
assert eq__Set_Set(self.space, r0, r1) == self.true
@@ -41,9 +42,9 @@
assert eq__Set_Set(self.space, r0, sr) == self.true
def test_compare(self):
- s = W_SetObject(self.space, None)
+ s = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, s, self.word)
- t = W_SetObject(self.space, None)
+ t = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, t, self.word)
assert self.space.eq_w(s,t)
u = self.space.wrap(set('simsalabim'))
From arigo at codespeak.net Tue Oct 26 13:45:02 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 13:45:02 +0200 (CEST)
Subject: [pypy-svn] r78289 - in
pypy/branch/set-object-cleanup/pypy/objspace/std: . test
Message-ID: <20101026114502.8C17B282C18@codespeak.net>
Author: arigo
Date: Tue Oct 26 13:45:01 2010
New Revision: 78289
Modified:
pypy/branch/set-object-cleanup/pypy/objspace/std/setobject.py
pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_setobject.py
Log:
Test and fixes.
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/setobject.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/setobject.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/setobject.py Tue Oct 26 13:45:01 2010
@@ -572,9 +572,9 @@
def set_union__Set_ANY(space, w_left, w_other):
- ld = w_left.setdata()
+ ld = w_left.setdata
result = ld.copy()
- for w_key in space.viewlist(w_other):
+ for w_key in space.listview(w_other):
result[w_key] = None
return w_left._newobj(space, result)
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_setobject.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_setobject.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_setobject.py Tue Oct 26 13:45:01 2010
@@ -57,6 +57,11 @@
b = a | set('abc')
assert type(b) is subset
+ def test_union(self):
+ a = set([4, 5])
+ b = a.union([5, 7])
+ assert sorted(b) == [4, 5, 7]
+
def test_compare(self):
raises(TypeError, cmp, set('abc'), set('abd'))
assert set('abc') != 'abc'
From arigo at codespeak.net Tue Oct 26 13:58:09 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 13:58:09 +0200 (CEST)
Subject: [pypy-svn] r78290 - pypy/branch/set-object-cleanup/pypy/interpreter
Message-ID: <20101026115809.A136A282C18@codespeak.net>
Author: arigo
Date: Tue Oct 26 13:58:08 2010
New Revision: 78290
Modified:
pypy/branch/set-object-cleanup/pypy/interpreter/baseobjspace.py
Log:
Fix the docstring.
Modified: pypy/branch/set-object-cleanup/pypy/interpreter/baseobjspace.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/interpreter/baseobjspace.py (original)
+++ pypy/branch/set-object-cleanup/pypy/interpreter/baseobjspace.py Tue Oct 26 13:58:08 2010
@@ -739,7 +739,7 @@
def unpackiterable(self, w_iterable, expected_length=-1):
"""Unpack an iterable object into a real (interpreter-level) list.
- Raise a real (subclass of) ValueError if the length is wrong."""
+ Raise an OperationError(w_ValueError) if the length is wrong."""
w_iterator = self.iter(w_iterable)
items = []
while True:
From arigo at codespeak.net Tue Oct 26 14:12:15 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 14:12:15 +0200 (CEST)
Subject: [pypy-svn] r78291 - pypy/branch/set-object-cleanup/pypy/objspace/std
Message-ID: <20101026121215.207885080C@codespeak.net>
Author: arigo
Date: Tue Oct 26 14:12:13 2010
New Revision: 78291
Modified:
pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py
Log:
A faster implementation of dict.fromkeys().
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py Tue Oct 26 14:12:13 2010
@@ -141,18 +141,17 @@
@gateway.unwrap_spec(ObjSpace, W_Root, W_Root, W_Root)
def descr_fromkeys(space, w_type, w_keys, w_fill=None):
+ from pypy.objspace.std.dictmultiobject import W_DictMultiObject
if w_fill is None:
w_fill = space.w_None
- w_dict = space.call_function(w_type)
- w_iter = space.iter(w_keys)
- while True:
- try:
- w_key = space.next(w_iter)
- except OperationError, e:
- if not e.match(space, space.w_StopIteration):
- raise
- break
- space.setitem(w_dict, w_key, w_fill)
+ if w_type is space.w_dict:
+ w_dict = W_DictMultiObject.allocate_and_init_instance(space, w_type)
+ for w_key in space.listview(w_keys):
+ w_dict.setitem(w_key, w_fill)
+ else:
+ w_dict = space.call_function(w_type)
+ for w_key in space.listview(w_keys):
+ space.setitem(w_dict, w_key, w_fill)
return w_dict
From arigo at codespeak.net Tue Oct 26 14:15:50 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 14:15:50 +0200 (CEST)
Subject: [pypy-svn] r78292 - pypy/branch/set-object-cleanup/pypy/objspace/std
Message-ID: <20101026121550.09FC8282C18@codespeak.net>
Author: arigo
Date: Tue Oct 26 14:15:49 2010
New Revision: 78292
Modified:
pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py
Log:
Kill code.
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py Tue Oct 26 14:15:49 2010
@@ -90,51 +90,17 @@
dict.__delitem__(d, k)
return k, v
- def get(d, k, v=None):
- if k in d:
- return dict.__getitem__(d, k)
- else:
- return v
-
def setdefault(d, k, v=None):
if k in d:
return dict.__getitem__(d, k)
else:
dict.__setitem__(d, k, v)
return v
-
- def pop(d, k, defaults): # XXX defaults is actually *defaults
- if len(defaults) > 1:
- raise TypeError, "pop expected at most 2 arguments, got %d" % (
- 1 + len(defaults))
- try:
- v = dict.__getitem__(d, k)
- dict.__delitem__(d, k)
- except KeyError, e:
- if defaults:
- return defaults[0]
- else:
- raise e
- return v
-
- def iteritems(d):
- return iter(dict.items(d))
-
- def iterkeys(d):
- return iter(dict.keys(d))
-
- def itervalues(d):
- return iter(dict.values(d))
''', filename=__file__)
dict_update__ANY = app.interphook("update")
dict_popitem__ANY = app.interphook("popitem")
-dict_get__ANY_ANY_ANY = app.interphook("get")
dict_setdefault__ANY_ANY_ANY = app.interphook("setdefault")
-dict_pop__ANY_ANY = app.interphook("pop")
-dict_iteritems__ANY = app.interphook("iteritems")
-dict_iterkeys__ANY = app.interphook("iterkeys")
-dict_itervalues__ANY = app.interphook("itervalues")
update1 = app.interphook("update1")
register_all(vars(), globals())
From arigo at codespeak.net Tue Oct 26 14:27:58 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 14:27:58 +0200 (CEST)
Subject: [pypy-svn] r78293 - pypy/branch/set-object-cleanup/pypy/objspace/std
Message-ID: <20101026122758.E0599282C18@codespeak.net>
Author: arigo
Date: Tue Oct 26 14:27:57 2010
New Revision: 78293
Modified:
pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py
pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py
Log:
Kill dict.update() at app-level, and implement it as just
a call to dict.__init__().
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py Tue Oct 26 14:27:57 2010
@@ -645,7 +645,7 @@
w_src, w_kwds = __args__.parse_obj(
None, 'dict',
init_signature, # signature
- init_defaults) # default argument
+ init_defaults) # default argument
if w_src is None:
pass
elif space.findattr(w_src, space.wrap("keys")) is None:
@@ -665,6 +665,9 @@
from pypy.objspace.std.dicttype import update1
update1(space, w_dict, w_kwds)
+def dict_update__DictMulti(space, w_dict, __args__):
+ init__DictMulti(space, w_dict, __args__)
+
def getitem__DictMulti_ANY(space, w_dict, w_key):
w_value = w_dict.getitem(w_key)
if w_value is not None:
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py Tue Oct 26 14:27:57 2010
@@ -72,15 +72,6 @@
for k,v in o:
dict.__setitem__(d, k, v)
- def update(d, *args, **kwargs):
- len_args = len(args)
- if len_args == 1:
- update1(d, args[0])
- elif len_args > 1:
- raise TypeError("update takes at most 1 (non-keyword) argument")
- if kwargs:
- update1(d, kwargs)
-
def popitem(d):
for k in dict.iterkeys(d):
break
@@ -98,7 +89,6 @@
return v
''', filename=__file__)
-dict_update__ANY = app.interphook("update")
dict_popitem__ANY = app.interphook("popitem")
dict_setdefault__ANY_ANY_ANY = app.interphook("setdefault")
update1 = app.interphook("update1")
From arigo at codespeak.net Tue Oct 26 14:35:06 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 14:35:06 +0200 (CEST)
Subject: [pypy-svn] r78294 - pypy/branch/set-object-cleanup/pypy/objspace/std
Message-ID: <20101026123506.66045282C18@codespeak.net>
Author: arigo
Date: Tue Oct 26 14:35:04 2010
New Revision: 78294
Modified:
pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py
Log:
Fix the name that appears in error messages.
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py Tue Oct 26 14:35:04 2010
@@ -641,9 +641,9 @@
init_signature = Signature(['seq_or_map'], None, 'kwargs')
init_defaults = [None]
-def init__DictMulti(space, w_dict, __args__):
+def init_or_update(space, w_dict, __args__, funcname):
w_src, w_kwds = __args__.parse_obj(
- None, 'dict',
+ None, funcname,
init_signature, # signature
init_defaults) # default argument
if w_src is None:
@@ -665,8 +665,11 @@
from pypy.objspace.std.dicttype import update1
update1(space, w_dict, w_kwds)
+def init__DictMulti(space, w_dict, __args__):
+ init_or_update(space, w_dict, __args__, 'dict')
+
def dict_update__DictMulti(space, w_dict, __args__):
- init__DictMulti(space, w_dict, __args__)
+ init_or_update(space, w_dict, __args__, 'dict.update')
def getitem__DictMulti_ANY(space, w_dict, w_key):
w_value = w_dict.getitem(w_key)
From arigo at codespeak.net Tue Oct 26 14:52:14 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 14:52:14 +0200 (CEST)
Subject: [pypy-svn] r78295 - in
pypy/branch/set-object-cleanup/pypy/objspace/std: . test
Message-ID: <20101026125214.041475080C@codespeak.net>
Author: arigo
Date: Tue Oct 26 14:52:13 2010
New Revision: 78295
Modified:
pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py
pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py
pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_dictmultiobject.py
Log:
Move update1() to interp-level.
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py Tue Oct 26 14:52:13 2010
@@ -641,28 +641,42 @@
init_signature = Signature(['seq_or_map'], None, 'kwargs')
init_defaults = [None]
+def update1(space, w_dict, w_data):
+ if space.findattr(w_data, space.wrap("keys")) is None:
+ # no 'keys' method, so we assume it is a sequence of pairs
+ for w_pair in space.listview(w_data):
+ pair = space.fixedview(w_pair)
+ if len(pair) != 2:
+ raise OperationError(space.w_ValueError,
+ space.wrap("sequence of pairs expected"))
+ w_key, w_value = pair
+ w_dict.setitem(w_key, w_value)
+ else:
+ if isinstance(w_data, W_DictMultiObject): # optimization case only
+ update1_dict_dict(space, w_dict, w_data)
+ else:
+ # general case -- "for k in o.keys(): dict.__setitem__(d, k, o[k])"
+ w_keys = space.call_method(w_data, "keys")
+ for w_key in space.listview(w_keys):
+ w_value = space.getitem(w_data, w_key)
+ w_dict.setitem(w_key, w_value)
+
+def update1_dict_dict(space, w_dict, w_data):
+ iterator = w_data.iter()
+ while 1:
+ w_key, w_value = iterator.next()
+ if w_key is None:
+ break
+ w_dict.setitem(w_key, w_value)
+
def init_or_update(space, w_dict, __args__, funcname):
w_src, w_kwds = __args__.parse_obj(
None, funcname,
init_signature, # signature
init_defaults) # default argument
- if w_src is None:
- pass
- elif space.findattr(w_src, space.wrap("keys")) is None:
- list_of_w_pairs = space.listview(w_src)
- for w_pair in list_of_w_pairs:
- pair = space.fixedview(w_pair)
- if len(pair)!=2:
- raise OperationError(space.w_ValueError,
- space.wrap("dict() takes a sequence of pairs"))
- w_k, w_v = pair
- w_dict.setitem(w_k, w_v)
- else:
- if space.is_true(w_src):
- from pypy.objspace.std.dicttype import update1
- update1(space, w_dict, w_src)
+ if w_src is not None:
+ update1(space, w_dict, w_src)
if space.is_true(w_kwds):
- from pypy.objspace.std.dicttype import update1
update1(space, w_dict, w_kwds)
def init__DictMulti(space, w_dict, __args__):
@@ -764,9 +778,8 @@
return w_res
def dict_copy__DictMulti(space, w_self):
- from pypy.objspace.std.dicttype import update1
w_new = W_DictMultiObject.allocate_and_init_instance(space)
- update1(space, w_new, w_self)
+ update1_dict_dict(space, w_new, w_self)
return w_new
def dict_items__DictMulti(space, w_self):
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py Tue Oct 26 14:52:13 2010
@@ -64,14 +64,6 @@
# CPython does not call it when doing builtin operations. The
# same for other operations.
- def update1(d, o):
- if hasattr(o, 'keys'):
- for k in o.keys():
- dict.__setitem__(d, k, o[k])
- else:
- for k,v in o:
- dict.__setitem__(d, k, v)
-
def popitem(d):
for k in dict.iterkeys(d):
break
@@ -91,7 +83,6 @@
dict_popitem__ANY = app.interphook("popitem")
dict_setdefault__ANY_ANY_ANY = app.interphook("setdefault")
-update1 = app.interphook("update1")
register_all(vars(), globals())
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_dictmultiobject.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_dictmultiobject.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_dictmultiobject.py Tue Oct 26 14:52:13 2010
@@ -446,7 +446,9 @@
d1 = {}
d2 = D(a='foo')
d1.update(d2)
- assert d1['a'] == 42 # fails on CPython, d1['a'] == 'foo'
+ assert d1['a'] == 'foo'
+ # a bit of an obscure case: now (from r78295) we get the same result
+ # as CPython does
def test_index_keyerror_unpacking(self):
d = {}
From arigo at codespeak.net Tue Oct 26 14:54:24 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 14:54:24 +0200 (CEST)
Subject: [pypy-svn] r78296 - pypy/branch/set-object-cleanup/pypy/objspace/std
Message-ID: <20101026125424.5DF74282C1A@codespeak.net>
Author: arigo
Date: Tue Oct 26 14:54:22 2010
New Revision: 78296
Modified:
pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py
pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py
Log:
Move dict.setdefault() to interp-level. For now it may still require two lookups
instead of one.
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py Tue Oct 26 14:54:22 2010
@@ -810,6 +810,15 @@
else:
return w_default
+def dict_setdefault__DictMulti_ANY_ANY(space, w_dict, w_key, w_default):
+ # XXX should be more efficient, with only one dict lookup
+ w_value = w_dict.getitem(w_key)
+ if w_value is not None:
+ return w_value
+ else:
+ w_dict.setitem(w_key, w_default)
+ return w_default
+
def dict_pop__DictMulti_ANY(space, w_dict, w_key, w_defaults):
defaults = space.listview(w_defaults)
len_defaults = len(defaults)
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py Tue Oct 26 14:54:22 2010
@@ -72,17 +72,9 @@
v = dict.__getitem__(d, k)
dict.__delitem__(d, k)
return k, v
-
- def setdefault(d, k, v=None):
- if k in d:
- return dict.__getitem__(d, k)
- else:
- dict.__setitem__(d, k, v)
- return v
''', filename=__file__)
dict_popitem__ANY = app.interphook("popitem")
-dict_setdefault__ANY_ANY_ANY = app.interphook("setdefault")
register_all(vars(), globals())
From arigo at codespeak.net Tue Oct 26 14:57:44 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 14:57:44 +0200 (CEST)
Subject: [pypy-svn] r78297 - pypy/branch/set-object-cleanup/pypy/objspace/std
Message-ID: <20101026125744.5543E282C1A@codespeak.net>
Author: arigo
Date: Tue Oct 26 14:57:42 2010
New Revision: 78297
Modified:
pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py
pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py
Log:
Interp-level implementation of dict.popitem().
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py Tue Oct 26 14:57:42 2010
@@ -836,6 +836,16 @@
w_dict.delitem(w_key)
return w_item
+def dict_popitem__DictMulti(space, w_dict):
+ # XXX should somehow use the same trick as CPython: saving the index
+ # of the last popped item in the hash table, so that the next call to
+ # popitem() can be more efficient, instead of always starting from the
+ # beginning of the hash table.
+ iterator = w_dict.iter()
+ w_key, w_value = iterator.next()
+ w_dict.delitem(w_key)
+ return space.newtuple([w_key, w_value])
+
app = gateway.applevel('''
def dictrepr(currently_in_repr, d):
# Now we only handle one implementation of dicts, this one.
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/dicttype.py Tue Oct 26 14:57:42 2010
@@ -53,29 +53,6 @@
# This can return when multimethods have been fixed
#dict_str = StdObjSpace.str
-# default application-level implementations for some operations
-# most of these (notably not popitem and update*) are overwritten
-# in dictmultiobject
-# gateway is imported in the stdtypedef module
-app = gateway.applevel('''
-
- # in the following functions we use dict.__setitem__ instead of
- # d[k]=... because when a subclass of dict override __setitem__,
- # CPython does not call it when doing builtin operations. The
- # same for other operations.
-
- def popitem(d):
- for k in dict.iterkeys(d):
- break
- else:
- raise KeyError("popitem(): dictionary is empty")
- v = dict.__getitem__(d, k)
- dict.__delitem__(d, k)
- return k, v
-''', filename=__file__)
-
-dict_popitem__ANY = app.interphook("popitem")
-
register_all(vars(), globals())
@gateway.unwrap_spec(ObjSpace, W_Root, W_Root, W_Root)
From arigo at codespeak.net Tue Oct 26 15:30:43 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 15:30:43 +0200 (CEST)
Subject: [pypy-svn] r78298 - in
pypy/branch/set-object-cleanup/pypy/objspace/std: . test
Message-ID: <20101026133043.D28065080E@codespeak.net>
Author: arigo
Date: Tue Oct 26 15:30:42 2010
New Revision: 78298
Modified:
pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py
pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_dictmultiobject.py
Log:
Test and fix.
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/dictmultiobject.py Tue Oct 26 15:30:42 2010
@@ -843,6 +843,9 @@
# beginning of the hash table.
iterator = w_dict.iter()
w_key, w_value = iterator.next()
+ if w_key is None:
+ raise OperationError(space.w_KeyError,
+ space.wrap("popitem(): dictionary is empty"))
w_dict.delitem(w_key)
return space.newtuple([w_key, w_value])
Modified: pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_dictmultiobject.py
==============================================================================
--- pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_dictmultiobject.py (original)
+++ pypy/branch/set-object-cleanup/pypy/objspace/std/test/test_dictmultiobject.py Tue Oct 26 15:30:42 2010
@@ -238,6 +238,7 @@
it1 = d.popitem()
assert len(d) == 0
assert (it!=it1) and (it1==(1,2) or it1==(3,4))
+ raises(KeyError, d.popitem)
def test_setdefault(self):
d = {1:2, 3:4}
From david at codespeak.net Tue Oct 26 16:35:11 2010
From: david at codespeak.net (david at codespeak.net)
Date: Tue, 26 Oct 2010 16:35:11 +0200 (CEST)
Subject: [pypy-svn] r78299 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101026143511.1A5A0282C18@codespeak.net>
Author: david
Date: Tue Oct 26 16:35:09 2010
New Revision: 78299
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
Log:
Add support for constant int values from args
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Tue Oct 26 16:35:09 2010
@@ -1,10 +1,11 @@
from pypy.jit.backend.arm import conditions as c
+from pypy.jit.backend.arm import locations
from pypy.jit.backend.arm import registers as r
from pypy.jit.backend.arm.arch import WORD, FUNC_ALIGN
from pypy.jit.backend.arm.codebuilder import ARMv7Builder, ARMv7InMemoryBuilder
from pypy.jit.backend.arm.regalloc import ARMRegisterManager
from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity
-from pypy.jit.metainterp.history import ConstInt, Box, BasicFailDescr
+from pypy.jit.metainterp.history import ConstInt, BoxInt, Box, BasicFailDescr
from pypy.jit.metainterp.resoperation import rop
from pypy.rlib import rgc
from pypy.rpython.annlowlevel import llhelper
@@ -46,6 +47,7 @@
Registers are saved on the stack
XXX Rest to follow"""
i = -1
+ fail_index = 0
while(True):
i += 1
r = enc[i]
@@ -53,11 +55,20 @@
continue
if r == '\xFF':
break
- reg = ord(enc[i])
- self.fail_boxes_int.setitem(i, self.decode32(stack, reg*WORD))
+ if r == '\xFD':
+ # imm value
+ value = self.decode32(enc, i+1)
+ i += 4
+ else:
+ reg = ord(enc[i])
+ value = self.decode32(stack, reg*WORD)
+
+ self.fail_boxes_int.setitem(fail_index, value)
+ fail_index += 1
+
assert enc[i] == '\xFF'
descr = self.decode32(enc, i+1)
- self.fail_boxes_count = i
+ self.fail_boxes_count = fail_index
return descr
def decode32(self, mem, index):
@@ -97,21 +108,31 @@
box = Box()
reg = regalloc.try_allocate_reg(box)
# XXX free this memory
- mem = lltype.malloc(rffi.CArray(lltype.Char), len(args)+5, flavor='raw')
- for i in range(len(args)):
+ mem = lltype.malloc(rffi.CArray(lltype.Char), (len(args)+5)*4, flavor='raw')
+ i = 0
+ j = 0
+ while(i < len(args)):
if args[i]:
- curreg = regalloc.try_allocate_reg(args[i])
- mem[i] = chr(curreg.value)
+ if not isinstance(args[i], ConstInt):
+ curreg = regalloc.try_allocate_reg(args[i])
+ mem[j] = chr(curreg.value)
+ j+=1
+ else:
+ mem[j] = '\xFD'
+ j+=1
+ self.encode32(mem, j, args[i].getint())
+ j+=4
else:
- mem[i] = '\xFE'
+ mem[j] = '\xFE'
+ j+=1
+ i+=1
- i = len(args)
- mem[i] = chr(0xFF)
+ mem[j] = chr(0xFF)
memaddr = rffi.cast(lltype.Signed, mem)
n = self.cpu.get_fail_descr_number(op.getdescr())
- self.encode32(mem, i+1, n)
+ self.encode32(mem, j+1, n)
self.mc.gen_load_int(r.lr.value, memaddr, cond=fcond)
self.mc.gen_load_int(reg.value, self.mc.baseaddr(), cond=fcond)
self.mc.MOV_rr(r.pc.value, reg.value, cond=fcond)
@@ -146,13 +167,13 @@
def assemble_loop(self, inputargs, operations, looptoken):
longevity = compute_vars_longevity(inputargs, operations)
regalloc = ARMRegisterManager(longevity, assembler=self.mc)
+ self.align()
loop_start=self.mc.curraddr()
self.gen_func_prolog()
self.gen_bootstrap_code(inputargs, regalloc, looptoken)
loop_head=self.mc.curraddr()
looptoken._arm_bootstrap_code = loop_start
looptoken._arm_loop_code = loop_head
- looptoken._temp_inputargs = inputargs#XXX remove
fcond=c.AL
for op in operations:
opnum = op.getopnum()
@@ -221,10 +242,17 @@
return c.GT
def emit_op_int_add(self, op, regalloc, fcond):
- reg = regalloc.try_allocate_reg(op.getarg(0))
+ if isinstance(op.getarg(0), BoxInt) and isinstance(op.getarg(1), BoxInt):
+ explode
res = regalloc.try_allocate_reg(op.result)
- assert isinstance(op.getarg(1), ConstInt)
- self.mc.ADD_ri(res.value, reg.value, op.getarg(1).getint())
+ if isinstance(op.getarg(1), ConstInt):
+ reg = regalloc.try_allocate_reg(op.getarg(0))
+ arg1 = op.getarg(1)
+ else:
+ reg = regalloc.try_allocate_reg(op.getarg(1))
+ arg1 = op.getarg(0)
+
+ self.mc.ADD_ri(res.value, reg.value, arg1.getint())
regalloc.possibly_free_vars_for_op(op)
return fcond
From afa at codespeak.net Tue Oct 26 16:49:50 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 26 Oct 2010 16:49:50 +0200 (CEST)
Subject: [pypy-svn] r78300 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101026144950.47A34282C18@codespeak.net>
Author: afa
Date: Tue Oct 26 16:49:43 2010
New Revision: 78300
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
Log:
Implement BufferedWriter: write(), flush().
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Tue Oct 26 16:49:43 2010
@@ -8,7 +8,7 @@
from pypy.rlib.rstring import StringBuilder
from pypy.rlib.rarithmetic import r_longlong
from pypy.module._io.interp_iobase import W_IOBase, convert_size
-from pypy.module._io.interp_io import DEFAULT_BUFFER_SIZE
+from pypy.module._io.interp_io import DEFAULT_BUFFER_SIZE, W_BlockingIOError
from pypy.module.thread.os_lock import Lock
class BlockingIOError(Exception):
@@ -17,14 +17,53 @@
class W_BufferedIOBase(W_IOBase):
def __init__(self, space):
W_IOBase.__init__(self, space)
+
self.buffer = lltype.nullptr(rffi.CCHARP.TO)
- self.pos = 0 # Current logical position in the buffer
- self.raw_pos = 0 # Position of the raw stream in the buffer.
+ self.pos = 0 # Current logical position in the buffer
+ self.raw_pos = 0 # Position of the raw stream in the buffer.
+
+ self.read_end = -1 # Just after the last buffered byte in the buffer,
+ # or -1 if the buffer isn't ready for reading
+
+ self.write_pos = 0 # Just after the last byte actually written
+ self.write_end = -1 # Just after the last byte waiting to be written,
+ # or -1 if the buffer isn't ready for writing.
+
self.lock = None
self.readable = False
self.writable = False
+ def _unsupportedoperation(self, space, message):
+ w_exc = space.getattr(space.getbuiltinmodule('_io'),
+ space.wrap('UnsupportedOperation'))
+ raise OperationError(w_exc, space.wrap(message))
+
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def read_w(self, space, w_size=None):
+ self._unsupportedoperation(space, "read")
+
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def write_w(self, space, w_size=None):
+ self._unsupportedoperation(space, "write")
+
+ def _reader_reset_buf(self):
+ self.read_end = -1
+
+ def _writer_reset_buf(self):
+ self.write_pos = 0
+ self.write_end = -1
+
+W_BufferedIOBase.typedef = TypeDef(
+ '_BufferedIOBase', W_IOBase.typedef,
+ __new__ = generic_new_descr(W_BufferedIOBase),
+ read = interp2app(W_BufferedIOBase.read_w),
+ write = interp2app(W_BufferedIOBase.write_w),
+ )
+
+class BufferedMixin:
+ _mixin_ = True
+
def _init(self, space):
if self.buffer_size <= 0:
raise OperationError(space.w_ValueError, space.wrap(
@@ -61,21 +100,12 @@
return 0
def _raw_offset(self):
- if self.raw_pos and (
+ if self.raw_pos >= 0 and (
(self.readable and self.read_end != -1) or
(self.writable and self.write_end != -1)):
return self.raw_pos - self.pos
return 0
- def _unsupportedoperation(self, space, message):
- w_exc = space.getattr(space.getbuiltinmodule('_io'),
- space.wrap('UnsupportedOperation'))
- raise OperationError(w_exc, space.wrap(message))
-
- @unwrap_spec('self', ObjSpace, W_Root)
- def read_w(self, space, w_size=None):
- self._unsupportedoperation(space, "read")
-
@unwrap_spec('self', ObjSpace, r_longlong, int)
def seek_w(self, space, pos, whence=0):
if whence not in (0, 1, 2):
@@ -100,7 +130,7 @@
# Fallback: invoke raw seek() method and clear buffer
with self.lock:
if self.writable:
- self._writer_flush_unlocked(restore_pos=False)
+ self._writer_flush_unlocked(space)
self._writer_reset_buf()
if whence == 1:
@@ -120,14 +150,80 @@
"Raw stream returned invalid position"))
return pos
-W_BufferedIOBase.typedef = TypeDef(
- '_BufferedIOBase', W_IOBase.typedef,
- __new__ = generic_new_descr(W_BufferedIOBase),
- read = interp2app(W_BufferedIOBase.read_w),
- seek = interp2app(W_BufferedIOBase.seek_w),
- )
+ def _closed(self, space):
+ return self.raw._closed(space)
+
+ @unwrap_spec('self', ObjSpace)
+ def close_w(self, space):
+ with self.lock:
+ if self._closed(space):
+ return
+ space.call_method(self, "flush")
+ with self.lock:
+ space.call_method(self.raw, "close")
+
+ @unwrap_spec('self', ObjSpace)
+ def flush_w(self, space):
+ return space.call_method(self.raw, "flush")
+
+ def _writer_flush_unlocked(self, space, restore_pos=False):
+ if self.write_end == -1 or self.write_pos == self.write_end:
+ return
+ # First, rewind
+ rewind = self._raw_offset() + (self.pos - self.write_pos)
+ if rewind != 0:
+ self._raw_seek(space, -rewind, 1)
+ self.raw_pos -= rewind
-class W_BufferedReader(W_BufferedIOBase):
+ written = 0
+ while self.write_pos < self.write_end:
+ try:
+ n = self._raw_write(space, self.write_pos, self.write_end)
+ except OperationError, e:
+ if not e.match(space, space.gettypeobject(
+ W_BlockingIOError.typedef)):
+ raise
+ self.write_pos += e.written
+ self.raw_pos = self.write_pos
+ written += e.written
+ # re-raise the error
+ e.written = written
+ raise
+ self.write_pos += n
+ self.raw_pos = self.write_pos
+ written += n
+ # Partial writes can return successfully when interrupted by a
+ # signal (see write(2)). We must run signal handlers before
+ # blocking another time, possibly indefinitely.
+ # XXX PyErr_CheckSignals()
+
+ if restore_pos:
+ forward = rewind - written
+ if forward:
+ self._raw_seek(space, forward, 1)
+ self.raw_pos += forward
+
+ self._writer_reset_buf()
+
+ def _write(self, space, data):
+ w_data = space.wrap(data)
+ w_written = space.call_method(self.raw, "write", w_data)
+ written = space.getindex_w(w_written, space.w_IOError)
+ if not 0 <= written <= len(data):
+ raise OperationError(space.w_IOError, space.wrap(
+ "raw write() returned invalid length"))
+ if self.abs_pos != -1:
+ self.abs_pos += written
+ return written
+
+ def _raw_write(self, space, start, end):
+ # XXX inefficient
+ l = []
+ for i in range(start, end):
+ l.append(self.buffer[i])
+ return self._write(space, ''.join(l))
+
+class W_BufferedReader(W_BufferedIOBase, BufferedMixin):
def __init__(self, space):
W_BufferedIOBase.__init__(self, space)
self.ok = False
@@ -145,12 +241,6 @@
self._init(space)
self._reader_reset_buf()
- def _reader_reset_buf(self):
- self.read_end = -1
-
- def _closed(self, space):
- return self.raw._closed(space)
-
@unwrap_spec('self', ObjSpace, W_Root)
def read_w(self, space, w_size=None):
self._check_closed(space, "read of closed file")
@@ -180,7 +270,7 @@
self._reader_reset_buf()
# We're going past the buffer's bounds, flush it
if self.writable:
- self._writer_flush_unlocked(restore_pos=True)
+ self._writer_flush_unlocked(space, restore_pos=True)
while True:
# Read until EOF or until read() would block
@@ -239,7 +329,7 @@
# XXX potential bug in CPython? The following is not enabled.
# We're going past the buffer's bounds, flush it
## if self.writable:
- ## self._writer_flush_unlocked(restore_pos=True)
+ ## self._writer_flush_unlocked(space, restore_pos=True)
# Read whole blocks, and don't buffer them
while remaining > 0:
@@ -302,13 +392,159 @@
__init__ = interp2app(W_BufferedReader.descr_init),
read = interp2app(W_BufferedReader.read_w),
+
+ # from the mixin class
+ seek = interp2app(W_BufferedReader.seek_w),
+ close = interp2app(W_BufferedReader.close_w),
+ flush = interp2app(W_BufferedReader.flush_w),
)
-class W_BufferedWriter(W_BufferedIOBase):
- pass
+class W_BufferedWriter(W_BufferedIOBase, BufferedMixin):
+ @unwrap_spec('self', ObjSpace, W_Root, int)
+ def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE):
+ raw = space.interp_w(W_IOBase, w_raw)
+ raw.check_writable_w(space)
+
+ self.raw = raw
+ self.buffer_size = buffer_size
+ self.writable = True
+
+ self._init(space)
+ self._writer_reset_buf()
+
+ def _adjust_position(self, new_pos):
+ self.pos = new_pos
+ if self.readable and self.read_end != -1 and self.read_end < new_pos:
+ self.read_end = self.pos
+
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def write_w(self, space, w_data):
+ self._check_closed(space, "write to closed file")
+ data = space.str_w(w_data)
+ size = len(data)
+
+ with self.lock:
+
+ if (not (self.readable and self.read_end == -1) and
+ not (self.writable and self.write_end == -1)):
+ self.pos = 0
+ self.raw_pos = 0
+ available = self.buffer_size - self.pos
+ # Fast path: the data to write can be fully buffered
+ if size <= available:
+ for i in range(size):
+ self.buffer[self.pos + i] = data[i]
+ if self.write_end == -1:
+ self.write_pos = self.pos
+ self._adjust_position(self.pos + size)
+ if self.pos > self.write_end:
+ self.write_end = self.pos
+ return space.wrap(size)
+
+ # First write the current buffer
+ try:
+ self._writer_flush_unlocked(space)
+ except OperationError, e:
+ if not e.match(space, space.gettypeobject(
+ W_BlockingIOError.typedef)):
+ raise
+ if self.readable:
+ self._reader_reset_buf()
+ # Make some place by shifting the buffer
+ for i in range(self.write_pos, self.write_end):
+ self.buffer[i - self.write_pos] = self.buffer[i]
+ self.write_end -= self.write_pos
+ self.raw_pos -= self.write_pos
+ self.pos -= self.write_pos
+ self.write_pos = 0
+ available = self.buffer_size - self.write_end
+ if size <= available:
+ # Everything can be buffered
+ for i in range(size):
+ self.buffer[self.write_end + i] = data[i]
+ self.write_end += size
+ return space.wrap(size)
+ # Buffer as much as possible
+ for i in range(available):
+ self.buffer[self.write_end + i] = data[i]
+ self.write_end += available
+ # Raise previous exception
+ e.written = available
+ raise
+
+ # Adjust the raw stream position if it is away from the logical
+ # stream position. This happens if the read buffer has been filled
+ # but not modified (and therefore _bufferedwriter_flush_unlocked()
+ # didn't rewind the raw stream by itself).
+ offset = self._raw_offset()
+ if offset:
+ self._raw_seek(space, -offset, 1)
+ self.raw_pos -= offset
+
+ # Then write buf itself. At this point the buffer has been emptied
+ remaining = size
+ written = 0
+ while remaining > self.buffer_size:
+ try:
+ n = self._write(space, data[written:size - written])
+ except OperationError, e:
+ if not e.match(space, space.gettypeobject(
+ W_BlockingIOError.typedef)):
+ raise
+ written += e.written
+ remaining -= e.written
+ if remaining > self.buffer_size:
+ # Can't buffer everything, still buffer as much as
+ # possible
+ for i in range(self.buffer_size):
+ self.buffer[i] = data[written + i]
+ self.raw_pos = 0
+ self._adjust_position(self.buffer_size)
+ self.write_end = self.buffer_size
+ e.written = written + self.buffer_size
+ raise
+ break
+ written += n
+ remaining -= n
+ # Partial writes can return successfully when interrupted by a
+ # signal (see write(2)). We must run signal handlers before
+ # blocking another time, possibly indefinitely.
+ # XXX PyErr_CheckSignals()
+
+ if self.readable:
+ self._reader_reset_buf()
+ if remaining > 0:
+ for i in range(remaining):
+ self.buffer[i] = data[written + i]
+ written += remaining
+ self.write_pos = 0
+ self.write_end = remaining
+ self._adjust_position(remaining)
+ self.raw_pos = 0
+ return space.wrap(written)
+
+ @unwrap_spec('self', ObjSpace)
+ def flush_w(self, space):
+ self._check_closed(space, "flush of closed file")
+ with self.lock:
+ self._writer_flush_unlocked(space)
+ if self.readable:
+ # Rewind the raw stream so that its position corresponds to
+ # the current logical position.
+ self._raw_seek(space, -self._raw_offset(), 1)
+ self._reader_reset_buf()
+
W_BufferedWriter.typedef = TypeDef(
'BufferedWriter', W_BufferedIOBase.typedef,
__new__ = generic_new_descr(W_BufferedWriter),
+ __init__ = interp2app(W_BufferedWriter.descr_init),
+
+ write = interp2app(W_BufferedWriter.write_w),
+ flush = interp2app(W_BufferedWriter.flush_w),
+
+ # from the mixin class
+ seek = interp2app(W_BufferedWriter.seek_w),
+ close = interp2app(W_BufferedWriter.close_w),
)
class W_BufferedRWPair(W_BufferedIOBase):
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Tue Oct 26 16:49:43 2010
@@ -1,7 +1,8 @@
from pypy.conftest import gettestobjspace
+from pypy.interpreter.gateway import interp2app
from pypy.tool.udir import udir
-class AppTestBufferedIO:
+class AppTestBufferedReader:
def setup_class(cls):
cls.space = gettestobjspace(usemodules=['_io'])
tmpfile = udir.join('tmpfile')
@@ -15,7 +16,7 @@
assert f.read() == "a\nb\nc"
f.close()
#
- raw.seek(0)
+ raw = _io.FileIO(self.tmpfile)
f = _io.BufferedReader(raw)
r = f.read(4)
assert r == "a\nb\n"
@@ -31,3 +32,28 @@
f.seek(-2, 2)
assert f.read() == "\nc"
f.close()
+
+class AppTestBufferedWriter:
+ def setup_class(cls):
+ cls.space = gettestobjspace(usemodules=['_io'])
+ tmpfile = udir.join('tmpfile')
+ cls.w_tmpfile = cls.space.wrap(str(tmpfile))
+ def readfile(space):
+ return space.wrap(tmpfile.read())
+ cls.w_readfile = cls.space.wrap(interp2app(readfile))
+
+ def test_write(self):
+ import _io
+ raw = _io.FileIO(self.tmpfile, 'w')
+ f = _io.BufferedWriter(raw)
+ f.write("abcd")
+ f.close()
+ assert self.readfile() == "abcd"
+
+ def test_largewrite(self):
+ import _io
+ raw = _io.FileIO(self.tmpfile, 'w')
+ f = _io.BufferedWriter(raw)
+ f.write("abcd" * 5000)
+ f.close()
+ assert self.readfile() == "abcd" * 5000
From arigo at codespeak.net Tue Oct 26 17:05:51 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 17:05:51 +0200 (CEST)
Subject: [pypy-svn] r78301 - in pypy/trunk/pypy: interpreter objspace/std
objspace/std/test
Message-ID: <20101026150551.B0CB1282C18@codespeak.net>
Author: arigo
Date: Tue Oct 26 17:05:48 2010
New Revision: 78301
Modified:
pypy/trunk/pypy/interpreter/baseobjspace.py
pypy/trunk/pypy/objspace/std/dictmultiobject.py
pypy/trunk/pypy/objspace/std/dicttype.py
pypy/trunk/pypy/objspace/std/frozensettype.py
pypy/trunk/pypy/objspace/std/setobject.py
pypy/trunk/pypy/objspace/std/settype.py
pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py
pypy/trunk/pypy/objspace/std/test/test_setobject.py
Log:
Merge branch/set-object-cleanup.
General clean-up of setobject.py, reducing the number of copies
of the whole data that we need.
In dicttype.py, remove some very old app-level helpers and implement them
at interp-level.
Modified: pypy/trunk/pypy/interpreter/baseobjspace.py
==============================================================================
--- pypy/trunk/pypy/interpreter/baseobjspace.py (original)
+++ pypy/trunk/pypy/interpreter/baseobjspace.py Tue Oct 26 17:05:48 2010
@@ -739,7 +739,7 @@
def unpackiterable(self, w_iterable, expected_length=-1):
"""Unpack an iterable object into a real (interpreter-level) list.
- Raise a real (subclass of) ValueError if the length is wrong."""
+ Raise an OperationError(w_ValueError) if the length is wrong."""
w_iterator = self.iter(w_iterable)
items = []
while True:
Modified: pypy/trunk/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/trunk/pypy/objspace/std/dictmultiobject.py Tue Oct 26 17:05:48 2010
@@ -641,30 +641,50 @@
init_signature = Signature(['seq_or_map'], None, 'kwargs')
init_defaults = [None]
-def init__DictMulti(space, w_dict, __args__):
- w_src, w_kwds = __args__.parse_obj(
- None, 'dict',
- init_signature, # signature
- init_defaults) # default argument
- if w_src is None:
- pass
- elif space.findattr(w_src, space.wrap("keys")) is None:
- list_of_w_pairs = space.listview(w_src)
- for w_pair in list_of_w_pairs:
+def update1(space, w_dict, w_data):
+ if space.findattr(w_data, space.wrap("keys")) is None:
+ # no 'keys' method, so we assume it is a sequence of pairs
+ for w_pair in space.listview(w_data):
pair = space.fixedview(w_pair)
- if len(pair)!=2:
+ if len(pair) != 2:
raise OperationError(space.w_ValueError,
- space.wrap("dict() takes a sequence of pairs"))
- w_k, w_v = pair
- w_dict.setitem(w_k, w_v)
+ space.wrap("sequence of pairs expected"))
+ w_key, w_value = pair
+ w_dict.setitem(w_key, w_value)
else:
- if space.is_true(w_src):
- from pypy.objspace.std.dicttype import update1
- update1(space, w_dict, w_src)
+ if isinstance(w_data, W_DictMultiObject): # optimization case only
+ update1_dict_dict(space, w_dict, w_data)
+ else:
+ # general case -- "for k in o.keys(): dict.__setitem__(d, k, o[k])"
+ w_keys = space.call_method(w_data, "keys")
+ for w_key in space.listview(w_keys):
+ w_value = space.getitem(w_data, w_key)
+ w_dict.setitem(w_key, w_value)
+
+def update1_dict_dict(space, w_dict, w_data):
+ iterator = w_data.iter()
+ while 1:
+ w_key, w_value = iterator.next()
+ if w_key is None:
+ break
+ w_dict.setitem(w_key, w_value)
+
+def init_or_update(space, w_dict, __args__, funcname):
+ w_src, w_kwds = __args__.parse_obj(
+ None, funcname,
+ init_signature, # signature
+ init_defaults) # default argument
+ if w_src is not None:
+ update1(space, w_dict, w_src)
if space.is_true(w_kwds):
- from pypy.objspace.std.dicttype import update1
update1(space, w_dict, w_kwds)
+def init__DictMulti(space, w_dict, __args__):
+ init_or_update(space, w_dict, __args__, 'dict')
+
+def dict_update__DictMulti(space, w_dict, __args__):
+ init_or_update(space, w_dict, __args__, 'dict.update')
+
def getitem__DictMulti_ANY(space, w_dict, w_key):
w_value = w_dict.getitem(w_key)
if w_value is not None:
@@ -758,9 +778,8 @@
return w_res
def dict_copy__DictMulti(space, w_self):
- from pypy.objspace.std.dicttype import update1
w_new = W_DictMultiObject.allocate_and_init_instance(space)
- update1(space, w_new, w_self)
+ update1_dict_dict(space, w_new, w_self)
return w_new
def dict_items__DictMulti(space, w_self):
@@ -791,6 +810,15 @@
else:
return w_default
+def dict_setdefault__DictMulti_ANY_ANY(space, w_dict, w_key, w_default):
+ # XXX should be more efficient, with only one dict lookup
+ w_value = w_dict.getitem(w_key)
+ if w_value is not None:
+ return w_value
+ else:
+ w_dict.setitem(w_key, w_default)
+ return w_default
+
def dict_pop__DictMulti_ANY(space, w_dict, w_key, w_defaults):
defaults = space.listview(w_defaults)
len_defaults = len(defaults)
@@ -808,6 +836,19 @@
w_dict.delitem(w_key)
return w_item
+def dict_popitem__DictMulti(space, w_dict):
+ # XXX should somehow use the same trick as CPython: saving the index
+ # of the last popped item in the hash table, so that the next call to
+ # popitem() can be more efficient, instead of always starting from the
+ # beginning of the hash table.
+ iterator = w_dict.iter()
+ w_key, w_value = iterator.next()
+ if w_key is None:
+ raise OperationError(space.w_KeyError,
+ space.wrap("popitem(): dictionary is empty"))
+ w_dict.delitem(w_key)
+ return space.newtuple([w_key, w_value])
+
app = gateway.applevel('''
def dictrepr(currently_in_repr, d):
# Now we only handle one implementation of dicts, this one.
Modified: pypy/trunk/pypy/objspace/std/dicttype.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/dicttype.py (original)
+++ pypy/trunk/pypy/objspace/std/dicttype.py Tue Oct 26 17:05:48 2010
@@ -53,106 +53,21 @@
# This can return when multimethods have been fixed
#dict_str = StdObjSpace.str
-# default application-level implementations for some operations
-# most of these (notably not popitem and update*) are overwritten
-# in dictmultiobject
-# gateway is imported in the stdtypedef module
-app = gateway.applevel('''
-
- # in the following functions we use dict.__setitem__ instead of
- # d[k]=... because when a subclass of dict override __setitem__,
- # CPython does not call it when doing builtin operations. The
- # same for other operations.
-
- def update1(d, o):
- if hasattr(o, 'keys'):
- for k in o.keys():
- dict.__setitem__(d, k, o[k])
- else:
- for k,v in o:
- dict.__setitem__(d, k, v)
-
- def update(d, *args, **kwargs):
- len_args = len(args)
- if len_args == 1:
- update1(d, args[0])
- elif len_args > 1:
- raise TypeError("update takes at most 1 (non-keyword) argument")
- if kwargs:
- update1(d, kwargs)
-
- def popitem(d):
- for k in dict.iterkeys(d):
- break
- else:
- raise KeyError("popitem(): dictionary is empty")
- v = dict.__getitem__(d, k)
- dict.__delitem__(d, k)
- return k, v
-
- def get(d, k, v=None):
- if k in d:
- return dict.__getitem__(d, k)
- else:
- return v
-
- def setdefault(d, k, v=None):
- if k in d:
- return dict.__getitem__(d, k)
- else:
- dict.__setitem__(d, k, v)
- return v
-
- def pop(d, k, defaults): # XXX defaults is actually *defaults
- if len(defaults) > 1:
- raise TypeError, "pop expected at most 2 arguments, got %d" % (
- 1 + len(defaults))
- try:
- v = dict.__getitem__(d, k)
- dict.__delitem__(d, k)
- except KeyError, e:
- if defaults:
- return defaults[0]
- else:
- raise e
- return v
-
- def iteritems(d):
- return iter(dict.items(d))
-
- def iterkeys(d):
- return iter(dict.keys(d))
-
- def itervalues(d):
- return iter(dict.values(d))
-''', filename=__file__)
-
-dict_update__ANY = app.interphook("update")
-dict_popitem__ANY = app.interphook("popitem")
-dict_get__ANY_ANY_ANY = app.interphook("get")
-dict_setdefault__ANY_ANY_ANY = app.interphook("setdefault")
-dict_pop__ANY_ANY = app.interphook("pop")
-dict_iteritems__ANY = app.interphook("iteritems")
-dict_iterkeys__ANY = app.interphook("iterkeys")
-dict_itervalues__ANY = app.interphook("itervalues")
-update1 = app.interphook("update1")
-
register_all(vars(), globals())
@gateway.unwrap_spec(ObjSpace, W_Root, W_Root, W_Root)
def descr_fromkeys(space, w_type, w_keys, w_fill=None):
+ from pypy.objspace.std.dictmultiobject import W_DictMultiObject
if w_fill is None:
w_fill = space.w_None
- w_dict = space.call_function(w_type)
- w_iter = space.iter(w_keys)
- while True:
- try:
- w_key = space.next(w_iter)
- except OperationError, e:
- if not e.match(space, space.w_StopIteration):
- raise
- break
- space.setitem(w_dict, w_key, w_fill)
+ if w_type is space.w_dict:
+ w_dict = W_DictMultiObject.allocate_and_init_instance(space, w_type)
+ for w_key in space.listview(w_keys):
+ w_dict.setitem(w_key, w_fill)
+ else:
+ w_dict = space.call_function(w_type)
+ for w_key in space.listview(w_keys):
+ space.setitem(w_dict, w_key, w_fill)
return w_dict
Modified: pypy/trunk/pypy/objspace/std/frozensettype.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/frozensettype.py (original)
+++ pypy/trunk/pypy/objspace/std/frozensettype.py Tue Oct 26 17:05:48 2010
@@ -39,13 +39,13 @@
def descr__frozenset__new__(space, w_frozensettype,
w_iterable=gateway.NoneNotWrapped):
from pypy.objspace.std.setobject import W_FrozensetObject
- from pypy.objspace.std.setobject import _is_frozenset_exact
+ from pypy.objspace.std.setobject import make_setdata_from_w_iterable
if (space.is_w(w_frozensettype, space.w_frozenset) and
- _is_frozenset_exact(w_iterable)):
+ w_iterable is not None and type(w_iterable) is W_FrozensetObject):
return w_iterable
w_obj = space.allocate_instance(W_FrozensetObject, w_frozensettype)
- W_FrozensetObject.__init__(w_obj, space, None)
-
+ data = make_setdata_from_w_iterable(space, w_iterable)
+ W_FrozensetObject.__init__(w_obj, space, data)
return w_obj
frozenset_typedef = StdTypeDef("frozenset",
Modified: pypy/trunk/pypy/objspace/std/setobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/setobject.py (original)
+++ pypy/trunk/pypy/objspace/std/setobject.py Tue Oct 26 17:05:48 2010
@@ -21,11 +21,10 @@
return False
- def __init__(w_self, space, setdata=None):
- if setdata is None:
- w_self.setdata = r_dict(space.eq_w, space.hash_w)
- else:
- w_self.setdata = setdata.copy()
+ def __init__(w_self, space, setdata):
+ """Initialize the set by taking ownership of 'setdata'."""
+ assert setdata is not None
+ w_self.setdata = setdata
def __repr__(w_self):
"""representation for debugging purposes"""
@@ -33,6 +32,7 @@
return "<%s(%s)>" % (w_self.__class__.__name__, ', '.join(reprlist))
def _newobj(w_self, space, rdict_w=None):
+ """Make a new set or frozenset by taking ownership of 'rdict_w'."""
#return space.call(space.type(w_self),W_SetIterObject(rdict_w))
objtype = type(w_self)
if objtype is W_SetObject:
@@ -55,10 +55,7 @@
class W_FrozensetObject(W_BaseSetObject):
from pypy.objspace.std.frozensettype import frozenset_typedef as typedef
-
- def __init__(w_self, space, setdata):
- W_BaseSetObject.__init__(w_self, space, setdata)
- w_self.hash = -1
+ hash = 0
registerimplementation(W_BaseSetObject)
registerimplementation(W_SetObject)
@@ -109,8 +106,14 @@
# some helper functions
+def newset(space):
+ return r_dict(space.eq_w, space.hash_w)
+
def make_setdata_from_w_iterable(space, w_iterable=None):
- data = r_dict(space.eq_w, space.hash_w)
+ """Return a new r_dict with the content of w_iterable."""
+ if isinstance(w_iterable, W_BaseSetObject):
+ return w_iterable.setdata.copy()
+ data = newset(space)
if w_iterable is not None:
for w_item in space.listview(w_iterable):
data[w_item] = None
@@ -119,7 +122,7 @@
def _initialize_set(space, w_obj, w_iterable=None):
w_obj.setdata.clear()
if w_iterable is not None:
- w_obj.setdata.update(make_setdata_from_w_iterable(space, w_iterable))
+ w_obj.setdata = make_setdata_from_w_iterable(space, w_iterable)
def _convert_set_to_frozenset(space, w_obj):
if space.is_true(space.isinstance(w_obj, space.w_set)):
@@ -130,12 +133,6 @@
# helper functions for set operation on dicts
-def _is_frozenset_exact(w_obj):
- if (w_obj is not None) and (type(w_obj) is W_FrozensetObject):
- return True
- else:
- return False
-
def _is_eq(ld, rd):
if len(ld) != len(rd):
return False
@@ -144,66 +141,41 @@
return False
return True
-def _union_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
- else:
- ld = ldict.copy()
- ld.update(rdict)
- return ld, rdict
-
-def _difference_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
- else:
- ld = ldict.copy()
- del_list_w = []
+def _difference_dict(space, ld, rd):
+ result = newset(space)
for w_key in ld:
- if w_key in rdict:
- del_list_w.append(w_key)
- for w_key in del_list_w:
- del ld[w_key]
-
- return ld, rdict
-
-def _intersection_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
- else:
- ld = ldict.copy()
- del_list_w = []
- for w_key in ld:
- if w_key not in rdict:
- del_list_w.append(w_key)
-
- for w_key in del_list_w:
- del ld[w_key]
-
- return ld, rdict
-
+ if w_key not in rd:
+ result[w_key] = None
+ return result
-def _symmetric_difference_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
+def _difference_dict_update(space, ld, rd):
+ if ld is rd:
+ ld.clear() # for the case 'a.difference_update(a)'
else:
- ld = ldict.copy()
- del_list_w = []
- add_list_w = []
+ for w_key in rd:
+ try:
+ del ld[w_key]
+ except KeyError:
+ pass
+
+def _intersection_dict(space, ld, rd):
+ result = newset(space)
+ if len(ld) > len(rd):
+ ld, rd = rd, ld # loop over the smaller dict
for w_key in ld:
- if w_key in rdict:
- del_list_w.append(w_key)
+ if w_key in rd:
+ result[w_key] = None
+ return result
- for w_key in rdict:
+def _symmetric_difference_dict(space, ld, rd):
+ result = newset(space)
+ for w_key in ld:
+ if w_key not in rd:
+ result[w_key] = None
+ for w_key in rd:
if w_key not in ld:
- add_list_w.append(w_key)
-
- for w_key in del_list_w:
- del ld[w_key]
-
- for w_key in add_list_w:
- ld[w_key] = None
-
- return ld, rdict
+ result[w_key] = None
+ return result
def _issubset_dict(ldict, rdict):
if len(ldict) > len(rdict):
@@ -220,14 +192,13 @@
def set_update__Set_BaseSet(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _union_dict(ld, rd, True)
- return space.w_None
+ ld.update(rd)
def set_update__Set_ANY(space, w_left, w_other):
"""Update a set with the union of itself and another."""
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _union_dict(ld, rd, True)
- return space.w_None
+ ld = w_left.setdata
+ for w_item in space.listview(w_other):
+ ld[w_item] = None
def inplace_or__Set_Set(space, w_left, w_other):
set_update__Set_BaseSet(space, w_left, w_other)
@@ -241,25 +212,23 @@
This has no effect if the element is already present.
"""
w_left.setdata[w_other] = None
- return space.w_None
def set_copy__Set(space, w_set):
- return w_set._newobj(space,w_set.setdata)
+ return w_set._newobj(space, w_set.setdata.copy())
def frozenset_copy__Frozenset(space, w_left):
- if _is_frozenset_exact(w_left):
+ if type(w_left) is W_FrozensetObject:
return w_left
else:
- return set_copy__Set(space,w_left)
+ return set_copy__Set(space, w_left)
def set_clear__Set(space, w_left):
w_left.setdata.clear()
- return space.w_None
def set_difference__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _difference_dict(ld, rd, False)
+ new_ld = _difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
set_difference__Set_Frozenset = set_difference__Set_Set
@@ -272,7 +241,7 @@
def set_difference__Set_ANY(space, w_left, w_other):
ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _difference_dict(ld, rd, False)
+ new_ld = _difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
frozenset_difference__Frozenset_ANY = set_difference__Set_ANY
@@ -281,15 +250,17 @@
def set_difference_update__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _difference_dict(ld, rd, True)
- return space.w_None
+ _difference_dict_update(space, ld, rd)
set_difference_update__Set_Frozenset = set_difference_update__Set_Set
def set_difference_update__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _difference_dict(ld, rd, True)
- return space.w_None
+ ld = w_left.setdata
+ for w_key in space.listview(w_other):
+ try:
+ del ld[w_key]
+ except KeyError:
+ pass
def inplace_sub__Set_Set(space, w_left, w_other):
set_difference_update__Set_Set(space, w_left, w_other)
@@ -467,7 +438,7 @@
def hash__Frozenset(space, w_set):
multi = r_uint(1822399083) + r_uint(1822399083) + 1
- if w_set.hash != -1:
+ if w_set.hash != 0:
return space.wrap(w_set.hash)
hash = 1927868237
hash *= (len(w_set.setdata) + 1)
@@ -476,7 +447,7 @@
value = ((h ^ (h << 16) ^ 89869747) * multi)
hash = intmask(hash ^ value)
hash = hash * 69069 + 907133923
- if hash == -1:
+ if hash == 0:
hash = 590923713
hash = intmask(hash)
w_set.hash = hash
@@ -484,29 +455,31 @@
return space.wrap(hash)
def set_pop__Set(space, w_left):
- if len(w_left.setdata) == 0:
+ for w_key in w_left.setdata:
+ break
+ else:
raise OperationError(space.w_KeyError,
space.wrap('pop from an empty set'))
- w_keys = w_left.setdata.keys()
- w_value = w_keys[0]
- del w_left.setdata[w_value]
-
- return w_value
+ del w_left.setdata[w_key]
+ return w_key
def set_intersection__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _intersection_dict(ld, rd, False)
- return w_left._newobj(space,new_ld)
+ new_ld = _intersection_dict(space, ld, rd)
+ return w_left._newobj(space, new_ld)
set_intersection__Set_Frozenset = set_intersection__Set_Set
set_intersection__Frozenset_Frozenset = set_intersection__Set_Set
set_intersection__Frozenset_Set = set_intersection__Set_Set
def set_intersection__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _intersection_dict(ld, rd, False)
- return w_left._newobj(space,new_ld)
+ result = newset(space)
+ ld = w_left.setdata
+ for w_key in space.listview(w_other):
+ if w_key in ld:
+ result[w_key] = None
+ return w_left._newobj(space, result)
frozenset_intersection__Frozenset_ANY = set_intersection__Set_ANY
@@ -518,15 +491,18 @@
def set_intersection_update__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _intersection_dict(ld, rd, True)
- return space.w_None
+ new_ld = _intersection_dict(space, ld, rd)
+ w_left.setdata = new_ld
set_intersection_update__Set_Frozenset = set_intersection_update__Set_Set
def set_intersection_update__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _intersection_dict(ld, rd, True)
- return space.w_None
+ result = newset(space)
+ ld = w_left.setdata
+ for w_key in space.listview(w_other):
+ if w_key in ld:
+ result[w_key] = None
+ w_left.setdata = result
def inplace_and__Set_Set(space, w_left, w_other):
set_intersection_update__Set_Set(space, w_left, w_other)
@@ -537,7 +513,7 @@
def set_symmetric_difference__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _symmetric_difference_dict(ld, rd, False)
+ new_ld = _symmetric_difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
set_symmetric_difference__Set_Frozenset = set_symmetric_difference__Set_Set
@@ -553,7 +529,7 @@
def set_symmetric_difference__Set_ANY(space, w_left, w_other):
ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _symmetric_difference_dict(ld, rd, False)
+ new_ld = _symmetric_difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
frozenset_symmetric_difference__Frozenset_ANY = \
@@ -562,16 +538,16 @@
def set_symmetric_difference_update__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _symmetric_difference_dict(ld, rd, True)
- return space.w_None
+ new_ld = _symmetric_difference_dict(space, ld, rd)
+ w_left.setdata = new_ld
set_symmetric_difference_update__Set_Frozenset = \
set_symmetric_difference_update__Set_Set
def set_symmetric_difference_update__Set_ANY(space, w_left, w_other):
ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _symmetric_difference_dict(ld, rd, True)
- return space.w_None
+ new_ld = _symmetric_difference_dict(space, ld, rd)
+ w_left.setdata = new_ld
def inplace_xor__Set_Set(space, w_left, w_other):
set_symmetric_difference_update__Set_Set(space, w_left, w_other)
@@ -582,8 +558,9 @@
def set_union__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _union_dict(ld, rd, False)
- return w_left._newobj(space, new_ld)
+ result = ld.copy()
+ result.update(rd)
+ return w_left._newobj(space, result)
set_union__Set_Frozenset = set_union__Set_Set
set_union__Frozenset_Set = set_union__Set_Set
@@ -595,9 +572,11 @@
def set_union__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _union_dict(ld, rd, False)
- return w_left._newobj(space, new_ld)
+ ld = w_left.setdata
+ result = ld.copy()
+ for w_key in space.listview(w_other):
+ result[w_key] = None
+ return w_left._newobj(space, result)
frozenset_union__Frozenset_ANY = set_union__Set_ANY
@@ -629,15 +608,6 @@
init_defaults)
_initialize_set(space, w_set, w_iterable)
-def init__Frozenset(space, w_set, __args__):
- w_iterable, = __args__.parse_obj(
- None, 'set',
- init_signature,
- init_defaults)
- if w_set.hash == -1:
- _initialize_set(space, w_set, w_iterable)
- hash__Frozenset(space, w_set)
-
app = gateway.applevel("""
def setrepr(currently_in_repr, s):
'The app-level part of repr().'
Modified: pypy/trunk/pypy/objspace/std/settype.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/settype.py (original)
+++ pypy/trunk/pypy/objspace/std/settype.py Tue Oct 26 17:05:48 2010
@@ -66,9 +66,9 @@
register_all(vars(), globals())
def descr__new__(space, w_settype, __args__):
- from pypy.objspace.std.setobject import W_SetObject
+ from pypy.objspace.std.setobject import W_SetObject, newset
w_obj = space.allocate_instance(W_SetObject, w_settype)
- W_SetObject.__init__(w_obj, space, None)
+ W_SetObject.__init__(w_obj, space, newset(space))
return w_obj
set_typedef = StdTypeDef("set",
Modified: pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py (original)
+++ pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py Tue Oct 26 17:05:48 2010
@@ -238,6 +238,7 @@
it1 = d.popitem()
assert len(d) == 0
assert (it!=it1) and (it1==(1,2) or it1==(3,4))
+ raises(KeyError, d.popitem)
def test_setdefault(self):
d = {1:2, 3:4}
@@ -446,7 +447,9 @@
d1 = {}
d2 = D(a='foo')
d1.update(d2)
- assert d1['a'] == 42 # fails on CPython, d1['a'] == 'foo'
+ assert d1['a'] == 'foo'
+ # a bit of an obscure case: now (from r78295) we get the same result
+ # as CPython does
def test_index_keyerror_unpacking(self):
d = {}
Modified: pypy/trunk/pypy/objspace/std/test/test_setobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/test/test_setobject.py (original)
+++ pypy/trunk/pypy/objspace/std/test/test_setobject.py Tue Oct 26 17:05:48 2010
@@ -10,6 +10,7 @@
import py.test
from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject
from pypy.objspace.std.setobject import _initialize_set
+from pypy.objspace.std.setobject import newset, make_setdata_from_w_iterable
from pypy.objspace.std.setobject import set_intersection__Set_Set
from pypy.objspace.std.setobject import set_intersection__Set_ANY
from pypy.objspace.std.setobject import eq__Set_Set
@@ -28,12 +29,12 @@
self.false = self.space.w_False
def test_and(self):
- s = W_SetObject(self.space, None)
+ s = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, s, self.word)
- t0 = W_SetObject(self.space, None)
+ t0 = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, t0, self.otherword)
- t1 = W_FrozensetObject(self.space, None)
- _initialize_set(self.space, t1, self.otherword)
+ t1 = W_FrozensetObject(self.space,
+ make_setdata_from_w_iterable(self.space, self.otherword))
r0 = set_intersection__Set_Set(self.space, s, t0)
r1 = set_intersection__Set_Set(self.space, s, t1)
assert eq__Set_Set(self.space, r0, r1) == self.true
@@ -41,9 +42,9 @@
assert eq__Set_Set(self.space, r0, sr) == self.true
def test_compare(self):
- s = W_SetObject(self.space, None)
+ s = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, s, self.word)
- t = W_SetObject(self.space, None)
+ t = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, t, self.word)
assert self.space.eq_w(s,t)
u = self.space.wrap(set('simsalabim'))
@@ -56,6 +57,11 @@
b = a | set('abc')
assert type(b) is subset
+ def test_union(self):
+ a = set([4, 5])
+ b = a.union([5, 7])
+ assert sorted(b) == [4, 5, 7]
+
def test_compare(self):
raises(TypeError, cmp, set('abc'), set('abd'))
assert set('abc') != 'abc'
From arigo at codespeak.net Tue Oct 26 17:06:10 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 17:06:10 +0200 (CEST)
Subject: [pypy-svn] r78302 - pypy/branch/set-object-cleanup
Message-ID: <20101026150610.5D9E6282C1A@codespeak.net>
Author: arigo
Date: Tue Oct 26 17:06:09 2010
New Revision: 78302
Removed:
pypy/branch/set-object-cleanup/
Log:
Remove merged branch.
From david at codespeak.net Tue Oct 26 17:12:27 2010
From: david at codespeak.net (david at codespeak.net)
Date: Tue, 26 Oct 2010 17:12:27 +0200 (CEST)
Subject: [pypy-svn] r78303 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101026151227.68AF5282C18@codespeak.net>
Author: david
Date: Tue Oct 26 17:12:25 2010
New Revision: 78303
Added:
pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
Log:
Implemente guard_false, int_sub, int_eq
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Tue Oct 26 17:12:25 2010
@@ -47,9 +47,10 @@
Registers are saved on the stack
XXX Rest to follow"""
i = -1
- fail_index = 0
+ fail_index = -1
while(True):
i += 1
+ fail_index += 1
r = enc[i]
if r == '\xFE':
continue
@@ -64,7 +65,6 @@
value = self.decode32(stack, reg*WORD)
self.fail_boxes_int.setitem(fail_index, value)
- fail_index += 1
assert enc[i] == '\xFF'
descr = self.decode32(enc, i+1)
@@ -116,16 +116,15 @@
if not isinstance(args[i], ConstInt):
curreg = regalloc.try_allocate_reg(args[i])
mem[j] = chr(curreg.value)
- j+=1
+ j += 1
else:
mem[j] = '\xFD'
- j+=1
- self.encode32(mem, j, args[i].getint())
- j+=4
+ self.encode32(mem, j+1, args[i].getint())
+ j += 5
else:
mem[j] = '\xFE'
- j+=1
- i+=1
+ j += 1
+ i += 1
mem[j] = chr(0xFF)
memaddr = rffi.cast(lltype.Signed, mem)
@@ -241,31 +240,70 @@
self.mc.CMP(reg.value, op.getarg(1).getint())
return c.GT
+ def emit_op_int_eq(self, op, regalloc, fcond):
+ reg = regalloc.try_allocate_reg(op.getarg(0))
+ assert isinstance(op.getarg(1), ConstInt)
+ self.mc.CMP(reg.value, op.getarg(1).getint())
+ return c.EQ
+
def emit_op_int_add(self, op, regalloc, fcond):
- if isinstance(op.getarg(0), BoxInt) and isinstance(op.getarg(1), BoxInt):
- explode
+ # assuming only one argument is constant
res = regalloc.try_allocate_reg(op.result)
if isinstance(op.getarg(1), ConstInt):
reg = regalloc.try_allocate_reg(op.getarg(0))
arg1 = op.getarg(1)
+ self.mc.ADD_ri(res.value, reg.value, arg1.getint())
+ elif isinstance(op.getarg(0), ConstInt):
+ reg = regalloc.try_allocate_reg(op.getarg(1))
+ arg1 = op.getarg(0)
+ self.mc.ADD_ri(res.value, reg.value, arg1.getint())
else:
+ r1 = regalloc.try_allocate_reg(op.getarg(0))
+ r2 = regalloc.try_allocate_reg(op.getarg(1))
+ self.mc.ADD_rr(res.value, r1.value, r2.value)
+
+ regalloc.possibly_free_vars_for_op(op)
+ return fcond
+
+ def emit_op_int_sub(self, op, regalloc, fcond):
+ # assuming only one argument is constant
+ res = regalloc.try_allocate_reg(op.result)
+ if isinstance(op.getarg(1), ConstInt):
+ reg = regalloc.try_allocate_reg(op.getarg(0))
+ arg1 = op.getarg(1)
+ self.mc.SUB_ri(res.value, reg.value, arg1.getint())
+ elif isinstance(op.getarg(0), ConstInt):
reg = regalloc.try_allocate_reg(op.getarg(1))
arg1 = op.getarg(0)
+ self.mc.SUB_ri(res.value, reg.value, arg1.getint())
+ else:
+ r1 = regalloc.try_allocate_reg(op.getarg(0))
+ r2 = regalloc.try_allocate_reg(op.getarg(1))
+ self.mc.SUB_rr(res.value, r1.value, r2.value)
- self.mc.ADD_ri(res.value, reg.value, arg1.getint())
regalloc.possibly_free_vars_for_op(op)
return fcond
- def emit_op_guard_true(self, op, regalloc, fcond):
- assert fcond == c.GT
+ # Guards
+ def _emit_guard(self, op, regalloc, fcond):
descr = op.getdescr()
assert isinstance(descr, BasicFailDescr)
descr._arm_guard_code = self.mc.curraddr()
memaddr = self._gen_path_to_exit_path(op, op.getfailargs(), regalloc, fcond)
descr._failure_recovery_code = memaddr
descr._arm_guard_cond = fcond
+
+ def emit_op_guard_true(self, op, regalloc, fcond):
+ assert fcond == c.GT
+ self._emit_guard(op, regalloc, fcond)
return c.AL
+ def emit_op_guard_false(self, op, regalloc, fcond):
+ assert fcond == c.EQ
+ self._emit_guard(op, regalloc, fcond)
+ return c.AL
+
+
def make_operation_list():
def notimplemented(self, op, regalloc, fcond):
raise NotImplementedError
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py Tue Oct 26 17:12:25 2010
@@ -0,0 +1,7 @@
+class AssemblerLocation(object):
+ pass
+
+class RegisterLocation(AssemblerLocation):
+
+ def __init__(self, value):
+ self.value = value
From afa at codespeak.net Tue Oct 26 17:57:45 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 26 Oct 2010 17:57:45 +0200 (CEST)
Subject: [pypy-svn] r78307 - pypy/branch/fast-forward/pypy/module/_io
Message-ID: <20101026155745.73CF5282C18@codespeak.net>
Author: afa
Date: Tue Oct 26 17:57:42 2010
New Revision: 78307
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
Log:
Fix translation
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Tue Oct 26 17:57:42 2010
@@ -486,7 +486,7 @@
written = 0
while remaining > self.buffer_size:
try:
- n = self._write(space, data[written:size - written])
+ n = self._write(space, data[written:])
except OperationError, e:
if not e.match(space, space.gettypeobject(
W_BlockingIOError.typedef)):
From cfbolz at codespeak.net Tue Oct 26 18:12:17 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Tue, 26 Oct 2010 18:12:17 +0200 (CEST)
Subject: [pypy-svn] r78308 - in pypy/trunk/pypy/objspace/std: . test
Message-ID: <20101026161217.8F001282C18@codespeak.net>
Author: cfbolz
Date: Tue Oct 26 18:12:15 2010
New Revision: 78308
Modified:
pypy/trunk/pypy/objspace/std/test/test_mapdict.py
pypy/trunk/pypy/objspace/std/test/test_shadowtracking.py
pypy/trunk/pypy/objspace/std/test/test_versionedtype.py
pypy/trunk/pypy/objspace/std/typeobject.py
Log:
(arigo, cfbolz): after some discussion we decided that a custom metaclass
doesn't interfere with versiontag.
Modified: pypy/trunk/pypy/objspace/std/test/test_mapdict.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/test/test_mapdict.py (original)
+++ pypy/trunk/pypy/objspace/std/test/test_mapdict.py Tue Oct 26 18:12:15 2010
@@ -753,6 +753,25 @@
return a.x
#
res = self.check(f, 'x')
+ assert res == (1, 0, 0)
+ res = self.check(f, 'x')
+ assert res == (0, 1, 0)
+ res = self.check(f, 'x')
+ assert res == (0, 1, 0)
+
+ def test_old_style_base(self):
+ class B:
+ pass
+ class C(object):
+ pass
+ class A(C, B):
+ pass
+ a = A()
+ a.x = 42
+ def f():
+ return a.x
+ #
+ res = self.check(f, 'x')
assert res == (0, 0, 1)
res = self.check(f, 'x')
assert res == (0, 0, 1)
Modified: pypy/trunk/pypy/objspace/std/test/test_shadowtracking.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/test/test_shadowtracking.py (original)
+++ pypy/trunk/pypy/objspace/std/test/test_shadowtracking.py Tue Oct 26 18:12:15 2010
@@ -140,10 +140,11 @@
def test_class_that_cannot_be_cached(self):
import __pypy__
- class metatype(type):
+ class X:
pass
- class A(object):
- __metaclass__ = metatype
+ class Y(object):
+ pass
+ class A(Y, X):
def f(self):
return 42
@@ -241,3 +242,21 @@
foo = 3
D.__bases__ = (C, F)
assert e.foo == 3
+
+ def test_custom_metaclass(self):
+ import __pypy__
+ class MetaA(type):
+ def __getattribute__(self, x):
+ return 1
+ def f(self):
+ return 42
+ A = type.__new__(MetaA, "A", (), {"f": f})
+ l = [type.__getattribute__(A, "__new__")(A)] * 10
+ __pypy__.reset_method_cache_counter()
+ for i, a in enumerate(l):
+ assert a.f() == 42
+ cache_counter = __pypy__.method_cache_counter("f")
+ assert cache_counter[0] >= 5
+ assert cache_counter[1] >= 1 # should be (27, 3)
+ assert sum(cache_counter) == 10
+
Modified: pypy/trunk/pypy/objspace/std/test/test_versionedtype.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/test/test_versionedtype.py (original)
+++ pypy/trunk/pypy/objspace/std/test/test_versionedtype.py Tue Oct 26 18:12:15 2010
@@ -12,10 +12,12 @@
def f(self): pass
class B(A):
pass
- class metatype(type):
+ class X:
+ pass
+ class Y(object):
+ pass
+ class C(Y, X):
pass
- class C(object):
- __metaclass__ = metatype
return A, B, C
""")
return space.unpackiterable(w_types)
Modified: pypy/trunk/pypy/objspace/std/typeobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/typeobject.py (original)
+++ pypy/trunk/pypy/objspace/std/typeobject.py Tue Oct 26 18:12:15 2010
@@ -104,14 +104,12 @@
if overridetypedef is not None:
setup_builtin_type(w_self)
- custom_metaclass = False
else:
setup_user_defined_type(w_self)
- custom_metaclass = not space.is_w(space.type(w_self), space.w_type)
w_self.w_same_layout_as = get_parent_layout(w_self)
if space.config.objspace.std.withtypeversion:
- if custom_metaclass or not is_mro_purely_of_types(w_self.mro_w):
+ if not is_mro_purely_of_types(w_self.mro_w):
pass
else:
# the _version_tag should change, whenever the content of
From cfbolz at codespeak.net Tue Oct 26 18:13:35 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Tue, 26 Oct 2010 18:13:35 +0200 (CEST)
Subject: [pypy-svn] r78309 - pypy/trunk/pypy/objspace/std
Message-ID: <20101026161335.223CB282C18@codespeak.net>
Author: cfbolz
Date: Tue Oct 26 18:13:33 2010
New Revision: 78309
Modified:
pypy/trunk/pypy/objspace/std/typeobject.py
Log:
(arigo, cfbolz): essential optimization: don't intern the slot name if the slot
will be discarded.
Modified: pypy/trunk/pypy/objspace/std/typeobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/typeobject.py (original)
+++ pypy/trunk/pypy/objspace/std/typeobject.py Tue Oct 26 18:13:33 2010
@@ -557,9 +557,9 @@
space.wrap('__slots__ must be identifiers'))
# create member
slot_name = _mangle(slot_name, w_self.name)
- # Force interning of slot names.
- slot_name = space.str_w(space.new_interned_str(slot_name))
if slot_name not in w_self.dict_w:
+ # Force interning of slot names.
+ slot_name = space.str_w(space.new_interned_str(slot_name))
# in cpython it is ignored less, but we probably don't care
member = Member(w_self.nslots, slot_name, w_self)
w_self.dict_w[slot_name] = space.wrap(member)
From arigo at codespeak.net Tue Oct 26 19:06:52 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Tue, 26 Oct 2010 19:06:52 +0200 (CEST)
Subject: [pypy-svn] r78311 - pypy/branch/bisect-module
Message-ID: <20101026170652.EE57D282C18@codespeak.net>
Author: arigo
Date: Tue Oct 26 19:06:51 2010
New Revision: 78311
Added:
pypy/branch/bisect-module/
- copied from r78310, pypy/trunk/
Log:
Write the _bisect module at interp-level.
From afa at codespeak.net Tue Oct 26 19:30:11 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 26 Oct 2010 19:30:11 +0200 (CEST)
Subject: [pypy-svn] r78313 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101026173011.B8F81282C18@codespeak.net>
Author: afa
Date: Tue Oct 26 19:30:09 2010
New Revision: 78313
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
Log:
Fix properties of incompletely initialized Buffered* objects.
This is especially important to avoid segfaults in __del__...
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Tue Oct 26 19:30:09 2010
@@ -1,6 +1,6 @@
from __future__ import with_statement
from pypy.interpreter.typedef import (
- TypeDef, generic_new_descr)
+ TypeDef, GetSetProperty, generic_new_descr)
from pypy.interpreter.gateway import interp2app, unwrap_spec
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
from pypy.interpreter.error import OperationError, operationerrfmt
@@ -11,12 +11,15 @@
from pypy.module._io.interp_io import DEFAULT_BUFFER_SIZE, W_BlockingIOError
from pypy.module.thread.os_lock import Lock
+STATE_ZERO, STATE_OK, STATE_DETACHED = range(3)
+
class BlockingIOError(Exception):
pass
class W_BufferedIOBase(W_IOBase):
def __init__(self, space):
W_IOBase.__init__(self, space)
+ self.state = STATE_ZERO
self.buffer = lltype.nullptr(rffi.CCHARP.TO)
self.pos = 0 # Current logical position in the buffer
@@ -84,6 +87,18 @@
except OperationError:
pass
+ def _check_init(self, space):
+ if self.state == STATE_ZERO:
+ raise OperationError(space.w_ValueError, space.wrap(
+ "I/O operation on uninitialized object"))
+ elif self.state == STATE_DETACHED:
+ raise OperationError(space.w_ValueError, space.wrap(
+ "raw stream has been detached"))
+
+ def _check_closed(self, space, message=None):
+ self._check_init(space)
+ W_IOBase._check_closed(self, space, message)
+
def _raw_tell(self, space):
w_pos = space.call_method(self.raw, "tell")
pos = space.r_longlong_w(w_pos)
@@ -94,6 +109,10 @@
self.abs_pos = pos
return pos
+ def closed_get_w(space, self):
+ self._check_init(space)
+ return space.getattr(self.raw, space.wrap("closed"))
+
def _readahead(self):
if self.readable and self.read_end != -1:
return self.read_end - self.pos
@@ -155,6 +174,7 @@
@unwrap_spec('self', ObjSpace)
def close_w(self, space):
+ self._check_init(space)
with self.lock:
if self._closed(space):
return
@@ -164,6 +184,7 @@
@unwrap_spec('self', ObjSpace)
def flush_w(self, space):
+ self._check_init(space)
return space.call_method(self.raw, "flush")
def _writer_flush_unlocked(self, space, restore_pos=False):
@@ -223,7 +244,7 @@
l.append(self.buffer[i])
return self._write(space, ''.join(l))
-class W_BufferedReader(W_BufferedIOBase, BufferedMixin):
+class W_BufferedReader(BufferedMixin, W_BufferedIOBase):
def __init__(self, space):
W_BufferedIOBase.__init__(self, space)
self.ok = False
@@ -231,6 +252,7 @@
@unwrap_spec('self', ObjSpace, W_Root, int)
def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE):
+ self.state = STATE_ZERO
raw = space.interp_w(W_IOBase, w_raw)
raw.check_readable_w(space)
@@ -240,6 +262,7 @@
self._init(space)
self._reader_reset_buf()
+ self.state = STATE_OK
@unwrap_spec('self', ObjSpace, W_Root)
def read_w(self, space, w_size=None):
@@ -397,11 +420,13 @@
seek = interp2app(W_BufferedReader.seek_w),
close = interp2app(W_BufferedReader.close_w),
flush = interp2app(W_BufferedReader.flush_w),
+ closed = GetSetProperty(W_BufferedReader.closed_get_w),
)
-class W_BufferedWriter(W_BufferedIOBase, BufferedMixin):
+class W_BufferedWriter(BufferedMixin, W_BufferedIOBase):
@unwrap_spec('self', ObjSpace, W_Root, int)
def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE):
+ self.state = STATE_ZERO
raw = space.interp_w(W_IOBase, w_raw)
raw.check_writable_w(space)
@@ -411,6 +436,7 @@
self._init(space)
self._writer_reset_buf()
+ self.state = STATE_OK
def _adjust_position(self, new_pos):
self.pos = new_pos
@@ -545,6 +571,7 @@
# from the mixin class
seek = interp2app(W_BufferedWriter.seek_w),
close = interp2app(W_BufferedWriter.close_w),
+ closed = GetSetProperty(W_BufferedWriter.closed_get_w),
)
class W_BufferedRWPair(W_BufferedIOBase):
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Tue Oct 26 19:30:09 2010
@@ -1,4 +1,4 @@
-from pypy.conftest import gettestobjspace
+from pypy.conftest import gettestobjspace, option
from pypy.interpreter.gateway import interp2app
from pypy.tool.udir import udir
@@ -38,9 +38,12 @@
cls.space = gettestobjspace(usemodules=['_io'])
tmpfile = udir.join('tmpfile')
cls.w_tmpfile = cls.space.wrap(str(tmpfile))
- def readfile(space):
- return space.wrap(tmpfile.read())
- cls.w_readfile = cls.space.wrap(interp2app(readfile))
+ if option.runappdirect:
+ cls.w_readfile = tmpfile.read
+ else:
+ def readfile(space):
+ return space.wrap(tmpfile.read())
+ cls.w_readfile = cls.space.wrap(interp2app(readfile))
def test_write(self):
import _io
@@ -57,3 +60,12 @@
f.write("abcd" * 5000)
f.close()
assert self.readfile() == "abcd" * 5000
+
+ def test_incomplete(self):
+ import _io
+ raw = _io.FileIO(self.tmpfile)
+ b = _io.BufferedWriter.__new__(_io.BufferedWriter)
+ raises(IOError, b.__init__, raw) # because file is not writable
+ raises(ValueError, getattr, b, 'closed')
+ raises(ValueError, b.flush)
+ raises(ValueError, b.close)
From afa at codespeak.net Tue Oct 26 19:56:02 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 26 Oct 2010 19:56:02 +0200 (CEST)
Subject: [pypy-svn] r78314 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101026175602.E77EF282C18@codespeak.net>
Author: afa
Date: Tue Oct 26 19:56:01 2010
New Revision: 78314
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
Log:
Test and fix
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Tue Oct 26 19:56:01 2010
@@ -451,8 +451,8 @@
with self.lock:
- if (not (self.readable and self.read_end == -1) and
- not (self.writable and self.write_end == -1)):
+ if (not (self.readable and self.read_end != -1) and
+ not (self.writable and self.write_end != -1)):
self.pos = 0
self.raw_pos = 0
available = self.buffer_size - self.pos
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Tue Oct 26 19:56:01 2010
@@ -69,3 +69,33 @@
raises(ValueError, getattr, b, 'closed')
raises(ValueError, b.flush)
raises(ValueError, b.close)
+
+ def test_check_several_writes(self):
+ import _io
+ raw = _io.FileIO(self.tmpfile, 'w')
+ b = _io.BufferedWriter(raw, 13)
+
+ for i in range(4):
+ assert b.write('x' * 10) == 10
+ b.flush()
+ assert self.readfile() == 'x' * 40
+
+ def test_destructor(self):
+ import _io
+
+ record = []
+ class MyIO(_io.BufferedWriter):
+ def __del__(self):
+ record.append(1)
+ super(MyIO, self).__del__()
+ def close(self):
+ record.append(2)
+ super(MyIO, self).close()
+ def flush(self):
+ record.append(3)
+ super(MyIO, self).flush()
+ raw = _io.FileIO(self.tmpfile, 'w')
+ MyIO(raw)
+ import gc; gc.collect()
+ assert record == [1, 2, 3]
+
From hakanardo at codespeak.net Tue Oct 26 19:57:58 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Tue, 26 Oct 2010 19:57:58 +0200 (CEST)
Subject: [pypy-svn] r78315 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test
Message-ID: <20101026175758.E42D8282C18@codespeak.net>
Author: hakanardo
Date: Tue Oct 26 19:57:57 2010
New Revision: 78315
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py
Log:
virtuals with circular refs
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Tue Oct 26 19:57:57 2010
@@ -47,9 +47,14 @@
def get_key_box(self):
return self.box
- # FIXME: Reise get_args_for_fail instead?
- def get_forced_boxes(self):
- return [self.force_box()]
+ # FIXME: Reuse get_args_for_fail instead?
+ def get_forced_boxes(self, already_seen):
+ key = self.get_key_box()
+ if key in already_seen:
+ return []
+ box = self.force_box()
+ already_seen.append(self.get_key_box())
+ return [box]
def get_args_for_fail(self, modifier):
pass
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Tue Oct 26 19:57:57 2010
@@ -52,7 +52,7 @@
inputargs = []
for arg in jump_args:
- for a in self.getvalue(arg).get_forced_boxes():
+ for a in self.getvalue(arg).get_forced_boxes([]):
if not isinstance(a, Const):
inputargs.append(a)
@@ -75,7 +75,7 @@
args = []
#for arg in newop.getarglist():
for arg in [argmap[a] for a in inputargs]:
- args.extend(self.getvalue(arg).get_forced_boxes())
+ args.extend(self.getvalue(arg).get_forced_boxes([]))
newop.initarglist(args + inputargs[len(args):])
#print 'P: ', str(newop)
@@ -104,7 +104,7 @@
jmp = self.optimizer.newoperations[-1]
if jmp.getopnum() == rop.JUMP:
newval = self.getvalue(argmap[a])
- newarg = newval.get_forced_boxes()
+ newarg = newval.get_forced_boxes([])
jmp.initarglist(jmp.getarglist() + newarg)
return inputargs
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Tue Oct 26 19:57:57 2010
@@ -121,13 +121,17 @@
fieldvalue = self._fields[ofs]
fieldvalue.get_args_for_fail(modifier)
- # FIXME: circular references
- def get_forced_boxes(self):
+ def get_forced_boxes(self, already_seen):
+ key = self.get_key_box()
+ if key in already_seen:
+ return []
+ already_seen.append(key)
if self.box is None:
lst = self._get_field_descr_list()
fieldboxes = []
for ofs in lst:
- fieldboxes.extend(self._fields[ofs].get_forced_boxes())
+ boxes = self._fields[ofs].get_forced_boxes(already_seen)
+ fieldboxes.extend(boxes)
return fieldboxes
else:
return [self.box]
@@ -204,11 +208,15 @@
def _make_virtual(self, modifier):
return modifier.make_varray(self.arraydescr)
- def get_forced_boxes(self):
+ def get_forced_boxes(self, already_seen):
+ key = self.get_key_box()
+ if key in already_seen:
+ return []
+ already_seen.append(key)
if self.box is None:
boxes = []
for itemvalue in self._items:
- boxes.extend(itemvalue.get_forced_boxes())
+ boxes.extend(itemvalue.get_forced_boxes(already_seen))
return boxes
else:
return [self.box]
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py Tue Oct 26 19:57:57 2010
@@ -34,6 +34,32 @@
self.check_loops(new=0, new_with_vtable=0,
getfield_gc=0, setfield_gc=0)
+ def test_virtualized_circular1(self):
+ class MyNode():
+ pass
+ myjitdriver = JitDriver(greens = [], reds = ['n', 'node'])
+ def f(n):
+ node = MyNode()
+ node.value = 0
+ node.extra = 0
+ node.ref = node
+ while n > 0:
+ myjitdriver.can_enter_jit(n=n, node=node)
+ myjitdriver.jit_merge_point(n=n, node=node)
+ next = MyNode()
+ next.value = node.ref.value + n
+ next.extra = node.ref.extra + 1
+ next.ref = next
+ node = next
+ n -= 1
+ return node.value * node.extra
+ assert f(10) == 55 * 10
+ res = self.meta_interp(f, [10])
+ assert res == 55 * 10
+ self.check_loop_count(1)
+ self.check_loops(new=0, new_with_vtable=0,
+ getfield_gc=0, setfield_gc=0)
+
def test_virtualized_float(self):
myjitdriver = JitDriver(greens = [], reds = ['n', 'node'])
def f(n):
From afa at codespeak.net Tue Oct 26 20:47:02 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 26 Oct 2010 20:47:02 +0200 (CEST)
Subject: [pypy-svn] r78316 - in pypy/branch/fast-forward/pypy: config
interpreter objspace/std objspace/std/test objspace/test
tool/release tool/release/test
Message-ID: <20101026184702.640E8282C18@codespeak.net>
Author: afa
Date: Tue Oct 26 20:46:59 2010
New Revision: 78316
Modified:
pypy/branch/fast-forward/pypy/config/pypyoption.py
pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py
pypy/branch/fast-forward/pypy/interpreter/function.py
pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py
pypy/branch/fast-forward/pypy/objspace/std/dicttype.py
pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py
pypy/branch/fast-forward/pypy/objspace/std/setobject.py
pypy/branch/fast-forward/pypy/objspace/std/settype.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_mapdict.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_shadowtracking.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_versionedtype.py
pypy/branch/fast-forward/pypy/objspace/std/typeobject.py
pypy/branch/fast-forward/pypy/objspace/test/test_descroperation.py
pypy/branch/fast-forward/pypy/tool/release/ (props changed)
pypy/branch/fast-forward/pypy/tool/release/__init__.py (props changed)
pypy/branch/fast-forward/pypy/tool/release/force-builds.py (props changed)
pypy/branch/fast-forward/pypy/tool/release/make_release.py (props changed)
pypy/branch/fast-forward/pypy/tool/release/test/ (props changed)
pypy/branch/fast-forward/pypy/tool/release/test/__init__.py (props changed)
pypy/branch/fast-forward/pypy/tool/release/test/test_make_release.py (props changed)
Log:
Merge from trunk: -r78251:78315
Modified: pypy/branch/fast-forward/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/fast-forward/pypy/config/pypyoption.py (original)
+++ pypy/branch/fast-forward/pypy/config/pypyoption.py Tue Oct 26 20:46:59 2010
@@ -76,6 +76,7 @@
# no _rawffi if importing pypy.rlib.clibffi raises ImportError
# or CompilationError
"_rawffi" : ["pypy.rlib.clibffi"],
+ "_ffi" : ["pypy.rlib.clibffi"],
"zlib" : ["pypy.rlib.rzlib"],
"bz2" : ["pypy.module.bz2.interp_bz2"],
Modified: pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py Tue Oct 26 20:46:59 2010
@@ -739,7 +739,7 @@
def unpackiterable(self, w_iterable, expected_length=-1):
"""Unpack an iterable object into a real (interpreter-level) list.
- Raise a real (subclass of) ValueError if the length is wrong."""
+ Raise an OperationError(w_ValueError) if the length is wrong."""
w_iterator = self.iter(w_iterable)
items = []
while True:
Modified: pypy/branch/fast-forward/pypy/interpreter/function.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/function.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/function.py Tue Oct 26 20:46:59 2010
@@ -589,12 +589,14 @@
w_klass = space.type(w_obj)
return space.wrap(Method(space, self.w_function, w_klass, space.w_None))
- def descr_classmethod__new__(space, w_type, w_function):
+ def descr_classmethod__new__(space, w_subtype, w_function):
if not space.is_true(space.callable(w_function)):
typename = space.type(w_function).getname(space, '?')
raise operationerrfmt(space.w_TypeError,
"'%s' object is not callable", typename)
- return space.wrap(ClassMethod(w_function))
+ instance = space.allocate_instance(ClassMethod, w_subtype)
+ instance.__init__(w_function)
+ return space.wrap(instance)
class FunctionWithFixedCode(Function):
can_change_code = False
Modified: pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py Tue Oct 26 20:46:59 2010
@@ -641,30 +641,50 @@
init_signature = Signature(['seq_or_map'], None, 'kwargs')
init_defaults = [None]
-def init__DictMulti(space, w_dict, __args__):
- w_src, w_kwds = __args__.parse_obj(
- None, 'dict',
- init_signature, # signature
- init_defaults) # default argument
- if w_src is None:
- pass
- elif space.findattr(w_src, space.wrap("keys")) is None:
- list_of_w_pairs = space.listview(w_src)
- for w_pair in list_of_w_pairs:
+def update1(space, w_dict, w_data):
+ if space.findattr(w_data, space.wrap("keys")) is None:
+ # no 'keys' method, so we assume it is a sequence of pairs
+ for w_pair in space.listview(w_data):
pair = space.fixedview(w_pair)
- if len(pair)!=2:
+ if len(pair) != 2:
raise OperationError(space.w_ValueError,
- space.wrap("dict() takes a sequence of pairs"))
- w_k, w_v = pair
- w_dict.setitem(w_k, w_v)
+ space.wrap("sequence of pairs expected"))
+ w_key, w_value = pair
+ w_dict.setitem(w_key, w_value)
else:
- if space.is_true(w_src):
- from pypy.objspace.std.dicttype import update1
- update1(space, w_dict, w_src)
+ if isinstance(w_data, W_DictMultiObject): # optimization case only
+ update1_dict_dict(space, w_dict, w_data)
+ else:
+ # general case -- "for k in o.keys(): dict.__setitem__(d, k, o[k])"
+ w_keys = space.call_method(w_data, "keys")
+ for w_key in space.listview(w_keys):
+ w_value = space.getitem(w_data, w_key)
+ w_dict.setitem(w_key, w_value)
+
+def update1_dict_dict(space, w_dict, w_data):
+ iterator = w_data.iter()
+ while 1:
+ w_key, w_value = iterator.next()
+ if w_key is None:
+ break
+ w_dict.setitem(w_key, w_value)
+
+def init_or_update(space, w_dict, __args__, funcname):
+ w_src, w_kwds = __args__.parse_obj(
+ None, funcname,
+ init_signature, # signature
+ init_defaults) # default argument
+ if w_src is not None:
+ update1(space, w_dict, w_src)
if space.is_true(w_kwds):
- from pypy.objspace.std.dicttype import update1
update1(space, w_dict, w_kwds)
+def init__DictMulti(space, w_dict, __args__):
+ init_or_update(space, w_dict, __args__, 'dict')
+
+def dict_update__DictMulti(space, w_dict, __args__):
+ init_or_update(space, w_dict, __args__, 'dict.update')
+
def getitem__DictMulti_ANY(space, w_dict, w_key):
w_value = w_dict.getitem(w_key)
if w_value is not None:
@@ -758,9 +778,8 @@
return w_res
def dict_copy__DictMulti(space, w_self):
- from pypy.objspace.std.dicttype import update1
w_new = W_DictMultiObject.allocate_and_init_instance(space)
- update1(space, w_new, w_self)
+ update1_dict_dict(space, w_new, w_self)
return w_new
def dict_items__DictMulti(space, w_self):
@@ -791,6 +810,15 @@
else:
return w_default
+def dict_setdefault__DictMulti_ANY_ANY(space, w_dict, w_key, w_default):
+ # XXX should be more efficient, with only one dict lookup
+ w_value = w_dict.getitem(w_key)
+ if w_value is not None:
+ return w_value
+ else:
+ w_dict.setitem(w_key, w_default)
+ return w_default
+
def dict_pop__DictMulti_ANY(space, w_dict, w_key, w_defaults):
defaults = space.listview(w_defaults)
len_defaults = len(defaults)
@@ -808,6 +836,19 @@
w_dict.delitem(w_key)
return w_item
+def dict_popitem__DictMulti(space, w_dict):
+ # XXX should somehow use the same trick as CPython: saving the index
+ # of the last popped item in the hash table, so that the next call to
+ # popitem() can be more efficient, instead of always starting from the
+ # beginning of the hash table.
+ iterator = w_dict.iter()
+ w_key, w_value = iterator.next()
+ if w_key is None:
+ raise OperationError(space.w_KeyError,
+ space.wrap("popitem(): dictionary is empty"))
+ w_dict.delitem(w_key)
+ return space.newtuple([w_key, w_value])
+
app = gateway.applevel('''
def dictrepr(currently_in_repr, d):
# Now we only handle one implementation of dicts, this one.
Modified: pypy/branch/fast-forward/pypy/objspace/std/dicttype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/dicttype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/dicttype.py Tue Oct 26 20:46:59 2010
@@ -53,106 +53,21 @@
# This can return when multimethods have been fixed
#dict_str = StdObjSpace.str
-# default application-level implementations for some operations
-# most of these (notably not popitem and update*) are overwritten
-# in dictmultiobject
-# gateway is imported in the stdtypedef module
-app = gateway.applevel('''
-
- # in the following functions we use dict.__setitem__ instead of
- # d[k]=... because when a subclass of dict override __setitem__,
- # CPython does not call it when doing builtin operations. The
- # same for other operations.
-
- def update1(d, o):
- if hasattr(o, 'keys'):
- for k in o.keys():
- dict.__setitem__(d, k, o[k])
- else:
- for k,v in o:
- dict.__setitem__(d, k, v)
-
- def update(d, *args, **kwargs):
- len_args = len(args)
- if len_args == 1:
- update1(d, args[0])
- elif len_args > 1:
- raise TypeError("update takes at most 1 (non-keyword) argument")
- if kwargs:
- update1(d, kwargs)
-
- def popitem(d):
- for k in dict.iterkeys(d):
- break
- else:
- raise KeyError("popitem(): dictionary is empty")
- v = dict.__getitem__(d, k)
- dict.__delitem__(d, k)
- return k, v
-
- def get(d, k, v=None):
- if k in d:
- return dict.__getitem__(d, k)
- else:
- return v
-
- def setdefault(d, k, v=None):
- if k in d:
- return dict.__getitem__(d, k)
- else:
- dict.__setitem__(d, k, v)
- return v
-
- def pop(d, k, defaults): # XXX defaults is actually *defaults
- if len(defaults) > 1:
- raise TypeError, "pop expected at most 2 arguments, got %d" % (
- 1 + len(defaults))
- try:
- v = dict.__getitem__(d, k)
- dict.__delitem__(d, k)
- except KeyError, e:
- if defaults:
- return defaults[0]
- else:
- raise e
- return v
-
- def iteritems(d):
- return iter(dict.items(d))
-
- def iterkeys(d):
- return iter(dict.keys(d))
-
- def itervalues(d):
- return iter(dict.values(d))
-''', filename=__file__)
-
-dict_update__ANY = app.interphook("update")
-dict_popitem__ANY = app.interphook("popitem")
-dict_get__ANY_ANY_ANY = app.interphook("get")
-dict_setdefault__ANY_ANY_ANY = app.interphook("setdefault")
-dict_pop__ANY_ANY = app.interphook("pop")
-dict_iteritems__ANY = app.interphook("iteritems")
-dict_iterkeys__ANY = app.interphook("iterkeys")
-dict_itervalues__ANY = app.interphook("itervalues")
-update1 = app.interphook("update1")
-
register_all(vars(), globals())
@gateway.unwrap_spec(ObjSpace, W_Root, W_Root, W_Root)
def descr_fromkeys(space, w_type, w_keys, w_fill=None):
+ from pypy.objspace.std.dictmultiobject import W_DictMultiObject
if w_fill is None:
w_fill = space.w_None
- w_dict = space.call_function(w_type)
- w_iter = space.iter(w_keys)
- while True:
- try:
- w_key = space.next(w_iter)
- except OperationError, e:
- if not e.match(space, space.w_StopIteration):
- raise
- break
- space.setitem(w_dict, w_key, w_fill)
+ if w_type is space.w_dict:
+ w_dict = W_DictMultiObject.allocate_and_init_instance(space, w_type)
+ for w_key in space.listview(w_keys):
+ w_dict.setitem(w_key, w_fill)
+ else:
+ w_dict = space.call_function(w_type)
+ for w_key in space.listview(w_keys):
+ space.setitem(w_dict, w_key, w_fill)
return w_dict
Modified: pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py Tue Oct 26 20:46:59 2010
@@ -39,13 +39,13 @@
def descr__frozenset__new__(space, w_frozensettype,
w_iterable=gateway.NoneNotWrapped):
from pypy.objspace.std.setobject import W_FrozensetObject
- from pypy.objspace.std.setobject import _is_frozenset_exact
+ from pypy.objspace.std.setobject import make_setdata_from_w_iterable
if (space.is_w(w_frozensettype, space.w_frozenset) and
- _is_frozenset_exact(w_iterable)):
+ w_iterable is not None and type(w_iterable) is W_FrozensetObject):
return w_iterable
w_obj = space.allocate_instance(W_FrozensetObject, w_frozensettype)
- W_FrozensetObject.__init__(w_obj, space, None)
-
+ data = make_setdata_from_w_iterable(space, w_iterable)
+ W_FrozensetObject.__init__(w_obj, space, data)
return w_obj
frozenset_typedef = StdTypeDef("frozenset",
Modified: pypy/branch/fast-forward/pypy/objspace/std/setobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/setobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/setobject.py Tue Oct 26 20:46:59 2010
@@ -21,11 +21,10 @@
return False
- def __init__(w_self, space, setdata=None):
- if setdata is None:
- w_self.setdata = r_dict(space.eq_w, space.hash_w)
- else:
- w_self.setdata = setdata.copy()
+ def __init__(w_self, space, setdata):
+ """Initialize the set by taking ownership of 'setdata'."""
+ assert setdata is not None
+ w_self.setdata = setdata
def __repr__(w_self):
"""representation for debugging purposes"""
@@ -33,6 +32,7 @@
return "<%s(%s)>" % (w_self.__class__.__name__, ', '.join(reprlist))
def _newobj(w_self, space, rdict_w=None):
+ """Make a new set or frozenset by taking ownership of 'rdict_w'."""
#return space.call(space.type(w_self),W_SetIterObject(rdict_w))
objtype = type(w_self)
if objtype is W_SetObject:
@@ -55,10 +55,7 @@
class W_FrozensetObject(W_BaseSetObject):
from pypy.objspace.std.frozensettype import frozenset_typedef as typedef
-
- def __init__(w_self, space, setdata):
- W_BaseSetObject.__init__(w_self, space, setdata)
- w_self.hash = -1
+ hash = 0
registerimplementation(W_BaseSetObject)
registerimplementation(W_SetObject)
@@ -109,8 +106,14 @@
# some helper functions
+def newset(space):
+ return r_dict(space.eq_w, space.hash_w)
+
def make_setdata_from_w_iterable(space, w_iterable=None):
- data = r_dict(space.eq_w, space.hash_w)
+ """Return a new r_dict with the content of w_iterable."""
+ if isinstance(w_iterable, W_BaseSetObject):
+ return w_iterable.setdata.copy()
+ data = newset(space)
if w_iterable is not None:
for w_item in space.listview(w_iterable):
data[w_item] = None
@@ -119,7 +122,7 @@
def _initialize_set(space, w_obj, w_iterable=None):
w_obj.setdata.clear()
if w_iterable is not None:
- w_obj.setdata.update(make_setdata_from_w_iterable(space, w_iterable))
+ w_obj.setdata = make_setdata_from_w_iterable(space, w_iterable)
def _convert_set_to_frozenset(space, w_obj):
if space.is_true(space.isinstance(w_obj, space.w_set)):
@@ -130,12 +133,6 @@
# helper functions for set operation on dicts
-def _is_frozenset_exact(w_obj):
- if (w_obj is not None) and (type(w_obj) is W_FrozensetObject):
- return True
- else:
- return False
-
def _is_eq(ld, rd):
if len(ld) != len(rd):
return False
@@ -144,66 +141,41 @@
return False
return True
-def _union_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
- else:
- ld = ldict.copy()
- ld.update(rdict)
- return ld, rdict
-
-def _difference_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
- else:
- ld = ldict.copy()
- del_list_w = []
+def _difference_dict(space, ld, rd):
+ result = newset(space)
for w_key in ld:
- if w_key in rdict:
- del_list_w.append(w_key)
- for w_key in del_list_w:
- del ld[w_key]
-
- return ld, rdict
-
-def _intersection_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
- else:
- ld = ldict.copy()
- del_list_w = []
- for w_key in ld:
- if w_key not in rdict:
- del_list_w.append(w_key)
-
- for w_key in del_list_w:
- del ld[w_key]
-
- return ld, rdict
-
+ if w_key not in rd:
+ result[w_key] = None
+ return result
-def _symmetric_difference_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
+def _difference_dict_update(space, ld, rd):
+ if ld is rd:
+ ld.clear() # for the case 'a.difference_update(a)'
else:
- ld = ldict.copy()
- del_list_w = []
- add_list_w = []
+ for w_key in rd:
+ try:
+ del ld[w_key]
+ except KeyError:
+ pass
+
+def _intersection_dict(space, ld, rd):
+ result = newset(space)
+ if len(ld) > len(rd):
+ ld, rd = rd, ld # loop over the smaller dict
for w_key in ld:
- if w_key in rdict:
- del_list_w.append(w_key)
+ if w_key in rd:
+ result[w_key] = None
+ return result
- for w_key in rdict:
+def _symmetric_difference_dict(space, ld, rd):
+ result = newset(space)
+ for w_key in ld:
+ if w_key not in rd:
+ result[w_key] = None
+ for w_key in rd:
if w_key not in ld:
- add_list_w.append(w_key)
-
- for w_key in del_list_w:
- del ld[w_key]
-
- for w_key in add_list_w:
- ld[w_key] = None
-
- return ld, rdict
+ result[w_key] = None
+ return result
def _issubset_dict(ldict, rdict):
if len(ldict) > len(rdict):
@@ -220,14 +192,13 @@
def set_update__Set_BaseSet(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _union_dict(ld, rd, True)
- return space.w_None
+ ld.update(rd)
def set_update__Set_ANY(space, w_left, w_other):
"""Update a set with the union of itself and another."""
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _union_dict(ld, rd, True)
- return space.w_None
+ ld = w_left.setdata
+ for w_item in space.listview(w_other):
+ ld[w_item] = None
def inplace_or__Set_Set(space, w_left, w_other):
set_update__Set_BaseSet(space, w_left, w_other)
@@ -241,25 +212,23 @@
This has no effect if the element is already present.
"""
w_left.setdata[w_other] = None
- return space.w_None
def set_copy__Set(space, w_set):
- return w_set._newobj(space,w_set.setdata)
+ return w_set._newobj(space, w_set.setdata.copy())
def frozenset_copy__Frozenset(space, w_left):
- if _is_frozenset_exact(w_left):
+ if type(w_left) is W_FrozensetObject:
return w_left
else:
- return set_copy__Set(space,w_left)
+ return set_copy__Set(space, w_left)
def set_clear__Set(space, w_left):
w_left.setdata.clear()
- return space.w_None
def set_difference__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _difference_dict(ld, rd, False)
+ new_ld = _difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
set_difference__Set_Frozenset = set_difference__Set_Set
@@ -272,7 +241,7 @@
def set_difference__Set_ANY(space, w_left, w_other):
ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _difference_dict(ld, rd, False)
+ new_ld = _difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
frozenset_difference__Frozenset_ANY = set_difference__Set_ANY
@@ -281,15 +250,17 @@
def set_difference_update__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _difference_dict(ld, rd, True)
- return space.w_None
+ _difference_dict_update(space, ld, rd)
set_difference_update__Set_Frozenset = set_difference_update__Set_Set
def set_difference_update__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _difference_dict(ld, rd, True)
- return space.w_None
+ ld = w_left.setdata
+ for w_key in space.listview(w_other):
+ try:
+ del ld[w_key]
+ except KeyError:
+ pass
def inplace_sub__Set_Set(space, w_left, w_other):
set_difference_update__Set_Set(space, w_left, w_other)
@@ -467,7 +438,7 @@
def hash__Frozenset(space, w_set):
multi = r_uint(1822399083) + r_uint(1822399083) + 1
- if w_set.hash != -1:
+ if w_set.hash != 0:
return space.wrap(w_set.hash)
hash = 1927868237
hash *= (len(w_set.setdata) + 1)
@@ -476,7 +447,7 @@
value = ((h ^ (h << 16) ^ 89869747) * multi)
hash = intmask(hash ^ value)
hash = hash * 69069 + 907133923
- if hash == -1:
+ if hash == 0:
hash = 590923713
hash = intmask(hash)
w_set.hash = hash
@@ -484,29 +455,31 @@
return space.wrap(hash)
def set_pop__Set(space, w_left):
- if len(w_left.setdata) == 0:
+ for w_key in w_left.setdata:
+ break
+ else:
raise OperationError(space.w_KeyError,
space.wrap('pop from an empty set'))
- w_keys = w_left.setdata.keys()
- w_value = w_keys[0]
- del w_left.setdata[w_value]
-
- return w_value
+ del w_left.setdata[w_key]
+ return w_key
def set_intersection__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _intersection_dict(ld, rd, False)
- return w_left._newobj(space,new_ld)
+ new_ld = _intersection_dict(space, ld, rd)
+ return w_left._newobj(space, new_ld)
set_intersection__Set_Frozenset = set_intersection__Set_Set
set_intersection__Frozenset_Frozenset = set_intersection__Set_Set
set_intersection__Frozenset_Set = set_intersection__Set_Set
def set_intersection__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _intersection_dict(ld, rd, False)
- return w_left._newobj(space,new_ld)
+ result = newset(space)
+ ld = w_left.setdata
+ for w_key in space.listview(w_other):
+ if w_key in ld:
+ result[w_key] = None
+ return w_left._newobj(space, result)
frozenset_intersection__Frozenset_ANY = set_intersection__Set_ANY
@@ -518,15 +491,18 @@
def set_intersection_update__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _intersection_dict(ld, rd, True)
- return space.w_None
+ new_ld = _intersection_dict(space, ld, rd)
+ w_left.setdata = new_ld
set_intersection_update__Set_Frozenset = set_intersection_update__Set_Set
def set_intersection_update__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _intersection_dict(ld, rd, True)
- return space.w_None
+ result = newset(space)
+ ld = w_left.setdata
+ for w_key in space.listview(w_other):
+ if w_key in ld:
+ result[w_key] = None
+ w_left.setdata = result
def inplace_and__Set_Set(space, w_left, w_other):
set_intersection_update__Set_Set(space, w_left, w_other)
@@ -537,7 +513,7 @@
def set_symmetric_difference__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _symmetric_difference_dict(ld, rd, False)
+ new_ld = _symmetric_difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
set_symmetric_difference__Set_Frozenset = set_symmetric_difference__Set_Set
@@ -553,7 +529,7 @@
def set_symmetric_difference__Set_ANY(space, w_left, w_other):
ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _symmetric_difference_dict(ld, rd, False)
+ new_ld = _symmetric_difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
frozenset_symmetric_difference__Frozenset_ANY = \
@@ -562,16 +538,16 @@
def set_symmetric_difference_update__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _symmetric_difference_dict(ld, rd, True)
- return space.w_None
+ new_ld = _symmetric_difference_dict(space, ld, rd)
+ w_left.setdata = new_ld
set_symmetric_difference_update__Set_Frozenset = \
set_symmetric_difference_update__Set_Set
def set_symmetric_difference_update__Set_ANY(space, w_left, w_other):
ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _symmetric_difference_dict(ld, rd, True)
- return space.w_None
+ new_ld = _symmetric_difference_dict(space, ld, rd)
+ w_left.setdata = new_ld
def inplace_xor__Set_Set(space, w_left, w_other):
set_symmetric_difference_update__Set_Set(space, w_left, w_other)
@@ -582,8 +558,9 @@
def set_union__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _union_dict(ld, rd, False)
- return w_left._newobj(space, new_ld)
+ result = ld.copy()
+ result.update(rd)
+ return w_left._newobj(space, result)
set_union__Set_Frozenset = set_union__Set_Set
set_union__Frozenset_Set = set_union__Set_Set
@@ -595,9 +572,11 @@
def set_union__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _union_dict(ld, rd, False)
- return w_left._newobj(space, new_ld)
+ ld = w_left.setdata
+ result = ld.copy()
+ for w_key in space.listview(w_other):
+ result[w_key] = None
+ return w_left._newobj(space, result)
frozenset_union__Frozenset_ANY = set_union__Set_ANY
@@ -629,15 +608,6 @@
init_defaults)
_initialize_set(space, w_set, w_iterable)
-def init__Frozenset(space, w_set, __args__):
- w_iterable, = __args__.parse_obj(
- None, 'set',
- init_signature,
- init_defaults)
- if w_set.hash == -1:
- _initialize_set(space, w_set, w_iterable)
- hash__Frozenset(space, w_set)
-
app = gateway.applevel("""
def setrepr(currently_in_repr, s):
'The app-level part of repr().'
Modified: pypy/branch/fast-forward/pypy/objspace/std/settype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/settype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/settype.py Tue Oct 26 20:46:59 2010
@@ -66,9 +66,9 @@
register_all(vars(), globals())
def descr__new__(space, w_settype, __args__):
- from pypy.objspace.std.setobject import W_SetObject
+ from pypy.objspace.std.setobject import W_SetObject, newset
w_obj = space.allocate_instance(W_SetObject, w_settype)
- W_SetObject.__init__(w_obj, space, None)
+ W_SetObject.__init__(w_obj, space, newset(space))
return w_obj
set_typedef = StdTypeDef("set",
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py Tue Oct 26 20:46:59 2010
@@ -238,6 +238,7 @@
it1 = d.popitem()
assert len(d) == 0
assert (it!=it1) and (it1==(1,2) or it1==(3,4))
+ raises(KeyError, d.popitem)
def test_setdefault(self):
d = {1:2, 3:4}
@@ -446,7 +447,9 @@
d1 = {}
d2 = D(a='foo')
d1.update(d2)
- assert d1['a'] == 42 # fails on CPython, d1['a'] == 'foo'
+ assert d1['a'] == 'foo'
+ # a bit of an obscure case: now (from r78295) we get the same result
+ # as CPython does
def test_index_keyerror_unpacking(self):
d = {}
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_mapdict.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_mapdict.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_mapdict.py Tue Oct 26 20:46:59 2010
@@ -753,6 +753,25 @@
return a.x
#
res = self.check(f, 'x')
+ assert res == (1, 0, 0)
+ res = self.check(f, 'x')
+ assert res == (0, 1, 0)
+ res = self.check(f, 'x')
+ assert res == (0, 1, 0)
+
+ def test_old_style_base(self):
+ class B:
+ pass
+ class C(object):
+ pass
+ class A(C, B):
+ pass
+ a = A()
+ a.x = 42
+ def f():
+ return a.x
+ #
+ res = self.check(f, 'x')
assert res == (0, 0, 1)
res = self.check(f, 'x')
assert res == (0, 0, 1)
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py Tue Oct 26 20:46:59 2010
@@ -10,6 +10,7 @@
import py.test
from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject
from pypy.objspace.std.setobject import _initialize_set
+from pypy.objspace.std.setobject import newset, make_setdata_from_w_iterable
from pypy.objspace.std.setobject import set_intersection__Set_Set
from pypy.objspace.std.setobject import set_intersection__Set_ANY
from pypy.objspace.std.setobject import eq__Set_Set
@@ -28,12 +29,12 @@
self.false = self.space.w_False
def test_and(self):
- s = W_SetObject(self.space, None)
+ s = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, s, self.word)
- t0 = W_SetObject(self.space, None)
+ t0 = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, t0, self.otherword)
- t1 = W_FrozensetObject(self.space, None)
- _initialize_set(self.space, t1, self.otherword)
+ t1 = W_FrozensetObject(self.space,
+ make_setdata_from_w_iterable(self.space, self.otherword))
r0 = set_intersection__Set_Set(self.space, s, t0)
r1 = set_intersection__Set_Set(self.space, s, t1)
assert eq__Set_Set(self.space, r0, r1) == self.true
@@ -41,9 +42,9 @@
assert eq__Set_Set(self.space, r0, sr) == self.true
def test_compare(self):
- s = W_SetObject(self.space, None)
+ s = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, s, self.word)
- t = W_SetObject(self.space, None)
+ t = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, t, self.word)
assert self.space.eq_w(s,t)
u = self.space.wrap(set('simsalabim'))
@@ -56,6 +57,11 @@
b = a | set('abc')
assert type(b) is subset
+ def test_union(self):
+ a = set([4, 5])
+ b = a.union([5, 7])
+ assert sorted(b) == [4, 5, 7]
+
def test_compare(self):
raises(TypeError, cmp, set('abc'), set('abd'))
assert set('abc') != 'abc'
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_shadowtracking.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_shadowtracking.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_shadowtracking.py Tue Oct 26 20:46:59 2010
@@ -140,10 +140,11 @@
def test_class_that_cannot_be_cached(self):
import __pypy__
- class metatype(type):
+ class X:
pass
- class A(object):
- __metaclass__ = metatype
+ class Y(object):
+ pass
+ class A(Y, X):
def f(self):
return 42
@@ -241,3 +242,21 @@
foo = 3
D.__bases__ = (C, F)
assert e.foo == 3
+
+ def test_custom_metaclass(self):
+ import __pypy__
+ class MetaA(type):
+ def __getattribute__(self, x):
+ return 1
+ def f(self):
+ return 42
+ A = type.__new__(MetaA, "A", (), {"f": f})
+ l = [type.__getattribute__(A, "__new__")(A)] * 10
+ __pypy__.reset_method_cache_counter()
+ for i, a in enumerate(l):
+ assert a.f() == 42
+ cache_counter = __pypy__.method_cache_counter("f")
+ assert cache_counter[0] >= 5
+ assert cache_counter[1] >= 1 # should be (27, 3)
+ assert sum(cache_counter) == 10
+
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py Tue Oct 26 20:46:59 2010
@@ -1109,4 +1109,12 @@
assert b == 1
-
+ def test_slots_with_method_in_class(self):
+ # this works in cpython...
+ class A(object):
+ __slots__ = ["f"]
+ def f(self, x):
+ return x + 1
+ a = A()
+ assert a.f(1) == 2
+
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_versionedtype.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_versionedtype.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_versionedtype.py Tue Oct 26 20:46:59 2010
@@ -12,10 +12,12 @@
def f(self): pass
class B(A):
pass
- class metatype(type):
+ class X:
+ pass
+ class Y(object):
+ pass
+ class C(Y, X):
pass
- class C(object):
- __metaclass__ = metatype
return A, B, C
""")
return space.unpackiterable(w_types)
Modified: pypy/branch/fast-forward/pypy/objspace/std/typeobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/typeobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/typeobject.py Tue Oct 26 20:46:59 2010
@@ -106,14 +106,12 @@
if overridetypedef is not None:
setup_builtin_type(w_self)
- custom_metaclass = False
else:
setup_user_defined_type(w_self)
- custom_metaclass = not space.is_w(space.type(w_self), space.w_type)
w_self.w_same_layout_as = get_parent_layout(w_self)
if space.config.objspace.std.withtypeversion:
- if custom_metaclass or not is_mro_purely_of_types(w_self.mro_w):
+ if not is_mro_purely_of_types(w_self.mro_w):
pass
else:
# the _version_tag should change, whenever the content of
@@ -581,11 +579,13 @@
space.wrap('__slots__ must be identifiers'))
# create member
slot_name = _mangle(slot_name, w_self.name)
- # Force interning of slot names.
- slot_name = space.str_w(space.new_interned_str(slot_name))
- member = Member(w_self.nslots, slot_name, w_self)
- w_self.dict_w[slot_name] = space.wrap(member)
- w_self.nslots += 1
+ if slot_name not in w_self.dict_w:
+ # Force interning of slot names.
+ slot_name = space.str_w(space.new_interned_str(slot_name))
+ # in cpython it is ignored less, but we probably don't care
+ member = Member(w_self.nslots, slot_name, w_self)
+ w_self.dict_w[slot_name] = space.wrap(member)
+ w_self.nslots += 1
def create_dict_slot(w_self):
if not w_self.hasdict:
Modified: pypy/branch/fast-forward/pypy/objspace/test/test_descroperation.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/test/test_descroperation.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/test/test_descroperation.py Tue Oct 26 20:46:59 2010
@@ -416,7 +416,7 @@
class D(object):
def __init__(self, a):
self.a = a
-
+
assert A(1) == B(1)
assert B(1) == A(1)
assert A(1) == C(1)
@@ -479,6 +479,20 @@
return self
raises(TypeError, iter, x())
+ def test_attribute_error(self):
+ class classmethodonly(classmethod):
+ def __get__(self, instance, type):
+ if instance is not None:
+ raise AttributeError("Must be called on a class, not an instance.")
+ return super(classmethodonly, self).__get__(instance, type)
+
+ class A(object):
+ @classmethodonly
+ def a(cls):
+ return 3
+
+ raises(AttributeError, lambda: A().a)
+
def test_isinstance_and_issubclass(self):
class Meta(type):
def __instancecheck__(cls, instance):
From afa at codespeak.net Tue Oct 26 21:36:36 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 26 Oct 2010 21:36:36 +0200 (CEST)
Subject: [pypy-svn] r78317 -
pypy/branch/fast-forward/lib-python/modified-2.7.0/test
Message-ID: <20101026193636.96711282C18@codespeak.net>
Author: afa
Date: Tue Oct 26 21:36:34 2010
New Revision: 78317
Added:
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_compile.py
- copied, changed from r78315, pypy/branch/fast-forward/lib-python/2.7.0/test/test_compile.py
Log:
Skip an implementation specific test
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_compile.py (from r78315, pypy/branch/fast-forward/lib-python/2.7.0/test/test_compile.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/test/test_compile.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_compile.py Tue Oct 26 21:36:34 2010
@@ -3,6 +3,7 @@
import _ast
from test import test_support
import textwrap
+from test.test_support import check_impl_detail
class TestSpecifics(unittest.TestCase):
@@ -90,12 +91,13 @@
self.assertEqual(m.results, ('z', g))
exec 'z = locals()' in g, m
self.assertEqual(m.results, ('z', m))
- try:
- exec 'z = b' in m
- except TypeError:
- pass
- else:
- self.fail('Did not validate globals as a real dict')
+ if check_impl_detail():
+ try:
+ exec 'z = b' in m
+ except TypeError:
+ pass
+ else:
+ self.fail('Did not validate globals as a real dict')
class A:
"Non-mapping"
From hakanardo at codespeak.net Tue Oct 26 21:58:22 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Tue, 26 Oct 2010 21:58:22 +0200 (CEST)
Subject: [pypy-svn] r78319 -
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test
Message-ID: <20101026195822.BA144282C18@codespeak.net>
Author: hakanardo
Date: Tue Oct 26 21:58:21 2010
New Revision: 78319
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_jump.py
Log:
failing test
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_jump.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_jump.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_jump.py Tue Oct 26 21:58:21 2010
@@ -142,3 +142,22 @@
('push', s12),
('mov', ebx, s12),
('pop', ebx)]
+def test_crash():
+ assembler = MockAssembler()
+ dstvalues = [-56, -64, -72, -80, -88, 15, 14, 15, 13, 12, 15, -96,
+ -104, -112, 10, 9, 8, 7, 6, 3, -120, 2, 1]
+ srcvalues = [-56, -64, -72, -80, -88, -224, -216, -224, -248, -232,
+ -224, -96, -104, -112, -184, -192, -264, 2, -312, -344,
+ 10, 8, 14]
+ print len(dstvalues), len(srcvalues)
+ class MyLocation(AssemblerLocation):
+ def __init__(self, value):
+ self.value = value
+ def location_code(self):
+ return 'b'
+
+ src = [MyLocation(i) for i in srcvalues]
+ dst = [MyLocation(i) for i in dstvalues]
+
+ remap_frame_layout(assembler, src, dst, '?')
+ assert assembler.ops == []
From afa at codespeak.net Tue Oct 26 22:09:49 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 26 Oct 2010 22:09:49 +0200 (CEST)
Subject: [pypy-svn] r78320 - in
pypy/branch/fast-forward/pypy/interpreter/pyparser: . test
Message-ID: <20101026200949.C694E282C18@codespeak.net>
Author: afa
Date: Tue Oct 26 22:09:48 2010
New Revision: 78320
Modified:
pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py
pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py
pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py
Log:
Allow \r as a newline;
also tokens that go to the end of the line
should stop on either \r or \n.
This also fixes a cpython test in test_compile.py
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py Tue Oct 26 22:09:48 2010
@@ -17,12 +17,17 @@
def makePyPseudoDFA ():
import string
states = []
+ def makeEOL():
+ return group(states,
+ newArcPair(states, "\n"),
+ chain(states,
+ newArcPair(states, "\r"),
+ maybe(states, newArcPair(states, "\n"))))
# ____________________________________________________________
def makeLineCont ():
return chain(states,
newArcPair(states, "\\"),
- maybe(states, newArcPair(states, "\r")),
- newArcPair(states, "\n"))
+ makeEOL())
# ____________________________________________________________
# Ignore stuff
def makeWhitespace ():
@@ -130,9 +135,7 @@
newArcPair(states, "~"))
bracket = groupStr(states, "[](){}")
special = group(states,
- chain(states,
- maybe(states, newArcPair(states, "\r")),
- newArcPair(states, "\n")),
+ makeEOL(),
groupStr(states, "@:;.,`"))
funny = group(states, operator, bracket, special)
# ____________________________________________________________
@@ -146,13 +149,13 @@
makeStrPrefix(),
newArcPair(states, "'"),
any(states,
- notGroupStr(states, "\n'\\")),
+ notGroupStr(states, "\r\n'\\")),
any(states,
chain(states,
newArcPair(states, "\\"),
newArcPair(states, DEFAULT),
any(states,
- notGroupStr(states, "\n'\\")))),
+ notGroupStr(states, "\r\n'\\")))),
group(states,
newArcPair(states, "'"),
makeLineCont())),
@@ -160,13 +163,13 @@
makeStrPrefix(),
newArcPair(states, '"'),
any(states,
- notGroupStr(states, '\n"\\')),
+ notGroupStr(states, '\r\n"\\')),
any(states,
chain(states,
newArcPair(states, "\\"),
newArcPair(states, DEFAULT),
any(states,
- notGroupStr(states, '\n"\\')))),
+ notGroupStr(states, '\r\n"\\')))),
group(states,
newArcPair(states, '"'),
makeLineCont())))
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py Tue Oct 26 22:09:48 2010
@@ -24,7 +24,7 @@
# Automatically generated DFA's
accepts = [True, True, True, True, True, True, True, True,
- True, True, False, True, True, True, False, False,
+ True, True, False, True, True, True, True, False,
False, False, True, True, True, True, True, False,
True, False, True, False, True, False, False,
True, False, False, False, False, True, False,
@@ -143,9 +143,11 @@
# 14
{'\n': 13},
# 15
- {automata.DEFAULT: 30, '\n': 27, "'": 28, '\\': 29},
+ {automata.DEFAULT: 30, '\n': 27,
+ '\r': 27, "'": 28, '\\': 29},
# 16
- {automata.DEFAULT: 33, '\n': 27, '"': 31, '\\': 32},
+ {automata.DEFAULT: 33, '\n': 27,
+ '\r': 27, '"': 31, '\\': 32},
# 17
{'\n': 13, '\r': 14},
# 18
@@ -195,13 +197,15 @@
# 29
{automata.DEFAULT: 37, '\n': 13, '\r': 14},
# 30
- {automata.DEFAULT: 30, '\n': 27, "'": 13, '\\': 29},
+ {automata.DEFAULT: 30, '\n': 27,
+ '\r': 27, "'": 13, '\\': 29},
# 31
{'"': 13},
# 32
{automata.DEFAULT: 38, '\n': 13, '\r': 14},
# 33
- {automata.DEFAULT: 33, '\n': 27, '"': 13, '\\': 32},
+ {automata.DEFAULT: 33, '\n': 27,
+ '\r': 27, '"': 13, '\\': 32},
# 34
{'+': 39, '-': 39, '0': 40, '1': 40,
'2': 40, '3': 40, '4': 40, '5': 40,
@@ -215,9 +219,11 @@
'4': 36, '5': 36, '6': 36, '7': 36,
'8': 36, '9': 36, 'J': 13, 'j': 13},
# 37
- {automata.DEFAULT: 37, '\n': 27, "'": 13, '\\': 29},
+ {automata.DEFAULT: 37, '\n': 27,
+ '\r': 27, "'": 13, '\\': 29},
# 38
- {automata.DEFAULT: 38, '\n': 27, '"': 13, '\\': 32},
+ {automata.DEFAULT: 38, '\n': 27,
+ '\r': 27, '"': 13, '\\': 32},
# 39
{'0': 40, '1': 40, '2': 40, '3': 40,
'4': 40, '5': 40, '6': 40, '7': 40,
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py Tue Oct 26 22:09:48 2010
@@ -96,6 +96,9 @@
exc = py.test.raises(IndentationError, parse, input).value
assert exc.msg == "unindent does not match any outer indentation level"
+ def test_mac_newline(self):
+ self.parse("this_is\ra_mac\rfile")
+
def test_mode(self):
assert self.parse("x = 43*54").type == syms.file_input
tree = self.parse("43**54", "eval")
From afa at codespeak.net Tue Oct 26 22:42:32 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 26 Oct 2010 22:42:32 +0200 (CEST)
Subject: [pypy-svn] r78321 - in
pypy/branch/fast-forward/pypy/interpreter/astcompiler: . test
Message-ID: <20101026204232.A98125080C@codespeak.net>
Author: afa
Date: Tue Oct 26 22:42:30 2010
New Revision: 78321
Modified:
pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py
pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py
Log:
"import None as x" is not a SyntaxError
(None is not an assignment target here)
Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py Tue Oct 26 22:42:30 2010
@@ -196,7 +196,8 @@
if len(import_name.children) == 1:
import_name = import_name.children[0]
continue
- alias = self.alias_for_import_name(import_name.children[0])
+ alias = self.alias_for_import_name(import_name.children[0],
+ store=False)
asname_node = import_name.children[2]
alias.asname = asname_node.value
self.check_forbidden_name(alias.asname, asname_node)
Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py Tue Oct 26 22:42:30 2010
@@ -795,6 +795,10 @@
input = template % (name,)
exc = py.test.raises(SyntaxError, self.get_ast, input).value
assert exc.msg == "cannot assign to %s" % (name,)
+ # This is ok.
+ self.get_ast("from None import x")
+ self.get_ast("from x import None as y")
+ self.get_ast("import None as x")
def test_lambda(self):
lam = self.get_first_expr("lambda x: expr")
From afa at codespeak.net Tue Oct 26 23:03:07 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 26 Oct 2010 23:03:07 +0200 (CEST)
Subject: [pypy-svn] r78323 - in pypy/branch/fast-forward/pypy: interpreter
module/__builtin__/test
Message-ID: <20101026210307.07552282C18@codespeak.net>
Author: afa
Date: Tue Oct 26 23:03:05 2010
New Revision: 78323
Modified:
pypy/branch/fast-forward/pypy/interpreter/pycompiler.py
pypy/branch/fast-forward/pypy/module/__builtin__/test/test_builtin.py
Log:
Check node type when compiling an AST tree
Modified: pypy/branch/fast-forward/pypy/interpreter/pycompiler.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pycompiler.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pycompiler.py Tue Oct 26 23:03:05 2010
@@ -6,7 +6,7 @@
from pypy.interpreter import pycode
from pypy.interpreter.pyparser import future, pyparse, error as parseerror
from pypy.interpreter.astcompiler import (astbuilder, codegen, consts, misc,
- optimize)
+ optimize, ast)
from pypy.interpreter.error import OperationError
@@ -107,6 +107,18 @@
self.compiler_flags = self.future_flags.allowed_flags
def compile_ast(self, node, filename, mode, flags):
+ if mode == 'eval':
+ check = isinstance(node, ast.Expression)
+ elif mode == 'exec':
+ check = isinstance(node, ast.Module)
+ elif mode == 'input':
+ check = isinstance(node, ast.Interactive)
+ else:
+ check = True
+ if not check:
+ raise OperationError(self.space.w_TypeError, self.space.wrap(
+ "invalid node type"))
+
future_pos = misc.parse_future(node)
info = pyparse.CompileInfo(filename, mode, flags, future_pos)
return self._compile_ast(node, info)
Modified: pypy/branch/fast-forward/pypy/module/__builtin__/test/test_builtin.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/__builtin__/test/test_builtin.py (original)
+++ pypy/branch/fast-forward/pypy/module/__builtin__/test/test_builtin.py Tue Oct 26 23:03:05 2010
@@ -453,7 +453,15 @@
def test_unicode_encoding_compile(self):
code = u"# -*- coding: utf-8 -*-\npass\n"
raises(SyntaxError, compile, code, "tmp", "exec")
-
+
+ def test_recompile_ast(self):
+ import _ast
+ # raise exception when node type doesn't match with compile mode
+ co1 = compile('print 1', '', 'exec', _ast.PyCF_ONLY_AST)
+ raises(TypeError, compile, co1, '', 'eval')
+ co2 = compile('1+1', '', 'eval', _ast.PyCF_ONLY_AST)
+ compile(co2, '', 'eval')
+
def test_isinstance(self):
assert isinstance(5, int)
assert isinstance(5, object)
From afa at codespeak.net Tue Oct 26 23:43:58 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 26 Oct 2010 23:43:58 +0200 (CEST)
Subject: [pypy-svn] r78326 - pypy/branch/fast-forward/lib_pypy
Message-ID: <20101026214358.464BF5080C@codespeak.net>
Author: afa
Date: Tue Oct 26 23:43:56 2010
New Revision: 78326
Modified:
pypy/branch/fast-forward/lib_pypy/datetime.py
Log:
Fix tests in test_datetime.py
Modified: pypy/branch/fast-forward/lib_pypy/datetime.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/datetime.py (original)
+++ pypy/branch/fast-forward/lib_pypy/datetime.py Tue Oct 26 23:43:56 2010
@@ -217,6 +217,11 @@
# strftime is going to have at this: escape %
Zreplace = s.replace('%', '%%')
newformat.append(Zreplace)
+ elif ch == 'f':
+ if isinstance(object, (time, datetime)):
+ newformat.append('%06d' % object.microsecond)
+ else:
+ newformat.append('000000')
else:
push('%')
push(ch)
@@ -575,6 +580,10 @@
microseconds = property(lambda self: self.__microseconds,
doc="microseconds")
+ def total_seconds(self):
+ return ((self.days * 86400 + self.seconds) * 10**6
+ + self.microseconds) / 1e6
+
def __add__(self, other):
if isinstance(other, timedelta):
# for CPython compatibility, we cannot use
@@ -803,6 +812,14 @@
__str__ = isoformat
+ def __format__(self, format):
+ if not isinstance(format, (str, unicode)):
+ raise ValueError("__format__ excepts str or unicode, not %s" %
+ format.__class__.__name__)
+ if not format:
+ return str(self)
+ return self.strftime(format)
+
# Read-only field accessors
year = property(lambda self: self.__year,
doc="year (%d-%d)" % (MINYEAR, MAXYEAR))
@@ -1238,6 +1255,14 @@
__str__ = isoformat
+ def __format__(self, format):
+ if not isinstance(format, (str, unicode)):
+ raise ValueError("__format__ excepts str or unicode, not %s" %
+ format.__class__.__name__)
+ if not format:
+ return str(self)
+ return self.strftime(format)
+
def strftime(self, fmt):
"""Format using strftime(). The date part of the timestamp passed
to underlying strftime should not be used.
@@ -1587,7 +1612,11 @@
@classmethod
def strptime(cls, date_string, format):
'string, format -> new datetime parsed from a string (like time.strptime()).'
- return cls(*_time.strptime(date_string, format)[0:6])
+ # _strptime._strptime returns a two-element tuple. The first
+ # element is a time.struct_time object. The second is the
+ # microseconds (which are not defined for time.struct_time).
+ struct, micros = _time.strptime(date_string, format)
+ return cls(*(struct[0:6] + (micros,)))
def utcoffset(self):
"""Return the timezone offset in minutes east of UTC (negative west of
From afa at codespeak.net Tue Oct 26 23:48:13 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Tue, 26 Oct 2010 23:48:13 +0200 (CEST)
Subject: [pypy-svn] r78327 -
pypy/branch/fast-forward/pypy/interpreter/pyparser
Message-ID: <20101026214813.0ED79282C18@codespeak.net>
Author: afa
Date: Tue Oct 26 23:48:12 2010
New Revision: 78327
Modified:
pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenizer.py
Log:
Use CPython error messages
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenizer.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenizer.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenizer.py Tue Oct 26 23:48:12 2010
@@ -92,8 +92,9 @@
if contstr:
if not line:
- raise TokenError("EOF while scanning triple-quoted string",
- line, lnum-1, 0, token_list)
+ raise TokenError(
+ "EOF while scanning triple-quoted string literal",
+ line, lnum-1, 0, token_list)
endmatch = endDFA.recognize(line)
if endmatch >= 0:
pos = end = endmatch
@@ -229,7 +230,7 @@
if start < 0:
start = pos
if start
Author: afa
Date: Wed Oct 27 00:58:52 2010
New Revision: 78330
Modified:
pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py
pypy/branch/fast-forward/pypy/module/zipimport/test/test_zipimport.py
Log:
Implement zipimporter._get_filename()
Modified: pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py (original)
+++ pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py Wed Oct 27 00:58:52 2010
@@ -303,6 +303,16 @@
"Cannot find source for %s in %s", filename, self.name)
get_source.unwrap_spec = ['self', ObjSpace, str]
+ def get_filename(self, space, fullname):
+ filename = self.mangle(fullname)
+ for _, is_package, ext in ENUMERATE_EXTS:
+ if self.have_modulefile(space, filename + ext):
+ return space.wrap(self.filename + os.path.sep +
+ self.corr_zname(filename + ext))
+ raise operationerrfmt(self.w_ZipImportError,
+ "Cannot find module %s in %s", filename, self.name)
+ get_filename.unwrap_spec = ['self', ObjSpace, str]
+
def is_package(self, space, fullname):
filename = self.mangle(fullname)
for _, is_package, ext in ENUMERATE_EXTS:
@@ -373,6 +383,7 @@
get_data = interp2app(W_ZipImporter.get_data),
get_code = interp2app(W_ZipImporter.get_code),
get_source = interp2app(W_ZipImporter.get_source),
+ _get_filename = interp2app(W_ZipImporter.get_filename),
is_package = interp2app(W_ZipImporter.is_package),
load_module = interp2app(W_ZipImporter.load_module),
archive = GetSetProperty(W_ZipImporter.getarchive),
Modified: pypy/branch/fast-forward/pypy/module/zipimport/test/test_zipimport.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/zipimport/test/test_zipimport.py (original)
+++ pypy/branch/fast-forward/pypy/module/zipimport/test/test_zipimport.py Wed Oct 27 00:58:52 2010
@@ -269,6 +269,8 @@
assert z.get_code('xx')
assert z.get_source('xx') == "5"
assert z.archive == self.zipfile
+ mod = z.load_module('xx')
+ assert z._get_filename('xx') == mod.__file__
def test_archive(self):
"""
From hakanardo at codespeak.net Wed Oct 27 08:17:49 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Wed, 27 Oct 2010 08:17:49 +0200 (CEST)
Subject: [pypy-svn] r78336 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test
Message-ID: <20101027061749.01CA5282C18@codespeak.net>
Author: hakanardo
Date: Wed Oct 27 08:17:48 2010
New Revision: 78336
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py
Log:
Dont duplicate inputargs
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Wed Oct 27 08:17:48 2010
@@ -51,8 +51,9 @@
self.snapshot_map ={None: None}
inputargs = []
+ seen = []
for arg in jump_args:
- for a in self.getvalue(arg).get_forced_boxes([]):
+ for a in self.getvalue(arg).get_forced_boxes(seen):
if not isinstance(a, Const):
inputargs.append(a)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py Wed Oct 27 08:17:48 2010
@@ -34,6 +34,28 @@
self.check_loops(new=0, new_with_vtable=0,
getfield_gc=0, setfield_gc=0)
+ def test_virtualized2(self):
+ myjitdriver = JitDriver(greens = [], reds = ['n', 'node1', 'node2'])
+ def f(n):
+ node1 = self._new()
+ node1.value = 0
+ node2 = self._new()
+ node2.value = 0
+ while n > 0:
+ myjitdriver.can_enter_jit(n=n, node1=node1, node2=node2)
+ myjitdriver.jit_merge_point(n=n, node1=node1, node2=node2)
+ next1 = self._new()
+ next1.value = node1.value + n + node2.value
+ next2 = self._new()
+ next2.value = next1.value
+ node1 = next1
+ node2 = next2
+ n -= 1
+ return node1.value * node2.value
+ assert f(10) == self.meta_interp(f, [10])
+ self.check_loops(new=0, new_with_vtable=0,
+ getfield_gc=0, setfield_gc=0)
+
def test_virtualized_circular1(self):
class MyNode():
pass
From afa at codespeak.net Wed Oct 27 08:35:28 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 27 Oct 2010 08:35:28 +0200 (CEST)
Subject: [pypy-svn] r78337 - in
pypy/branch/fast-forward/pypy/module/zipimport: . test
Message-ID: <20101027063528.77E85282C18@codespeak.net>
Author: afa
Date: Wed Oct 27 08:35:26 2010
New Revision: 78337
Modified:
pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py
pypy/branch/fast-forward/pypy/module/zipimport/test/test_zipimport.py
Log:
Fix zipimporter in the case of sub-subdirectories
Modified: pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py (original)
+++ pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py Wed Oct 27 08:35:26 2010
@@ -204,7 +204,6 @@
def have_modulefile(self, space, filename):
if ZIPSEP != os.path.sep:
filename = filename.replace(os.path.sep, ZIPSEP)
- w = space.wrap
try:
self.dir[filename]
return True
@@ -212,18 +211,20 @@
return False
def find_module(self, space, fullname, w_path=None):
- filename = self.mangle(fullname)
+ filename = self.make_filename(fullname)
for _, _, ext in ENUMERATE_EXTS:
if self.have_modulefile(space, filename + ext):
return space.wrap(self)
find_module.unwrap_spec = ['self', ObjSpace, str, W_Root]
- def mangle(self, name):
- return name.replace('.', ZIPSEP)
+ def make_filename(self, fullname):
+ i = fullname.rfind('.')
+ subname = fullname[i+1:]
+ return self.prefix + subname
def load_module(self, space, fullname):
w = space.wrap
- filename = self.mangle(fullname)
+ filename = self.make_filename(fullname)
last_exc = None
for compiled, is_package, ext in ENUMERATE_EXTS:
fname = filename + ext
@@ -237,7 +238,8 @@
pass
else:
if is_package:
- pkgpath = self.name + os.path.sep + filename
+ pkgpath = (self.name + os.path.sep +
+ self.corr_zname(filename))
else:
pkgpath = None
try:
@@ -272,7 +274,7 @@
get_data.unwrap_spec = ['self', ObjSpace, str]
def get_code(self, space, fullname):
- filename = self.mangle(fullname)
+ filename = self.make_filename(fullname)
w = space.wrap
for compiled, _, ext in ENUMERATE_EXTS:
if self.have_modulefile(space, filename + ext):
@@ -288,7 +290,7 @@
get_code.unwrap_spec = ['self', ObjSpace, str]
def get_source(self, space, fullname):
- filename = self.mangle(fullname)
+ filename = self.make_filename(fullname)
found = False
for compiled, _, ext in ENUMERATE_EXTS:
fname = filename + ext
@@ -304,7 +306,7 @@
get_source.unwrap_spec = ['self', ObjSpace, str]
def get_filename(self, space, fullname):
- filename = self.mangle(fullname)
+ filename = self.make_filename(fullname)
for _, is_package, ext in ENUMERATE_EXTS:
if self.have_modulefile(space, filename + ext):
return space.wrap(self.filename + os.path.sep +
@@ -314,7 +316,7 @@
get_filename.unwrap_spec = ['self', ObjSpace, str]
def is_package(self, space, fullname):
- filename = self.mangle(fullname)
+ filename = self.make_filename(fullname)
for _, is_package, ext in ENUMERATE_EXTS:
if self.have_modulefile(space, filename + ext):
return space.wrap(is_package)
Modified: pypy/branch/fast-forward/pypy/module/zipimport/test/test_zipimport.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/zipimport/test/test_zipimport.py (original)
+++ pypy/branch/fast-forward/pypy/module/zipimport/test/test_zipimport.py Wed Oct 27 08:35:26 2010
@@ -292,6 +292,16 @@
assert archive == self.zipfile
assert realprefix == prefix
+ def test_subdirectory_importer(self):
+ import os
+ import zipimport
+ self.writefile(
+ self, os.sep.join(("directory", "package", "__init__.py")), "")
+ z = zipimport.zipimporter(self.zipfile + "/directory")
+ mod = z.load_module("package")
+ assert z.is_package("package")
+ assert z._get_filename("package") == mod.__file__
+
def test_zip_directory_cache(self):
""" Check full dictionary interface
"""
From afa at codespeak.net Wed Oct 27 09:14:28 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 27 Oct 2010 09:14:28 +0200 (CEST)
Subject: [pypy-svn] r78338 - pypy/branch/fast-forward/pypy/module/zipimport
Message-ID: <20101027071428.C97D0282C18@codespeak.net>
Author: afa
Date: Wed Oct 27 09:14:27 2010
New Revision: 78338
Modified:
pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py
Log:
Fix translation
Modified: pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py (original)
+++ pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py Wed Oct 27 09:14:27 2010
@@ -218,8 +218,9 @@
find_module.unwrap_spec = ['self', ObjSpace, str, W_Root]
def make_filename(self, fullname):
- i = fullname.rfind('.')
- subname = fullname[i+1:]
+ startpos = fullname.rfind('.') + 1 # 0 when not found
+ assert startpos >= 0
+ subname = fullname[start:]
return self.prefix + subname
def load_module(self, space, fullname):
From afa at codespeak.net Wed Oct 27 09:50:42 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 27 Oct 2010 09:50:42 +0200 (CEST)
Subject: [pypy-svn] r78339 - pypy/branch/fast-forward/pypy/module/zipimport
Message-ID: <20101027075042.45509282C18@codespeak.net>
Author: afa
Date: Wed Oct 27 09:50:40 2010
New Revision: 78339
Modified:
pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py
Log:
Stupid typo
Modified: pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py (original)
+++ pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py Wed Oct 27 09:50:40 2010
@@ -220,7 +220,7 @@
def make_filename(self, fullname):
startpos = fullname.rfind('.') + 1 # 0 when not found
assert startpos >= 0
- subname = fullname[start:]
+ subname = fullname[startpos:]
return self.prefix + subname
def load_module(self, space, fullname):
From david at codespeak.net Wed Oct 27 10:36:38 2010
From: david at codespeak.net (david at codespeak.net)
Date: Wed, 27 Oct 2010 10:36:38 +0200 (CEST)
Subject: [pypy-svn] r78340 - pypy/extradoc/sprintinfo/ddorf2010
Message-ID: <20101027083638.BD39A282C1A@codespeak.net>
Author: david
Date: Wed Oct 27 10:36:34 2010
New Revision: 78340
Modified:
pypy/extradoc/sprintinfo/ddorf2010/planning.txt
Log:
(All) planning for today
Modified: pypy/extradoc/sprintinfo/ddorf2010/planning.txt
==============================================================================
--- pypy/extradoc/sprintinfo/ddorf2010/planning.txt (original)
+++ pypy/extradoc/sprintinfo/ddorf2010/planning.txt Wed Oct 27 10:36:34 2010
@@ -11,19 +11,22 @@
- get mmap to work on Beagleboard DONE
- bridges in the ARM backend SORT OF DONE
-- add spilling and location support (David, Armin around)
-- hg conversion WE HAVE A PLAN (Anto, Ronny)
-- memory benchmarks IN PROGRESS (Carl Friedrich, Lukas when he shows up)
+- add spilling and location support IN PROGRESS (David, Armin around)
+- ARM int trace operations (David)
+- hg conversion WE HAVE MORE OF A PLAN (Anto, Ronny)
+- memory benchmarks DONE
+- refine heap analysis CONFUSING (Carl Friedrich, Lukas)
+- (Re)write _bisect module (Armin)
- buildbot improvements, locks, email on failing tests
- run pypy-c-nojit in the benchmarks
- make it possible to run benchmark on branches
- understand problems of mapdict (Carl Friedrich, Armin)
- rewrite ctypes to use _ffi
-- can classes with non-type metaclasses use typeversion?
+- can classes with non-type metaclasses use typeversion? DONE
scheduled discussions:
- hg migration planning
- understand h?kan's branch
-- JIT introduction (today after lunch)
+- JIT introduction (still today after lunch)
From arigo at codespeak.net Wed Oct 27 11:44:30 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 11:44:30 +0200 (CEST)
Subject: [pypy-svn] r78343 -
pypy/branch/cleanup-dict-impl/pypy/objspace/std/test
Message-ID: <20101027094430.2F324282C1A@codespeak.net>
Author: arigo
Date: Wed Oct 27 11:44:26 2010
New Revision: 78343
Added:
pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_shadowtracking.py
- copied unchanged from r78139, pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_shadowtracking.py
Log:
(cfbolz, arigo)
Restore test_shadowtracking because it contains tests of the
method cache.
From arigo at codespeak.net Wed Oct 27 11:47:06 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 11:47:06 +0200 (CEST)
Subject: [pypy-svn] r78344 -
pypy/branch/cleanup-dict-impl/pypy/objspace/std/test
Message-ID: <20101027094706.D8E1C282C1A@codespeak.net>
Author: arigo
Date: Wed Oct 27 11:47:05 2010
New Revision: 78344
Added:
pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_methodcache.py
- copied, changed from r78343, pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_shadowtracking.py
Removed:
pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_shadowtracking.py
Log:
(cfbolz, arigo)
Rename test_shadowtracking to test_methodcache, and drop the tests
about shadow tracking.
Copied: pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_methodcache.py (from r78343, pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_shadowtracking.py)
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_shadowtracking.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_methodcache.py Wed Oct 27 11:47:05 2010
@@ -1,119 +1,8 @@
from pypy.conftest import gettestobjspace
from pypy.objspace.std.test.test_typeobject import AppTestTypeObject
-class TestShadowTracking(object):
- def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withshadowtracking": True,
- "objspace.std.withmapdict": False})
-
- def test_simple_shadowing(self):
- space = self.space
- w_inst = space.appexec([], """():
- class A(object):
- def f(self):
- return 42
- a = A()
- return a
- """)
- assert not w_inst.getdict().shadows_anything()
- space.appexec([w_inst], """(a):
- a.g = "foo"
- """)
- assert not w_inst.getdict().shadows_anything()
- space.appexec([w_inst], """(a):
- a.f = "foo"
- """)
- assert w_inst.getdict().shadows_anything()
-
- def test_shadowing_via__dict__(self):
- space = self.space
- w_inst = space.appexec([], """():
- class A(object):
- def f(self):
- return 42
- a = A()
- return a
- """)
- assert not w_inst.getdict().shadows_anything()
- space.appexec([w_inst], """(a):
- a.__dict__["g"] = "foo"
- """)
- assert not w_inst.getdict().shadows_anything()
- space.appexec([w_inst], """(a):
- a.__dict__["f"] = "foo"
- """)
- assert w_inst.getdict().shadows_anything()
-
- def test_changing__dict__(self):
- space = self.space
- w_inst = space.appexec([], """():
- class A(object):
- def f(self):
- return 42
- a = A()
- return a
- """)
- assert not w_inst.getdict().shadows_anything()
- space.appexec([w_inst], """(a):
- a.__dict__ = {}
- """)
- assert w_inst.getdict().shadows_anything()
-
- def test_changing__class__(self):
- space = self.space
- w_inst = space.appexec([], """():
- class A(object):
- def f(self):
- return 42
- a = A()
- return a
- """)
- assert not w_inst.getdict().shadows_anything()
- space.appexec([w_inst], """(a):
- class B(object):
- def g(self):
- return 42
- a.__class__ = B
- """)
- assert w_inst.getdict().shadows_anything()
-
- def test_changing_the_type(self):
- space = self.space
- w_inst = space.appexec([], """():
- class A(object):
- pass
- a = A()
- a.x = 72
- return a
- """)
- assert not w_inst.getdict().shadows_anything()
- w_x = space.appexec([w_inst], """(a):
- a.__class__.x = 42
- return a.x
- """)
- assert space.unwrap(w_x) == 72
- assert w_inst.getdict().shadows_anything()
-
-class AppTestShadowTracking(object):
- def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withshadowtracking": True})
-
- def test_shadowtracking_does_not_blow_up(self):
- class A(object):
- def f(self):
- return 42
- a = A()
- assert a.f() == 42
- a.f = lambda : 43
- assert a.f() == 43
-
-class AppTestTypeWithMethodCache(AppTestTypeObject):
-
- def setup_class(cls):
- cls.space = gettestobjspace(
- **{"objspace.std.withmethodcachecounter" : True})
-class AppTestMethodCaching(AppTestShadowTracking):
+class AppTestMethodCaching(AppTestTypeObject):
def setup_class(cls):
cls.space = gettestobjspace(
**{"objspace.std.withmethodcachecounter": True})
From afa at codespeak.net Wed Oct 27 11:54:10 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 27 Oct 2010 11:54:10 +0200 (CEST)
Subject: [pypy-svn] r78345 - pypy/branch/fast-forward/pypy/module/_ast/test
Message-ID: <20101027095410.E391B282C1A@codespeak.net>
Author: afa
Date: Wed Oct 27 11:54:09 2010
New Revision: 78345
Modified:
pypy/branch/fast-forward/pypy/module/_ast/test/test_ast.py
Log:
Fix test, which also failed with -A.
Modified: pypy/branch/fast-forward/pypy/module/_ast/test/test_ast.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_ast/test/test_ast.py (original)
+++ pypy/branch/fast-forward/pypy/module/_ast/test/test_ast.py Wed Oct 27 11:54:09 2010
@@ -43,7 +43,7 @@
for op in mod.body.ops:
assert isinstance(op, ast.Lt)
mod.body.ops[0] = ast.Gt()
- co = compile(mod, "", "exec")
+ co = compile(mod, "", "eval")
assert not eval(co)
def test_string(self):
From arigo at codespeak.net Wed Oct 27 12:58:43 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 12:58:43 +0200 (CEST)
Subject: [pypy-svn] r78346 - pypy/branch/cleanup-dict-impl/pypy/objspace/std
Message-ID: <20101027105843.006B2282C1C@codespeak.net>
Author: arigo
Date: Wed Oct 27 12:58:41 2010
New Revision: 78346
Modified:
pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py
Log:
(cfbolz, arigo) ObjectMixin is not used there actually.
Modified: pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py Wed Oct 27 12:58:41 2010
@@ -418,7 +418,7 @@
rangen = unroll.unrolling_iterable(range(n))
nmin1 = n - 1
rangenmin1 = unroll.unrolling_iterable(range(nmin1))
- class subcls(ObjectMixin, BaseMapdictObject, supercls):
+ class subcls(BaseMapdictObject, supercls):
def _init_empty(self, map):
from pypy.rlib.debug import make_sure_not_resized
for i in rangen:
From arigo at codespeak.net Wed Oct 27 13:04:13 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 13:04:13 +0200 (CEST)
Subject: [pypy-svn] r78347 - pypy/branch/cleanup-dict-impl/pypy/objspace/std
Message-ID: <20101027110413.1FDBC282C1C@codespeak.net>
Author: arigo
Date: Wed Oct 27 13:04:11 2010
New Revision: 78347
Modified:
pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py
Log:
(cfbolz, arigo)
Using the JIT, we actually don't really want to have many different
RPython-level classes, because promoting their class gives many
different paths in the assembler code. So for now we fix the number
at exactly 5.
Modified: pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py (original)
+++ pypy/branch/cleanup-dict-impl/pypy/objspace/std/mapdict.py Wed Oct 27 13:04:11 2010
@@ -388,6 +388,8 @@
assert space.config.objspace.std.withmapdict
map = w_type.terminator
classes = memo_get_subclass_of_correct_size(space, cls)
+ if SUBCLASSES_MIN_FIELDS == SUBCLASSES_MAX_FIELDS:
+ return classes[0]
size = map.size_estimate()
debug.check_nonneg(size)
if size < len(classes):
@@ -396,7 +398,8 @@
return classes[len(classes)-1]
get_subclass_of_correct_size._annspecialcase_ = "specialize:arg(1)"
-NUM_SUBCLASSES = 10 # XXX tweak this number
+SUBCLASSES_MIN_FIELDS = 5 # XXX tweak these numbers
+SUBCLASSES_MAX_FIELDS = 5
def memo_get_subclass_of_correct_size(space, supercls):
key = space, supercls
@@ -405,9 +408,12 @@
except KeyError:
assert not hasattr(supercls, "__del__")
result = []
- for i in range(1, NUM_SUBCLASSES+1):
+ for i in range(SUBCLASSES_MIN_FIELDS, SUBCLASSES_MAX_FIELDS+1):
result.append(_make_subclass_size_n(supercls, i))
- result.insert(0, result[0]) # for 0, use the same class as for 1
+ for i in range(SUBCLASSES_MIN_FIELDS):
+ result.insert(0, result[0])
+ if SUBCLASSES_MIN_FIELDS == SUBCLASSES_MAX_FIELDS:
+ assert len(set(result)) == 1
_subclass_cache[key] = result
return result
memo_get_subclass_of_correct_size._annspecialcase_ = "specialize:memo"
From david at codespeak.net Wed Oct 27 13:18:30 2010
From: david at codespeak.net (david at codespeak.net)
Date: Wed, 27 Oct 2010 13:18:30 +0200 (CEST)
Subject: [pypy-svn] r78348 - in
pypy/branch/arm-backend/pypy/jit/backend/arm: . test
Message-ID: <20101027111830.7E8F0282C1C@codespeak.net>
Author: david
Date: Wed Oct 27 13:18:28 2010
New Revision: 78348
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
Log:
Extend int_sub and int_add instruction encoding
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Wed Oct 27 13:18:28 2010
@@ -178,6 +178,7 @@
opnum = op.getopnum()
fcond = self.operations[opnum](self, op, regalloc, fcond)
self.gen_func_epilog()
+ print inputargs, operations
if self._debug_asm:
self._dump_trace('loop.asm')
print 'Done assembling'
@@ -249,14 +250,18 @@
def emit_op_int_add(self, op, regalloc, fcond):
# assuming only one argument is constant
res = regalloc.try_allocate_reg(op.result)
- if isinstance(op.getarg(1), ConstInt):
- reg = regalloc.try_allocate_reg(op.getarg(0))
- arg1 = op.getarg(1)
- self.mc.ADD_ri(res.value, reg.value, arg1.getint())
- elif isinstance(op.getarg(0), ConstInt):
- reg = regalloc.try_allocate_reg(op.getarg(1))
- arg1 = op.getarg(0)
- self.mc.ADD_ri(res.value, reg.value, arg1.getint())
+ if isinstance(op.getarg(0), ConstInt) or isinstance(op.getarg(1), ConstInt):
+ if isinstance(op.getarg(1), ConstInt):
+ reg = regalloc.try_allocate_reg(op.getarg(0))
+ arg1 = op.getarg(1)
+ elif isinstance(op.getarg(0), ConstInt):
+ reg = regalloc.try_allocate_reg(op.getarg(1))
+ arg1 = op.getarg(0)
+ value = arg1.getint()
+ if value < 0:
+ self.mc.SUB_ri(res.value, reg.value, -1 * value)
+ else:
+ self.mc.ADD_ri(res.value, reg.value, value)
else:
r1 = regalloc.try_allocate_reg(op.getarg(0))
r2 = regalloc.try_allocate_reg(op.getarg(1))
@@ -268,14 +273,23 @@
def emit_op_int_sub(self, op, regalloc, fcond):
# assuming only one argument is constant
res = regalloc.try_allocate_reg(op.result)
- if isinstance(op.getarg(1), ConstInt):
- reg = regalloc.try_allocate_reg(op.getarg(0))
- arg1 = op.getarg(1)
- self.mc.SUB_ri(res.value, reg.value, arg1.getint())
- elif isinstance(op.getarg(0), ConstInt):
- reg = regalloc.try_allocate_reg(op.getarg(1))
- arg1 = op.getarg(0)
- self.mc.SUB_ri(res.value, reg.value, arg1.getint())
+ if isinstance(op.getarg(0), ConstInt) or isinstance(op.getarg(1), ConstInt):
+ if isinstance(op.getarg(1), ConstInt):
+ reg = regalloc.try_allocate_reg(op.getarg(0))
+ value = op.getarg(1).getint()
+ if value < 0:
+ self.mc.ADD_ri(res.value, reg.value, -1 * value)
+ else:
+ self.mc.SUB_ri(res.value, reg.value, value)
+ elif isinstance(op.getarg(0), ConstInt):
+ reg = regalloc.try_allocate_reg(op.getarg(1))
+ value = op.getarg(0).getint()
+ if value < 0:
+ self.mc.ADD_ri(res.value, reg.value, -1 * value)
+ self.mc.MVN_rr(res.value, res.value)
+ else:
+ # reverse substract ftw
+ self.mc.RSB_ri(res.value, reg.value, value)
else:
r1 = regalloc.try_allocate_reg(op.getarg(0))
r2 = regalloc.try_allocate_reg(op.getarg(1))
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py Wed Oct 27 13:18:28 2010
@@ -35,6 +35,8 @@
| (table['op'] & 0x1F) << 20)
if table['result'] and table['base']:
def imm_func(self, rd, rn, imm=0, cond=cond.AL, s=0):
+ if imm < 0:
+ raise ValueError
# XXX check condition on rn
self.write32(n
| cond << 28
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Wed Oct 27 13:18:28 2010
@@ -16,10 +16,12 @@
def hexdump(self):
return ''.join(self.buffer)
+
class ASMTest(object):
def assert_equal(self, asm):
assert self.cb.hexdump() == assemble(asm)
+
class TestInstrCodeBuilder(ASMTest):
def setup_method(self, ffuu_method):
self.cb = CodeBuilder()
@@ -151,6 +153,7 @@
func = getattr(self.cb, name)
func(r.r3.value, r.r7.value, 23)
self.assert_equal('%s r3, r7, #23' % name[:name.index('_')])
+ py.test.raises(ValueError, 'func(r.r3.value, r.r7.value, -12)')
elif not table['base']:
def f(self):
func = getattr(self.cb, name)
From david at codespeak.net Wed Oct 27 14:59:27 2010
From: david at codespeak.net (david at codespeak.net)
Date: Wed, 27 Oct 2010 14:59:27 +0200 (CEST)
Subject: [pypy-svn] r78352 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101027125927.12C51282C1C@codespeak.net>
Author: david
Date: Wed Oct 27 14:59:25 2010
New Revision: 78352
Added:
pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
Log:
move resop implementations to mixins
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Wed Oct 27 14:59:25 2010
@@ -10,11 +10,14 @@
from pypy.rlib import rgc
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.lltypesystem import lltype, rffi, llmemory
+from pypy.jit.backend.arm.opassembler import (GuardOpAssembler,
+ IntOpAsslember,
+ OpAssembler)
# XXX Move to llsupport
from pypy.jit.backend.x86.support import values_array
-class AssemblerARM(object):
+class AssemblerARM(GuardOpAssembler, IntOpAsslember, OpAssembler):
def __init__(self, cpu, failargs_limit=1000):
self.mc = ARMv7Builder()
@@ -215,112 +218,9 @@
b.MOV_rr(r.pc.value, reg.value, cond=fcond)
- # Resoperations
- def emit_op_jump(self, op, regalloc, fcond):
- tmp = Box()
- tmpreg = regalloc.try_allocate_reg(tmp)
- registers = op.getdescr()._arm_arglocs
- for i in range(op.numargs()):
- reg = regalloc.try_allocate_reg(op.getarg(i))
- inpreg = registers[i]
- # XXX only if every value is in a register
- self.mc.MOV_rr(inpreg.value, reg.value)
- loop_code = op.getdescr()._arm_loop_code
- self.mc.gen_load_int(tmpreg.value, loop_code)
- self.mc.MOV_rr(r.pc.value, tmpreg.value)
- regalloc.possibly_free_var(tmpreg)
- return fcond
-
- def emit_op_finish(self, op, regalloc, fcond):
- self._gen_path_to_exit_path(op, op.getarglist(), regalloc, fcond)
- return fcond
-
- def emit_op_int_le(self, op, regalloc, fcond):
- reg = regalloc.try_allocate_reg(op.getarg(0))
- assert isinstance(op.getarg(1), ConstInt)
- self.mc.CMP(reg.value, op.getarg(1).getint())
- return c.GT
-
- def emit_op_int_eq(self, op, regalloc, fcond):
- reg = regalloc.try_allocate_reg(op.getarg(0))
- assert isinstance(op.getarg(1), ConstInt)
- self.mc.CMP(reg.value, op.getarg(1).getint())
- return c.EQ
-
- def emit_op_int_add(self, op, regalloc, fcond):
- # assuming only one argument is constant
- res = regalloc.try_allocate_reg(op.result)
- if isinstance(op.getarg(0), ConstInt) or isinstance(op.getarg(1), ConstInt):
- if isinstance(op.getarg(1), ConstInt):
- reg = regalloc.try_allocate_reg(op.getarg(0))
- arg1 = op.getarg(1)
- elif isinstance(op.getarg(0), ConstInt):
- reg = regalloc.try_allocate_reg(op.getarg(1))
- arg1 = op.getarg(0)
- value = arg1.getint()
- if value < 0:
- self.mc.SUB_ri(res.value, reg.value, -1 * value)
- else:
- self.mc.ADD_ri(res.value, reg.value, value)
- else:
- r1 = regalloc.try_allocate_reg(op.getarg(0))
- r2 = regalloc.try_allocate_reg(op.getarg(1))
- self.mc.ADD_rr(res.value, r1.value, r2.value)
-
- regalloc.possibly_free_vars_for_op(op)
- return fcond
-
- def emit_op_int_sub(self, op, regalloc, fcond):
- # assuming only one argument is constant
- res = regalloc.try_allocate_reg(op.result)
- if isinstance(op.getarg(0), ConstInt) or isinstance(op.getarg(1), ConstInt):
- if isinstance(op.getarg(1), ConstInt):
- reg = regalloc.try_allocate_reg(op.getarg(0))
- value = op.getarg(1).getint()
- if value < 0:
- self.mc.ADD_ri(res.value, reg.value, -1 * value)
- else:
- self.mc.SUB_ri(res.value, reg.value, value)
- elif isinstance(op.getarg(0), ConstInt):
- reg = regalloc.try_allocate_reg(op.getarg(1))
- value = op.getarg(0).getint()
- if value < 0:
- self.mc.ADD_ri(res.value, reg.value, -1 * value)
- self.mc.MVN_rr(res.value, res.value)
- else:
- # reverse substract ftw
- self.mc.RSB_ri(res.value, reg.value, value)
- else:
- r1 = regalloc.try_allocate_reg(op.getarg(0))
- r2 = regalloc.try_allocate_reg(op.getarg(1))
- self.mc.SUB_rr(res.value, r1.value, r2.value)
-
- regalloc.possibly_free_vars_for_op(op)
- return fcond
-
- # Guards
- def _emit_guard(self, op, regalloc, fcond):
- descr = op.getdescr()
- assert isinstance(descr, BasicFailDescr)
- descr._arm_guard_code = self.mc.curraddr()
- memaddr = self._gen_path_to_exit_path(op, op.getfailargs(), regalloc, fcond)
- descr._failure_recovery_code = memaddr
- descr._arm_guard_cond = fcond
-
- def emit_op_guard_true(self, op, regalloc, fcond):
- assert fcond == c.GT
- self._emit_guard(op, regalloc, fcond)
- return c.AL
-
- def emit_op_guard_false(self, op, regalloc, fcond):
- assert fcond == c.EQ
- self._emit_guard(op, regalloc, fcond)
- return c.AL
-
-
def make_operation_list():
def notimplemented(self, op, regalloc, fcond):
- raise NotImplementedError
+ raise NotImplementedError, op
operations = [None] * (rop._LAST+1)
for key, value in rop.__dict__.items():
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Wed Oct 27 14:59:25 2010
@@ -0,0 +1,126 @@
+from pypy.jit.backend.arm import conditions as c
+from pypy.jit.backend.arm import locations
+from pypy.jit.backend.arm import registers as r
+from pypy.jit.backend.arm.arch import WORD, FUNC_ALIGN
+from pypy.jit.backend.arm.codebuilder import ARMv7Builder, ARMv7InMemoryBuilder
+from pypy.jit.backend.arm.regalloc import ARMRegisterManager
+from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity
+from pypy.jit.metainterp.history import ConstInt, BoxInt, Box, BasicFailDescr
+from pypy.jit.metainterp.resoperation import rop
+from pypy.rlib import rgc
+from pypy.rpython.annlowlevel import llhelper
+from pypy.rpython.lltypesystem import lltype, rffi, llmemory
+
+class IntOpAsslember(object):
+ _mixin_ = True
+
+ # XXX support constants larger than imm
+
+ def emit_op_int_add(self, op, regalloc, fcond):
+ # assuming only one argument is constant
+ res = regalloc.try_allocate_reg(op.result)
+ if isinstance(op.getarg(0), ConstInt) or isinstance(op.getarg(1), ConstInt):
+ if isinstance(op.getarg(1), ConstInt):
+ reg = regalloc.try_allocate_reg(op.getarg(0))
+ arg1 = op.getarg(1)
+ elif isinstance(op.getarg(0), ConstInt):
+ reg = regalloc.try_allocate_reg(op.getarg(1))
+ arg1 = op.getarg(0)
+ value = arg1.getint()
+ if value < 0:
+ self.mc.SUB_ri(res.value, reg.value, -1 * value)
+ else:
+ self.mc.ADD_ri(res.value, reg.value, value)
+ else:
+ r1 = regalloc.try_allocate_reg(op.getarg(0))
+ r2 = regalloc.try_allocate_reg(op.getarg(1))
+ self.mc.ADD_rr(res.value, r1.value, r2.value)
+
+ regalloc.possibly_free_vars_for_op(op)
+ return fcond
+
+ def emit_op_int_sub(self, op, regalloc, fcond):
+ # assuming only one argument is constant
+ res = regalloc.try_allocate_reg(op.result)
+ if isinstance(op.getarg(0), ConstInt) or isinstance(op.getarg(1), ConstInt):
+ if isinstance(op.getarg(1), ConstInt):
+ reg = regalloc.try_allocate_reg(op.getarg(0))
+ value = op.getarg(1).getint()
+ if value < 0:
+ self.mc.ADD_ri(res.value, reg.value, -1 * value)
+ else:
+ self.mc.SUB_ri(res.value, reg.value, value)
+ elif isinstance(op.getarg(0), ConstInt):
+ reg = regalloc.try_allocate_reg(op.getarg(1))
+ value = op.getarg(0).getint()
+ if value < 0:
+ self.mc.ADD_ri(res.value, reg.value, -1 * value)
+ self.mc.MVN_rr(res.value, res.value)
+ else:
+ # reverse substract ftw
+ self.mc.RSB_ri(res.value, reg.value, value)
+ else:
+ r1 = regalloc.try_allocate_reg(op.getarg(0))
+ r2 = regalloc.try_allocate_reg(op.getarg(1))
+ self.mc.SUB_rr(res.value, r1.value, r2.value)
+
+ regalloc.possibly_free_vars_for_op(op)
+ return fcond
+
+ def emit_op_int_mul(self, op, regalloc, fcond):
+ import pdb; pdb.set_trace()
+
+class GuardOpAssembler(object):
+ _mixin_ = True
+
+ def _emit_guard(self, op, regalloc, fcond):
+ descr = op.getdescr()
+ assert isinstance(descr, BasicFailDescr)
+ descr._arm_guard_code = self.mc.curraddr()
+ memaddr = self._gen_path_to_exit_path(op, op.getfailargs(), regalloc, fcond)
+ descr._failure_recovery_code = memaddr
+ descr._arm_guard_cond = fcond
+
+ def emit_op_guard_true(self, op, regalloc, fcond):
+ assert fcond == c.GT
+ self._emit_guard(op, regalloc, fcond)
+ return c.AL
+
+ def emit_op_guard_false(self, op, regalloc, fcond):
+ assert fcond == c.EQ
+ self._emit_guard(op, regalloc, fcond)
+ return c.AL
+
+class OpAssembler(object):
+ _mixin_ = True
+
+ def emit_op_jump(self, op, regalloc, fcond):
+ tmp = Box()
+ tmpreg = regalloc.try_allocate_reg(tmp)
+ registers = op.getdescr()._arm_arglocs
+ for i in range(op.numargs()):
+ reg = regalloc.try_allocate_reg(op.getarg(i))
+ inpreg = registers[i]
+ # XXX only if every value is in a register
+ self.mc.MOV_rr(inpreg.value, reg.value)
+ loop_code = op.getdescr()._arm_loop_code
+ self.mc.gen_load_int(tmpreg.value, loop_code)
+ self.mc.MOV_rr(r.pc.value, tmpreg.value)
+ regalloc.possibly_free_var(tmpreg)
+ return fcond
+
+ def emit_op_finish(self, op, regalloc, fcond):
+ self._gen_path_to_exit_path(op, op.getarglist(), regalloc, fcond)
+ return fcond
+
+ def emit_op_int_le(self, op, regalloc, fcond):
+ reg = regalloc.try_allocate_reg(op.getarg(0))
+ assert isinstance(op.getarg(1), ConstInt)
+ self.mc.CMP(reg.value, op.getarg(1).getint())
+ return c.GT
+
+ def emit_op_int_eq(self, op, regalloc, fcond):
+ reg = regalloc.try_allocate_reg(op.getarg(0))
+ assert isinstance(op.getarg(1), ConstInt)
+ self.mc.CMP(reg.value, op.getarg(1).getint())
+ return c.EQ
From arigo at codespeak.net Wed Oct 27 15:15:11 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 15:15:11 +0200 (CEST)
Subject: [pypy-svn] r78353 - in pypy/trunk/pypy: module/gc module/gc/test
objspace/std
Message-ID: <20101027131511.4EE55282C1C@codespeak.net>
Author: arigo
Date: Wed Oct 27 15:15:09 2010
New Revision: 78353
Modified:
pypy/trunk/pypy/module/gc/interp_gc.py
pypy/trunk/pypy/module/gc/test/test_gc.py
pypy/trunk/pypy/objspace/std/typeobject.py
Log:
(amaury on irc, arigo)
When doing an explicit gc.collect(), also clear the method cache.
This is the easiest way to fix the "semi-leaks" shown by the new
test in test_gc. We could also make weak pointers in the cache,
but so far it would be messy and seriously slower.
Modified: pypy/trunk/pypy/module/gc/interp_gc.py
==============================================================================
--- pypy/trunk/pypy/module/gc/interp_gc.py (original)
+++ pypy/trunk/pypy/module/gc/interp_gc.py Wed Oct 27 15:15:09 2010
@@ -5,6 +5,11 @@
def collect(space):
"Run a full collection."
+ # First clear the method cache. See test_gc for an example of why.
+ if space.config.objspace.std.withmethodcache:
+ from pypy.objspace.std.typeobject import MethodCache
+ cache = space.fromcache(MethodCache)
+ cache.clear()
rgc.collect()
return space.wrap(0)
Modified: pypy/trunk/pypy/module/gc/test/test_gc.py
==============================================================================
--- pypy/trunk/pypy/module/gc/test/test_gc.py (original)
+++ pypy/trunk/pypy/module/gc/test/test_gc.py Wed Oct 27 15:15:09 2010
@@ -103,3 +103,22 @@
import gc
gc.dump_heap_stats(self.fname)
+
+class AppTestGcMethodCache(object):
+ def setup_class(cls):
+ cls.space = gettestobjspace(**{"objspace.std.withmethodcache": True})
+
+ def test_clear_method_cache(self):
+ import gc, weakref
+ rlist = []
+ def f():
+ class C(object):
+ def f(self):
+ pass
+ C().f() # Fill the method cache
+ rlist.append(weakref.ref(C))
+ for i in range(5):
+ f()
+ gc.collect() # the classes C should all go away here
+ for r in rlist:
+ assert r() is None
Modified: pypy/trunk/pypy/objspace/std/typeobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/typeobject.py (original)
+++ pypy/trunk/pypy/objspace/std/typeobject.py Wed Oct 27 15:15:09 2010
@@ -60,6 +60,15 @@
self.hits = {}
self.misses = {}
+ def clear(self):
+ None_None = (None, None)
+ for i in range(len(self.versions)):
+ self.versions[i] = None
+ for i in range(len(self.names)):
+ self.names[i] = None
+ for i in range(len(self.lookup_where)):
+ self.lookup_where[i] = None_None
+
class W_TypeObject(W_Object):
from pypy.objspace.std.typetype import type_typedef as typedef
From arigo at codespeak.net Wed Oct 27 15:20:42 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 15:20:42 +0200 (CEST)
Subject: [pypy-svn] r78354 - in pypy/branch/cleanup-dict-impl/pypy/doc: .
config
Message-ID: <20101027132042.C0AB9282C1C@codespeak.net>
Author: arigo
Date: Wed Oct 27 15:20:41 2010
New Revision: 78354
Removed:
pypy/branch/cleanup-dict-impl/pypy/doc/config/objspace.std.withinlineddict.txt
pypy/branch/cleanup-dict-impl/pypy/doc/config/objspace.std.withsharingdict.txt
Modified:
pypy/branch/cleanup-dict-impl/pypy/doc/interpreter-optimizations.txt
Log:
Fixes to docs.
Modified: pypy/branch/cleanup-dict-impl/pypy/doc/interpreter-optimizations.txt
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/doc/interpreter-optimizations.txt (original)
+++ pypy/branch/cleanup-dict-impl/pypy/doc/interpreter-optimizations.txt Wed Oct 27 15:20:41 2010
@@ -153,8 +153,8 @@
dicts:
the representation of the instance dict contains only a list of values.
-You can enable this feature with the :config:`objspace.std.withsharingdict`
-option.
+A more advanced version of sharing dicts, called *map dicts,* is available
+with the :config:`objspace.std.withmapdict` option.
Builtin-Shadowing
+++++++++++++++++
@@ -219,8 +219,7 @@
shadowing the class attribute. If we know that there is no shadowing (since
instance dict tells us that) we can save this lookup on the instance dictionary.
-You can enable this feature with the :config:`objspace.std.withshadowtracking`
-option.
+*This was deprecated and is no longer available.*
Method Caching
From david at codespeak.net Wed Oct 27 15:29:10 2010
From: david at codespeak.net (david at codespeak.net)
Date: Wed, 27 Oct 2010 15:29:10 +0200 (CEST)
Subject: [pypy-svn] r78355 - in
pypy/branch/arm-backend/pypy/jit/backend/arm: . test
Message-ID: <20101027132910.3F78D282C1C@codespeak.net>
Author: david
Date: Wed Oct 27 15:29:08 2010
New Revision: 78355
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
Log:
Encode multiplication operations
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py Wed Oct 27 15:29:08 2010
@@ -107,6 +107,38 @@
| (crm & 0xF))
return f
+def define_multiply_instructions(name, table):
+ n = (table['op'] & 0xF) << 20 | 0x9 << 4
+ if 'acc' in table and table['acc']:
+ def f(self, rd, rn, rm, ra, cond=cond.AL, s=0):
+ self.write32(n
+ | cond << 28
+ | (s & 0x1)
+ | (rd & 0xF) << 16
+ | (ra & 0xF) << 12
+ | (rm & 0xF) << 8
+ | (rn & 0xF))
+ elif 'long' in table and table['long']:
+ def f(self, rdhi, rdlo, rn, rm, cond=cond.AL, s=0):
+ assert rdhi != rdlo
+ self.write32(n
+ | cond << 28
+ | (s & 0x1)
+ | (rdhi & 0xF) << 16
+ | (rdlo & 0xF) << 12
+ | (rm & 0xF) << 8
+ | (rn & 0xF))
+ else:
+ def f(self, rd, rn, rm, cond=cond.AL, s=0):
+ self.write32(n
+ | cond << 28
+ | (s & 0x1)
+ | (rd & 0xF) << 16
+ | (rm & 0xF) << 8
+ | (rn & 0xF))
+
+ return f
+
def imm_operation(rt, rn, imm):
return ((rn & 0xFF) << 16
| (rt & 0xFF) << 12
@@ -126,14 +158,12 @@
setattr(target, key, f)
def define_instructions(target):
- for key, val in instructions.load_store.iteritems():
- define_instruction(define_load_store_func, key, val, target)
-
- for key, val in instructions.data_proc.iteritems():
- define_instruction(define_data_proc, key, val, target)
-
- for key, val in instructions.data_proc_imm.iteritems():
- define_instruction(define_data_proc_imm, key, val, target)
-
- for key, val in instructions.supervisor_and_coproc.iteritems():
- define_instruction(define_supervisor_and_coproc, key, val, target)
+ i_g_map = [(instructions.load_store, define_load_store_func),
+ (instructions.data_proc, define_data_proc),
+ (instructions.data_proc_imm, define_data_proc_imm),
+ (instructions.supervisor_and_coproc, define_supervisor_and_coproc),
+ (instructions.multiply, define_multiply_instructions)]
+
+ for inss, gen in i_g_map:
+ for key, val in inss.iteritems():
+ define_instruction(gen, key, val, target)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py Wed Oct 27 15:29:08 2010
@@ -71,3 +71,14 @@
'B': {'op': 0x20},
'BL': {'op': 0x30},
}
+
+multiply = {
+ 'MUL': {'op':0x0},
+ 'MLA': {'op':0x2, 'acc': True},
+ 'UMAAL': {'op':0x4, 'long': True},
+ 'MLS': {'op':0x6, 'acc': True},
+ 'UMULL': {'op':0x8, 'long': True},
+ 'UMLAL': {'op':0xA, 'long': True},
+ 'SMULL': {'op':0xC, 'long': True},
+ 'SMLAL': {'op':0xE, 'long': True},
+}
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Wed Oct 27 15:29:08 2010
@@ -141,6 +141,8 @@
for key, value, in instructions.data_proc_imm.iteritems():
build_test(gen_test_data_proc_imm_func, key, value, test_name)
+ for key, value, in instructions.multiply.iteritems():
+ build_test(gen_test_mul_func, key, value, test_name)
# XXX refactor this functions
def build_test(builder, key, value, test_name):
@@ -180,6 +182,24 @@
self.assert_equal('%s r3, [r7, r12]' % name[:name.index('_')])
return f
+def gen_test_mul_func(name, table):
+ if 'acc' in table and table['acc']:
+ def f(self):
+ func = getattr(self.cb, name)
+ func(r.r3.value, r.r7.value, r.r12.value, r.r13.value)
+ self.assert_equal('%s r3, r7, r12, r13' % name)
+ elif 'long' in table and table['long']:
+ def f(self):
+ func = getattr(self.cb, name)
+ func(r.r3.value, r.r13.value, r.r7.value, r.r12.value)
+ self.assert_equal('%s r13, r3, r7, r12' % name)
+ else:
+ def f(self):
+ func = getattr(self.cb, name)
+ func(r.r3.value, r.r7.value, r.r12.value)
+ self.assert_equal('%s r3, r7, r12' % name)
+ return f
+
def gen_test_data_reg_func(name, table):
if name[-2:] == 'ri':
def f(self):
From arigo at codespeak.net Wed Oct 27 15:29:51 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 15:29:51 +0200 (CEST)
Subject: [pypy-svn] r78356 - pypy/build/bot2/pypybuildbot
Message-ID: <20101027132951.C28F0282C1C@codespeak.net>
Author: arigo
Date: Wed Oct 27 15:29:50 2010
New Revision: 78356
Modified:
pypy/build/bot2/pypybuildbot/master.py
Log:
Add "macmini-mvt" to the list of buildslaves.
No nightly scheduled run so far.
Modified: pypy/build/bot2/pypybuildbot/master.py
==============================================================================
--- pypy/build/bot2/pypybuildbot/master.py (original)
+++ pypy/build/bot2/pypybuildbot/master.py Wed Oct 27 15:29:50 2010
@@ -138,7 +138,7 @@
)
pypyJITTranslatedTestFactoryOSX = pypybuilds.Translated(
- platform='osx',
+ platform='osx', # works also for "OS/X 64"
translationArgs=jit_translation_args,
targetArgs=[],
lib_python=True,
@@ -178,7 +178,7 @@
JITLINUX32 = "pypy-c-jit-linux-x86-32"
JITLINUX64 = "pypy-c-jit-linux-x86-64"
OJITLINUX32 = "pypy-c-Ojit-no-jit-linux-x86-32"
-JITMACOSX32 = "pypy-c-jit-macosx-x86-32"
+JITMACOSX64 = "pypy-c-jit-macosx-x86-64"
JITWIN32 = "pypy-c-jit-win-x86-32"
JITONLYLINUX32 = "jitonly-own-linux-x86-32"
@@ -288,9 +288,9 @@
'factory': pypyJITTranslatedTestFactory64,
'category': 'jit64',
},
- {"name" : JITMACOSX32,
- "slavenames": ["minime"],
- 'builddir' : JITMACOSX32,
+ {"name" : JITMACOSX64,
+ "slavenames": ["macmini-mvt"],
+ 'builddir' : JITMACOSX64,
'factory' : pypyJITTranslatedTestFactoryOSX,
'category' : 'jit',
},
From arigo at codespeak.net Wed Oct 27 15:37:29 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 15:37:29 +0200 (CEST)
Subject: [pypy-svn] r78357 - pypy/trunk/pypy/jit/backend
Message-ID: <20101027133729.AA9975080C@codespeak.net>
Author: arigo
Date: Wed Oct 27 15:37:28 2010
New Revision: 78357
Modified:
pypy/trunk/pypy/jit/backend/detect_cpu.py
Log:
Detect 32-vs-64-bits processors by relying systematically
on the value of sys.maxint.
Modified: pypy/trunk/pypy/jit/backend/detect_cpu.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/detect_cpu.py (original)
+++ pypy/trunk/pypy/jit/backend/detect_cpu.py Wed Oct 27 15:37:28 2010
@@ -23,11 +23,6 @@
mach = os.popen('uname -m', 'r').read().strip()
if not mach:
raise ProcessorAutodetectError, "cannot run 'uname -m'"
- if mach == 'x86_64':
- if sys.maxint == 2147483647:
- mach = 'x86' # it's a 64-bit processor but in 32-bits mode, maybe
- else:
- assert sys.maxint == 2 ** 63 - 1
try:
return {'i386': 'x86',
'i486': 'x86',
@@ -36,17 +31,21 @@
'i86pc': 'x86', # Solaris/Intel
'x86': 'x86', # Apple
'Power Macintosh': 'ppc',
- 'x86_64': 'x86_64',
+ 'x86_64': 'x86',
}[mach]
except KeyError:
raise ProcessorAutodetectError, "unsupported processor '%s'" % mach
def autodetect():
model = autodetect_main_model()
- if model == 'x86':
- from pypy.jit.backend.x86.detect_sse2 import detect_sse2
- if not detect_sse2():
- model = 'x86-without-sse2'
+ if sys.maxint == 2**63-1:
+ model += '_64'
+ else:
+ assert sys.maxint == 2**31-1
+ if model == 'x86':
+ from pypy.jit.backend.x86.detect_sse2 import detect_sse2
+ if not detect_sse2():
+ model = 'x86-without-sse2'
return model
def getcpuclassname(backend_name="auto"):
From arigo at codespeak.net Wed Oct 27 16:03:52 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 16:03:52 +0200 (CEST)
Subject: [pypy-svn] r78358 - pypy/branch/cleanup-dict-impl/pypy/doc/config
Message-ID: <20101027140352.C99B5282C1C@codespeak.net>
Author: arigo
Date: Wed Oct 27 16:03:50 2010
New Revision: 78358
Modified:
pypy/branch/cleanup-dict-impl/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt
Log:
Kill dead link.
Modified: pypy/branch/cleanup-dict-impl/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt (original)
+++ pypy/branch/cleanup-dict-impl/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt Wed Oct 27 16:03:50 2010
@@ -5,8 +5,6 @@
case. So far, this only works for calls with no keyword, no ``*arg``
and no ``**arg`` but it would be easy to extend.
-Gives the best results combined with :config:`objspace.std.withshadowtracking`.
-
For more information, see the section in `Standard Interpreter Optimizations`_.
.. _`Standard Interpreter Optimizations`: ../interpreter-optimizations.html#lookup-method-call-method
From arigo at codespeak.net Wed Oct 27 16:20:29 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 16:20:29 +0200 (CEST)
Subject: [pypy-svn] r78359 - in pypy/trunk/pypy: config config/test doc
doc/config interpreter jit/codewriter jit/tl
module/__builtin__ module/__builtin__/test
module/pypyjit/test objspace objspace/std objspace/std/test
Message-ID: <20101027142029.C28495080C@codespeak.net>
Author: arigo
Date: Wed Oct 27 16:20:26 2010
New Revision: 78359
Added:
pypy/trunk/pypy/objspace/std/test/test_methodcache.py
- copied, changed from r78358, pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_methodcache.py
Removed:
pypy/trunk/pypy/doc/config/objspace.std.withinlineddict.txt
pypy/trunk/pypy/doc/config/objspace.std.withshadowtracking.txt
pypy/trunk/pypy/doc/config/objspace.std.withsharingdict.txt
pypy/trunk/pypy/objspace/std/inlinedict.py
pypy/trunk/pypy/objspace/std/sharingdict.py
pypy/trunk/pypy/objspace/std/test/test_inlinedict.py
pypy/trunk/pypy/objspace/std/test/test_shadowtracking.py
pypy/trunk/pypy/objspace/std/test/test_sharingdict.py
Modified:
pypy/trunk/pypy/config/pypyoption.py
pypy/trunk/pypy/config/test/test_pypyoption.py
pypy/trunk/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt
pypy/trunk/pypy/doc/interpreter-optimizations.txt
pypy/trunk/pypy/interpreter/baseobjspace.py
pypy/trunk/pypy/interpreter/typedef.py
pypy/trunk/pypy/jit/codewriter/jtransform.py
pypy/trunk/pypy/jit/tl/pypyjit.py
pypy/trunk/pypy/module/__builtin__/interp_classobj.py
pypy/trunk/pypy/module/__builtin__/test/test_classobj.py
pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
pypy/trunk/pypy/objspace/descroperation.py
pypy/trunk/pypy/objspace/std/callmethod.py
pypy/trunk/pypy/objspace/std/celldict.py
pypy/trunk/pypy/objspace/std/dictmultiobject.py
pypy/trunk/pypy/objspace/std/mapdict.py
pypy/trunk/pypy/objspace/std/objspace.py
pypy/trunk/pypy/objspace/std/proxyobject.py
pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py
Log:
(cfbolz, arigo)
Merge branch/cleanup-dict-impl: get rid of the inlinedict and sharingdict
and use mapdict instead -- now by default with the JIT, including a fix
that stops mapdict from creating tons of versions of the code.
Modified: pypy/trunk/pypy/config/pypyoption.py
==============================================================================
--- pypy/trunk/pypy/config/pypyoption.py (original)
+++ pypy/trunk/pypy/config/pypyoption.py Wed Oct 27 16:20:26 2010
@@ -229,27 +229,15 @@
requires=[("objspace.opcodes.CALL_LIKELY_BUILTIN", False),
("objspace.honor__builtins__", False)]),
- BoolOption("withsharingdict",
- "use dictionaries that share the keys part",
- default=False),
-
BoolOption("withdictmeasurement",
"create huge files with masses of information "
"about dictionaries",
default=False),
- BoolOption("withinlineddict",
- "make instances more compact by revoming a level of indirection",
- default=False,
- requires=[("objspace.std.withshadowtracking", False)]),
-
BoolOption("withmapdict",
"make instances really small but slow without the JIT",
default=False,
- requires=[("objspace.std.withshadowtracking", False),
- ("objspace.std.withinlineddict", False),
- ("objspace.std.withsharingdict", False),
- ("objspace.std.getattributeshortcut", True),
+ requires=[("objspace.std.getattributeshortcut", True),
("objspace.std.withtypeversion", True),
]),
@@ -266,12 +254,6 @@
# weakrefs needed, because of get_subclasses()
requires=[("translation.rweakref", True)]),
- BoolOption("withshadowtracking",
- "track whether an instance attribute shadows a type"
- " attribute",
- default=False,
- requires=[("objspace.std.withtypeversion", True),
- ("translation.rweakref", True)]),
BoolOption("withmethodcache",
"try to cache method lookups",
default=False,
@@ -343,9 +325,6 @@
config.objspace.std.suggest(optimized_list_getitem=True)
config.objspace.std.suggest(getattributeshortcut=True)
config.objspace.std.suggest(newshortcut=True)
- if type_system != 'ootype':
- config.objspace.std.suggest(withsharingdict=True)
- config.objspace.std.suggest(withinlineddict=True)
# extra costly optimizations only go in level 3
if level == '3':
@@ -374,7 +353,7 @@
# extra optimizations with the JIT
if level == 'jit':
config.objspace.std.suggest(withcelldict=True)
- #config.objspace.std.suggest(withmapdict=True)
+ config.objspace.std.suggest(withmapdict=True)
def enable_allworkingmodules(config):
Modified: pypy/trunk/pypy/config/test/test_pypyoption.py
==============================================================================
--- pypy/trunk/pypy/config/test/test_pypyoption.py (original)
+++ pypy/trunk/pypy/config/test/test_pypyoption.py Wed Oct 27 16:20:26 2010
@@ -47,7 +47,7 @@
def test_set_pypy_opt_level():
conf = get_pypy_config()
set_pypy_opt_level(conf, '2')
- assert conf.objspace.std.withsharingdict
+ assert conf.objspace.std.newshortcut
conf = get_pypy_config()
set_pypy_opt_level(conf, '0')
assert not conf.objspace.std.newshortcut
@@ -59,7 +59,6 @@
assert not conf.objspace.std.withtypeversion
assert not conf.objspace.std.withmethodcache
- assert not conf.objspace.std.withshadowtracking
def test_check_documentation():
def check_file_exists(fn):
Modified: pypy/trunk/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt
==============================================================================
--- pypy/trunk/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt (original)
+++ pypy/trunk/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt Wed Oct 27 16:20:26 2010
@@ -5,8 +5,6 @@
case. So far, this only works for calls with no keyword, no ``*arg``
and no ``**arg`` but it would be easy to extend.
-Gives the best results combined with :config:`objspace.std.withshadowtracking`.
-
For more information, see the section in `Standard Interpreter Optimizations`_.
.. _`Standard Interpreter Optimizations`: ../interpreter-optimizations.html#lookup-method-call-method
Modified: pypy/trunk/pypy/doc/interpreter-optimizations.txt
==============================================================================
--- pypy/trunk/pypy/doc/interpreter-optimizations.txt (original)
+++ pypy/trunk/pypy/doc/interpreter-optimizations.txt Wed Oct 27 16:20:26 2010
@@ -153,8 +153,8 @@
dicts:
the representation of the instance dict contains only a list of values.
-You can enable this feature with the :config:`objspace.std.withsharingdict`
-option.
+A more advanced version of sharing dicts, called *map dicts,* is available
+with the :config:`objspace.std.withmapdict` option.
Builtin-Shadowing
+++++++++++++++++
@@ -219,8 +219,7 @@
shadowing the class attribute. If we know that there is no shadowing (since
instance dict tells us that) we can save this lookup on the instance dictionary.
-You can enable this feature with the :config:`objspace.std.withshadowtracking`
-option.
+*This was deprecated and is no longer available.*
Method Caching
Modified: pypy/trunk/pypy/interpreter/baseobjspace.py
==============================================================================
--- pypy/trunk/pypy/interpreter/baseobjspace.py (original)
+++ pypy/trunk/pypy/interpreter/baseobjspace.py Wed Oct 27 16:20:26 2010
@@ -36,13 +36,10 @@
return space.finditem_str(w_dict, attr)
return None
- def getdictvalue_attr_is_in_class(self, space, attr):
- return self.getdictvalue(space, attr)
-
- def setdictvalue(self, space, attr, w_value, shadows_type=True):
+ def setdictvalue(self, space, attr, w_value):
w_dict = self.getdict()
if w_dict is not None:
- space.setitem_str(w_dict, attr, w_value, shadows_type)
+ space.setitem_str(w_dict, attr, w_value)
return True
return False
@@ -657,7 +654,7 @@
"""shortcut for space.int_w(space.hash(w_obj))"""
return self.int_w(self.hash(w_obj))
- def setitem_str(self, w_obj, key, w_value, shadows_type=True):
+ def setitem_str(self, w_obj, key, w_value):
return self.setitem(w_obj, self.wrap(key), w_value)
def finditem_str(self, w_obj, key):
Modified: pypy/trunk/pypy/interpreter/typedef.py
==============================================================================
--- pypy/trunk/pypy/interpreter/typedef.py (original)
+++ pypy/trunk/pypy/interpreter/typedef.py Wed Oct 27 16:20:26 2010
@@ -262,16 +262,7 @@
return self.slots_w[index]
add(Proto)
- wantdict = "dict" in features
- if wantdict and config.objspace.std.withinlineddict:
- from pypy.objspace.std.objectobject import W_ObjectObject
- from pypy.objspace.std.inlinedict import make_mixin
- if supercls is W_ObjectObject:
- Mixin = make_mixin(config)
- add(Mixin)
- wantdict = False
-
- if wantdict:
+ if "dict" in features:
base_user_setup = supercls.user_setup.im_func
if "user_setup" in body:
base_user_setup = body["user_setup"]
@@ -290,15 +281,6 @@
def setclass(self, space, w_subtype):
# only used by descr_set___class__
self.w__class__ = w_subtype
- if space.config.objspace.std.withshadowtracking:
- self.w__dict__.set_shadows_anything()
-
- def getdictvalue_attr_is_in_class(self, space, name):
- w_dict = self.w__dict__
- if space.config.objspace.std.withshadowtracking:
- if not w_dict.shadows_anything():
- return None
- return space.finditem_str(w_dict, name)
add(Proto)
Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/trunk/pypy/jit/codewriter/jtransform.py Wed Oct 27 16:20:26 2010
@@ -172,6 +172,7 @@
def rewrite_op_same_as(self, op): pass
def rewrite_op_cast_pointer(self, op): pass
+ def rewrite_op_cast_opaque_ptr(self, op): pass # rlib.rerased
def rewrite_op_cast_primitive(self, op): pass
def rewrite_op_cast_bool_to_int(self, op): pass
def rewrite_op_cast_bool_to_uint(self, op): pass
Modified: pypy/trunk/pypy/jit/tl/pypyjit.py
==============================================================================
--- pypy/trunk/pypy/jit/tl/pypyjit.py (original)
+++ pypy/trunk/pypy/jit/tl/pypyjit.py Wed Oct 27 16:20:26 2010
@@ -44,7 +44,6 @@
config.objspace.usemodules._ffi = True
#
set_pypy_opt_level(config, level='jit')
-config.objspace.std.withinlineddict = True
if BACKEND == 'c':
config.objspace.std.multimethods = 'mrd'
Modified: pypy/trunk/pypy/module/__builtin__/interp_classobj.py
==============================================================================
--- pypy/trunk/pypy/module/__builtin__/interp_classobj.py (original)
+++ pypy/trunk/pypy/module/__builtin__/interp_classobj.py Wed Oct 27 16:20:26 2010
@@ -410,8 +410,7 @@
if w_meth is not None:
space.call_function(w_meth, w_name, w_value)
else:
- # bit obscure: appease normalization
- self.setdictvalue(space, name, w_value, True)
+ self.setdictvalue(space, name, w_value)
def descr_delattr(self, space, w_name):
name = unwrap_attr(space, w_name)
Modified: pypy/trunk/pypy/module/__builtin__/test/test_classobj.py
==============================================================================
--- pypy/trunk/pypy/module/__builtin__/test/test_classobj.py (original)
+++ pypy/trunk/pypy/module/__builtin__/test/test_classobj.py Wed Oct 27 16:20:26 2010
@@ -954,28 +954,7 @@
raises(TypeError, descr.__delete__, a)
-class AppTestOldStyleSharing(AppTestOldstyle):
- def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True})
- if option.runappdirect:
- py.test.skip("can only be run on py.py")
- def is_sharing(space, w_inst):
- from pypy.objspace.std.sharingdict import SharedDictImplementation
- w_d = w_inst.getdict()
- return space.wrap(isinstance(w_d, SharedDictImplementation) and w_d.r_dict_content is None)
- cls.w_is_sharing = cls.space.wrap(gateway.interp2app(is_sharing))
-
-
- def test_real_sharing(self):
- class A:
- def __init__(self):
- self.x = 42
- A1, A2, A3 = A(), A(), A()
- assert self.is_sharing(A3)
- assert self.is_sharing(A2)
- assert self.is_sharing(A1)
-
-class AppTestOldStyleModDict(object):
+class AppTestOldStyleClassStrDict(object):
def setup_class(cls):
if option.runappdirect:
py.test.skip("can only be run on py.py")
Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Wed Oct 27 16:20:26 2010
@@ -136,9 +136,10 @@
assert opslogfile.check()
log = logparser.parse_log_file(str(opslogfile))
parts = logparser.extract_category(log, 'jit-log-opt-')
+ self.rawloops = [part for part in parts
+ if not from_entry_bridge(part, parts)]
# skip entry bridges, they can contain random things
- self.loops = [parse(part, no_namespace=True) for part in parts
- if not from_entry_bridge(part, parts)]
+ self.loops = [parse(part, no_namespace=True) for part in self.rawloops]
self.sliced_loops = [] # contains all bytecodes of all loops
self.total_ops = 0
for loop in self.loops:
@@ -162,12 +163,11 @@
return [ops for ops in self.sliced_loops if ops.bytecode == name]
def print_loops(self):
- for loop in self.loops:
+ for rawloop in self.rawloops:
print
print '@' * 79
print
- for op in loop.operations:
- print op
+ print rawloop.rstrip()
print
print '@' * 79
@@ -278,7 +278,7 @@
assert len(ops) == 2
assert not ops[0].get_opnames("call")
assert not ops[0].get_opnames("new")
- assert len(ops[0].get_opnames("guard")) <= 3 # we get 2 withmapdict
+ assert len(ops[0].get_opnames("guard")) <= 2
assert not ops[1] # second LOOKUP_METHOD folded away
ops = self.get_by_bytecode("CALL_METHOD")
@@ -323,8 +323,8 @@
assert len(ops) == 2
assert not ops[0].get_opnames("call")
assert not ops[0].get_opnames("new")
- assert len(ops[0].get_opnames("guard")) <= 3 # we get 2 withmapdict
- assert len(ops[0].get_opnames("getfield")) <= 5 # we get <5 withmapdict
+ assert len(ops[0].get_opnames("guard")) <= 2
+ assert len(ops[0].get_opnames("getfield")) <= 4
assert not ops[1] # second LOOKUP_METHOD folded away
def test_default_and_kw(self):
Modified: pypy/trunk/pypy/objspace/descroperation.py
==============================================================================
--- pypy/trunk/pypy/objspace/descroperation.py (original)
+++ pypy/trunk/pypy/objspace/descroperation.py Wed Oct 27 16:20:26 2010
@@ -64,9 +64,7 @@
w_type = space.type(w_obj)
return space.get_and_call_function(w_get, w_descr, w_obj,
w_type)
- w_value = w_obj.getdictvalue_attr_is_in_class(space, name)
- else:
- w_value = w_obj.getdictvalue(space, name)
+ w_value = w_obj.getdictvalue(space, name)
if w_value is not None:
return w_value
if w_descr is not None:
@@ -76,13 +74,11 @@
def descr__setattr__(space, w_obj, w_name, w_value):
name = space.str_w(w_name)
w_descr = space.lookup(w_obj, name)
- shadows_type = False
if w_descr is not None:
if space.is_data_descr(w_descr):
space.set(w_descr, w_obj, w_value)
return
- shadows_type = True
- if w_obj.setdictvalue(space, name, w_value, shadows_type):
+ if w_obj.setdictvalue(space, name, w_value):
return
raiseattrerror(space, w_obj, name, w_descr)
Modified: pypy/trunk/pypy/objspace/std/callmethod.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/callmethod.py (original)
+++ pypy/trunk/pypy/objspace/std/callmethod.py Wed Oct 27 16:20:26 2010
@@ -44,7 +44,7 @@
else:
typ = type(w_descr)
if typ is function.Function or typ is function.FunctionWithFixedCode:
- w_value = w_obj.getdictvalue_attr_is_in_class(space, name)
+ w_value = w_obj.getdictvalue(space, name)
if w_value is None:
# fast method path: a function object in the class,
# nothing in the instance
@@ -103,7 +103,7 @@
w_descr = space.lookup(w_obj, methname)
typ = type(w_descr)
if typ is function.Function or typ is function.FunctionWithFixedCode:
- w_value = w_obj.getdictvalue_attr_is_in_class(space, methname)
+ w_value = w_obj.getdictvalue(space, methname)
if w_value is None:
# fast method path: a function object in the class,
# nothing in the instance
Modified: pypy/trunk/pypy/objspace/std/celldict.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/celldict.py (original)
+++ pypy/trunk/pypy/objspace/std/celldict.py Wed Oct 27 16:20:26 2010
@@ -47,7 +47,7 @@
else:
self._as_rdict().impl_fallback_setitem(w_key, w_value)
- def impl_setitem_str(self, name, w_value, shadows_type=True):
+ def impl_setitem_str(self, name, w_value):
self.getcell(name, True).w_value = w_value
def impl_delitem(self, w_key):
Modified: pypy/trunk/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/trunk/pypy/objspace/std/dictmultiobject.py Wed Oct 27 16:20:26 2010
@@ -49,14 +49,6 @@
elif space.config.objspace.std.withdictmeasurement:
assert w_type is None
return MeasuringDictImplementation(space)
- elif space.config.objspace.std.withsharingdict and instance:
- from pypy.objspace.std.sharingdict import SharedDictImplementation
- assert w_type is None
- return SharedDictImplementation(space)
- elif (space.config.objspace.std.withshadowtracking and instance and
- classofinstance is not None):
- assert w_type is None
- return ShadowDetectingDictImplementation(space, classofinstance)
elif instance or strdict or module:
assert w_type is None
return StrDictImplementation(space)
@@ -112,7 +104,7 @@
#return w_value or None
raise NotImplementedError("abstract base class")
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
raise NotImplementedError("abstract base class")
def impl_setitem(self, w_key, w_value):
@@ -165,20 +157,13 @@
key = OPTIMIZED_BUILTINS[i]
return self.impl_getitem_str(key)
- # this method will only be seen whan a certain config option is used
- def impl_shadows_anything(self):
- return True
-
- def impl_set_shadows_anything(self):
- pass
-
# _________________________________________________________________
# fallback implementation methods
def impl_fallback_setitem(self, w_key, w_value):
self.r_dict_content[w_key] = w_value
- def impl_fallback_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_fallback_setitem_str(self, key, w_value):
return self.impl_fallback_setitem(self.space.wrap(key), w_value)
def impl_fallback_delitem(self, w_key):
@@ -211,18 +196,12 @@
key = OPTIMIZED_BUILTINS[i]
return self.impl_fallback_getitem_str(key)
- def impl_fallback_shadows_anything(self):
- return True
-
- def impl_fallback_set_shadows_anything(self):
- pass
-
implementation_methods = [
("getitem", 1),
("getitem_str", 1),
("length", 0),
- ("setitem_str", 3),
+ ("setitem_str", 2),
("setitem", 2),
("delitem", 1),
("iter", 0),
@@ -231,8 +210,6 @@
("keys", 0),
("clear", 0),
("get_builtin_indexed", 1),
- ("shadows_anything", 0),
- ("set_shadows_anything", 0),
]
@@ -312,7 +289,7 @@
else:
self._as_rdict().impl_fallback_setitem(w_key, w_value)
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
self.content[key] = w_value
def impl_delitem(self, w_key):
@@ -388,47 +365,12 @@
return None, None
-class ShadowDetectingDictImplementation(StrDictImplementation):
- def __init__(self, space, w_type):
- StrDictImplementation.__init__(self, space)
- self.w_type = w_type
- self.original_version_tag = w_type.version_tag()
- if self.original_version_tag is None:
- self._shadows_anything = True
- else:
- self._shadows_anything = False
-
- def impl_setitem_str(self, key, w_value, shadows_type=True):
- if shadows_type:
- self._shadows_anything = True
- StrDictImplementation.impl_setitem_str(
- self, key, w_value, shadows_type)
-
- def impl_setitem(self, w_key, w_value):
- space = self.space
- if space.is_w(space.type(w_key), space.w_str):
- if not self._shadows_anything:
- w_obj = self.w_type.lookup(space.str_w(w_key))
- if w_obj is not None:
- self._shadows_anything = True
- StrDictImplementation.impl_setitem_str(
- self, self.space.str_w(w_key), w_value, False)
- else:
- self._as_rdict().impl_fallback_setitem(w_key, w_value)
-
- def impl_shadows_anything(self):
- return (self._shadows_anything or
- self.w_type.version_tag() is not self.original_version_tag)
-
- def impl_set_shadows_anything(self):
- self._shadows_anything = True
-
class WaryDictImplementation(StrDictImplementation):
def __init__(self, space):
StrDictImplementation.__init__(self, space)
self.shadowed = [None] * len(BUILTIN_TO_INDEX)
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
i = BUILTIN_TO_INDEX.get(key, -1)
if i != -1:
self.shadowed[i] = w_value
@@ -558,7 +500,7 @@
self.info.writes += 1
self.content[w_key] = w_value
self.info.maxcontents = max(self.info.maxcontents, len(self.content))
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
self.info.setitem_strs += 1
self.impl_setitem(self.space.wrap(key), w_value)
def impl_delitem(self, w_key):
Modified: pypy/trunk/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/mapdict.py (original)
+++ pypy/trunk/pypy/objspace/std/mapdict.py Wed Oct 27 16:20:26 2010
@@ -1,4 +1,4 @@
-from pypy.rlib import jit, objectmodel
+from pypy.rlib import jit, objectmodel, debug
from pypy.interpreter.baseobjspace import W_Root
from pypy.objspace.std.dictmultiobject import W_DictMultiObject
@@ -10,6 +10,9 @@
# attribute shapes
NUM_DIGITS = 4
+NUM_DIGITS_POW2 = 1 << NUM_DIGITS
+# note: we use "x * NUM_DIGITS_POW2" instead of "x << NUM_DIGITS" because
+# we want to propagate knowledge that the result cannot be negative
class AbstractAttribute(object):
_immutable_fields_ = ['w_cls']
@@ -69,8 +72,10 @@
attr = self._get_new_attr(selector[0], selector[1])
oldattr = obj._get_mapdict_map()
if not jit.we_are_jitted():
- oldattr._size_estimate += attr.size_estimate() - oldattr.size_estimate()
- assert oldattr.size_estimate() >= oldattr.length()
+ size_est = (oldattr._size_estimate + attr.size_estimate()
+ - oldattr.size_estimate())
+ assert size_est >= (oldattr.length() * NUM_DIGITS_POW2)
+ oldattr._size_estimate = size_est
if attr.length() > obj._mapdict_storage_length():
# note that attr.size_estimate() is always at least attr.length()
new_storage = [None] * attr.size_estimate()
@@ -188,7 +193,7 @@
self.selector = selector
self.position = back.length()
self.back = back
- self._size_estimate = self.length() << NUM_DIGITS
+ self._size_estimate = self.length() * NUM_DIGITS_POW2
def _copy_attr(self, obj, new_obj):
w_value = self.read(obj, self.selector)
@@ -291,7 +296,7 @@
def getdictvalue(self, space, attrname):
return self._get_mapdict_map().read(self, (attrname, DICT))
- def setdictvalue(self, space, attrname, w_value, shadows_type=True):
+ def setdictvalue(self, space, attrname, w_value):
return self._get_mapdict_map().write(self, (attrname, DICT), w_value)
def deldictvalue(self, space, w_name):
@@ -383,16 +388,18 @@
assert space.config.objspace.std.withmapdict
map = w_type.terminator
classes = memo_get_subclass_of_correct_size(space, cls)
+ if SUBCLASSES_MIN_FIELDS == SUBCLASSES_MAX_FIELDS:
+ return classes[0]
size = map.size_estimate()
- if not size:
- size = 1
- try:
- return classes[size - 1]
- except IndexError:
- return classes[-1]
+ debug.check_nonneg(size)
+ if size < len(classes):
+ return classes[size]
+ else:
+ return classes[len(classes)-1]
get_subclass_of_correct_size._annspecialcase_ = "specialize:arg(1)"
-NUM_SUBCLASSES = 10 # XXX tweak this number
+SUBCLASSES_MIN_FIELDS = 5 # XXX tweak these numbers
+SUBCLASSES_MAX_FIELDS = 5
def memo_get_subclass_of_correct_size(space, supercls):
key = space, supercls
@@ -401,8 +408,12 @@
except KeyError:
assert not hasattr(supercls, "__del__")
result = []
- for i in range(1, NUM_SUBCLASSES+1):
+ for i in range(SUBCLASSES_MIN_FIELDS, SUBCLASSES_MAX_FIELDS+1):
result.append(_make_subclass_size_n(supercls, i))
+ for i in range(SUBCLASSES_MIN_FIELDS):
+ result.insert(0, result[0])
+ if SUBCLASSES_MIN_FIELDS == SUBCLASSES_MAX_FIELDS:
+ assert len(set(result)) == 1
_subclass_cache[key] = result
return result
memo_get_subclass_of_correct_size._annspecialcase_ = "specialize:memo"
@@ -413,7 +424,7 @@
rangen = unroll.unrolling_iterable(range(n))
nmin1 = n - 1
rangenmin1 = unroll.unrolling_iterable(range(nmin1))
- class subcls(ObjectMixin, BaseMapdictObject, supercls):
+ class subcls(BaseMapdictObject, supercls):
def _init_empty(self, map):
from pypy.rlib.debug import make_sure_not_resized
for i in rangen:
@@ -506,8 +517,8 @@
def impl_getitem_str(self, key):
return self.w_obj.getdictvalue(self.space, key)
- def impl_setitem_str(self, key, w_value, shadows_type=True):
- flag = self.w_obj.setdictvalue(self.space, key, w_value, shadows_type)
+ def impl_setitem_str(self, key, w_value):
+ flag = self.w_obj.setdictvalue(self.space, key, w_value)
assert flag
def impl_setitem(self, w_key, w_value):
@@ -588,8 +599,6 @@
# ____________________________________________________________
# Magic caching
-# XXX we also would like getdictvalue_attr_is_in_class() above
-
class CacheEntry(object):
map = None
version_tag = None
Modified: pypy/trunk/pypy/objspace/std/objspace.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/objspace.py (original)
+++ pypy/trunk/pypy/objspace/std/objspace.py Wed Oct 27 16:20:26 2010
@@ -437,7 +437,7 @@
if is_data:
w_get = self.lookup(w_descr, "__get__")
if w_get is None:
- w_value = w_obj.getdictvalue_attr_is_in_class(self, name)
+ w_value = w_obj.getdictvalue(self, name)
if w_value is not None:
return w_value
if not is_data:
@@ -489,14 +489,12 @@
return w_obj.getitem(w_key)
return ObjSpace.finditem(self, w_obj, w_key)
- def setitem_str(self, w_obj, key, w_value, shadows_type=True):
+ def setitem_str(self, w_obj, key, w_value):
""" Same as setitem, but takes string instead of any wrapped object
-
- XXX what shadows_type means???
"""
if (isinstance(w_obj, W_DictMultiObject) and
not w_obj.user_overridden_class):
- w_obj.setitem_str(key, w_value, shadows_type)
+ w_obj.setitem_str(key, w_value)
else:
self.setitem(w_obj, self.wrap(key), w_value)
Modified: pypy/trunk/pypy/objspace/std/proxyobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/proxyobject.py (original)
+++ pypy/trunk/pypy/objspace/std/proxyobject.py Wed Oct 27 16:20:26 2010
@@ -43,7 +43,7 @@
raise
return None
- def setdictvalue(self, space, attr, w_value, shadows_type=True):
+ def setdictvalue(self, space, attr, w_value):
try:
space.call_function(self.w_controller, space.wrap('__setattr__'),
space.wrap(attr), w_value)
Modified: pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py (original)
+++ pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py Wed Oct 27 16:20:26 2010
@@ -4,7 +4,6 @@
StrDictImplementation
from pypy.objspace.std.celldict import ModuleDictImplementation
-from pypy.objspace.std.sharingdict import SharedDictImplementation
from pypy.conftest import gettestobjspace
@@ -513,32 +512,6 @@
assert getattr(a, s) == 42
-class TestW_DictSharing(TestW_DictObject):
- def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True})
-
-class AppTest_DictSharing(AppTest_DictObject):
- def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True})
-
- def test_values_does_not_share(self):
- class A(object):
- pass
- a = A()
- a.abc = 12
- l = a.__dict__.values()
- assert l == [12]
- l[0] = 24
- assert a.abc == 12
-
- def test_items(self):
- class A(object):
- pass
- a = A()
- a.abc = 12
- a.__dict__.items() == [("abc", 12)]
-
-
class AppTestModuleDict(object):
def setup_class(cls):
cls.space = gettestobjspace(**{"objspace.std.withcelldict": True})
@@ -632,10 +605,8 @@
class objspace:
class std:
withdictmeasurement = False
- withsharingdict = False
withsmalldicts = False
withcelldict = False
- withshadowtracking = False
class opcodes:
CALL_LIKELY_BUILTIN = False
@@ -770,9 +741,6 @@
string = "int"
string2 = "isinstance"
-class TestSharedDictImplementation(BaseTestRDictImplementation):
- ImplementionClass = SharedDictImplementation
-
class BaseTestDevolvedDictImplementation(BaseTestRDictImplementation):
def fill_impl(self):
@@ -794,8 +762,6 @@
string = "int"
string2 = "isinstance"
-class TestDevolvedSharedDictImplementation(BaseTestDevolvedDictImplementation):
- ImplementionClass = SharedDictImplementation
def test_module_uses_strdict():
fakespace = FakeSpace()
Copied: pypy/trunk/pypy/objspace/std/test/test_methodcache.py (from r78358, pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_methodcache.py)
==============================================================================
--- pypy/branch/cleanup-dict-impl/pypy/objspace/std/test/test_methodcache.py (original)
+++ pypy/trunk/pypy/objspace/std/test/test_methodcache.py Wed Oct 27 16:20:26 2010
@@ -29,10 +29,11 @@
def test_class_that_cannot_be_cached(self):
import __pypy__
- class metatype(type):
+ class X:
pass
- class A(object):
- __metaclass__ = metatype
+ class Y(object):
+ pass
+ class A(Y, X):
def f(self):
return 42
@@ -130,3 +131,20 @@
foo = 3
D.__bases__ = (C, F)
assert e.foo == 3
+
+ def test_custom_metaclass(self):
+ import __pypy__
+ class MetaA(type):
+ def __getattribute__(self, x):
+ return 1
+ def f(self):
+ return 42
+ A = type.__new__(MetaA, "A", (), {"f": f})
+ l = [type.__getattribute__(A, "__new__")(A)] * 10
+ __pypy__.reset_method_cache_counter()
+ for i, a in enumerate(l):
+ assert a.f() == 42
+ cache_counter = __pypy__.method_cache_counter("f")
+ assert cache_counter[0] >= 5
+ assert cache_counter[1] >= 1 # should be (27, 3)
+ assert sum(cache_counter) == 10
From arigo at codespeak.net Wed Oct 27 16:21:46 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 16:21:46 +0200 (CEST)
Subject: [pypy-svn] r78361 - pypy/branch/cleanup-dict-impl
Message-ID: <20101027142146.91E14282C1C@codespeak.net>
Author: arigo
Date: Wed Oct 27 16:21:45 2010
New Revision: 78361
Removed:
pypy/branch/cleanup-dict-impl/
Log:
(cfbolz, arigo)
Kill merged branch.
From arigo at codespeak.net Wed Oct 27 16:22:41 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 16:22:41 +0200 (CEST)
Subject: [pypy-svn] r78362 - pypy/branch/mapdict-without-jit
Message-ID: <20101027142241.7E87C282C1C@codespeak.net>
Author: arigo
Date: Wed Oct 27 16:22:40 2010
New Revision: 78362
Added:
pypy/branch/mapdict-without-jit/
- copied from r78361, pypy/trunk/
Log:
(cfbolz, arigo)
A branch where we speed up mapdict if you don't use the JIT.
From arigo at codespeak.net Wed Oct 27 16:47:56 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 16:47:56 +0200 (CEST)
Subject: [pypy-svn] r78364 - in
pypy/branch/mapdict-without-jit/pypy/objspace/std: . test
Message-ID: <20101027144756.A15C4282C1C@codespeak.net>
Author: arigo
Date: Wed Oct 27 16:47:54 2010
New Revision: 78364
Modified:
pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py
pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py
Log:
(cfbolz, arigo)
By tiny steps that all make sense, in the end we reach something
completely ridiculous.
Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py (original)
+++ pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py Wed Oct 27 16:47:54 2010
@@ -15,13 +15,14 @@
# we want to propagate knowledge that the result cannot be negative
class AbstractAttribute(object):
- _immutable_fields_ = ['w_cls']
+ _immutable_fields_ = ['terminator']
cache_attrs = None
_size_estimate = 0
- def __init__(self, space, w_cls):
+ def __init__(self, space, terminator):
self.space = space
- self.w_cls = w_cls
+ assert isinstance(terminator, Terminator)
+ self.terminator = terminator
def read(self, obj, selector):
raise NotImplementedError("abstract base class")
@@ -42,7 +43,7 @@
raise NotImplementedError("abstract base class")
def get_terminator(self):
- raise NotImplementedError("abstract base class")
+ return self.terminator
def set_terminator(self, obj, terminator):
raise NotImplementedError("abstract base class")
@@ -95,10 +96,15 @@
raise NotImplementedError("abstract base class")
def __repr__(self):
- return "<%s w_cls=%s>" % (self.__class__.__name__, self.w_cls)
+ return "<%s>" % (self.__class__.__name__,)
class Terminator(AbstractAttribute):
+ _immutable_fields_ = ['w_cls']
+
+ def __init__(self, space, w_cls):
+ AbstractAttribute.__init__(self, space, self)
+ self.w_cls = w_cls
def read(self, obj, selector):
return None
@@ -116,9 +122,6 @@
def length(self):
return 0
- def get_terminator(self):
- return self
-
def set_terminator(self, obj, terminator):
result = Object()
result.space = self.space
@@ -128,6 +131,9 @@
def remove_dict_entries(self, obj):
return self.copy(obj)
+ def __repr__(self):
+ return "<%s w_cls=%s>" % (self.__class__.__name__, self.w_cls)
+
class DictTerminator(Terminator):
_immutable_fields_ = ['devolved_dict_terminator']
def __init__(self, space, w_cls):
@@ -189,7 +195,7 @@
class PlainAttribute(AbstractAttribute):
_immutable_fields_ = ['selector', 'position', 'back']
def __init__(self, selector, back):
- AbstractAttribute.__init__(self, back.space, back.w_cls)
+ AbstractAttribute.__init__(self, back.space, back.terminator)
self.selector = selector
self.position = back.length()
self.back = back
@@ -232,9 +238,6 @@
def length(self):
return self.position + 1
- def get_terminator(self):
- return self.back.get_terminator()
-
def set_terminator(self, obj, terminator):
new_obj = self.back.set_terminator(obj, terminator)
self._copy_attr(obj, new_obj)
@@ -328,7 +331,7 @@
assert flag
def getclass(self, space):
- return self._get_mapdict_map().w_cls
+ return self._get_mapdict_map().terminator.w_cls
def setclass(self, space, w_cls):
new_obj = self._get_mapdict_map().set_terminator(self, w_cls.terminator)
@@ -609,7 +612,7 @@
INVALID_CACHE_ENTRY = CacheEntry()
INVALID_CACHE_ENTRY.map = objectmodel.instantiate(AbstractAttribute)
# different from any real map ^^^
-INVALID_CACHE_ENTRY.map.w_cls = None
+INVALID_CACHE_ENTRY.map.terminator = None
def init_mapdict_cache(pycode):
num_entries = len(pycode.co_names_w)
@@ -621,7 +624,7 @@
entry = pycode._mapdict_caches[nameindex]
map = w_obj._get_mapdict_map()
if map is entry.map:
- version_tag = map.w_cls.version_tag()
+ version_tag = map.terminator.w_cls.version_tag()
if version_tag is entry.version_tag:
# everything matches, it's incredibly fast
if pycode.space.config.objspace.std.withmethodcachecounter:
@@ -634,7 +637,7 @@
space = pycode.space
w_name = pycode.co_names_w[nameindex]
if map is not None:
- w_type = map.w_cls
+ w_type = map.terminator.w_cls
w_descr = w_type.getattribute_if_not_from_object()
if w_descr is not None:
return space._handle_getattribute(w_descr, w_obj, w_name)
Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py
==============================================================================
--- pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py (original)
+++ pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py Wed Oct 27 16:47:54 2010
@@ -30,7 +30,8 @@
PlainAttribute(("a", DICT),
Terminator(space, w_cls)))
assert aa.space is space
- assert aa.w_cls is w_cls
+ assert aa.terminator.w_cls is w_cls
+ assert aa.get_terminator() is aa.terminator
obj = Object()
obj.map, obj.storage = aa, [10, 20]
From arigo at codespeak.net Wed Oct 27 16:51:11 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 16:51:11 +0200 (CEST)
Subject: [pypy-svn] r78365 -
pypy/branch/mapdict-without-jit/pypy/objspace/std
Message-ID: <20101027145111.9ADA736C230@codespeak.net>
Author: arigo
Date: Wed Oct 27 16:51:10 2010
New Revision: 78365
Modified:
pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py
Log:
(cfbolz, arigo)
Make read() use index().
Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py (original)
+++ pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py Wed Oct 27 16:51:10 2010
@@ -25,7 +25,10 @@
self.terminator = terminator
def read(self, obj, selector):
- raise NotImplementedError("abstract base class")
+ index = self.index(selector)
+ if index < 0:
+ return self.terminator._read_terminator(obj, selector)
+ return obj._mapdict_read_storage(index)
def write(self, obj, selector, w_value):
raise NotImplementedError("abstract base class")
@@ -106,7 +109,7 @@
AbstractAttribute.__init__(self, space, self)
self.w_cls = w_cls
- def read(self, obj, selector):
+ def _read_terminator(self, obj, selector):
return None
def write(self, obj, selector, w_value):
@@ -155,12 +158,12 @@
class DevolvedDictTerminator(Terminator):
- def read(self, obj, selector):
+ def _read_terminator(self, obj, selector):
if selector[1] == DICT:
w_dict = obj.getdict()
space = self.space
return space.finditem_str(w_dict, selector[0])
- return Terminator.read(self, obj, selector)
+ return Terminator._read_terminator(self, obj, selector)
def write(self, obj, selector, w_value):
if selector[1] == DICT:
@@ -205,11 +208,6 @@
w_value = self.read(obj, self.selector)
new_obj._get_mapdict_map().add_attr(new_obj, self.selector, w_value)
- def read(self, obj, selector):
- if selector == self.selector:
- return obj._mapdict_read_storage(self.position)
- return self.back.read(obj, selector)
-
def write(self, obj, selector, w_value):
if selector == self.selector:
obj._mapdict_write_storage(self.position, w_value)
From arigo at codespeak.net Wed Oct 27 16:53:52 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 16:53:52 +0200 (CEST)
Subject: [pypy-svn] r78366 -
pypy/branch/mapdict-without-jit/pypy/objspace/std
Message-ID: <20101027145352.CF27A36C230@codespeak.net>
Author: arigo
Date: Wed Oct 27 16:53:50 2010
New Revision: 78366
Modified:
pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py
Log:
(cfbolz, arigo)
Replace write() with a call to index().
Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py (original)
+++ pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py Wed Oct 27 16:53:50 2010
@@ -31,7 +31,11 @@
return obj._mapdict_read_storage(index)
def write(self, obj, selector, w_value):
- raise NotImplementedError("abstract base class")
+ index = self.index(selector)
+ if index < 0:
+ return self.terminator._write_terminator(obj, selector, w_value)
+ obj._mapdict_write_storage(index, w_value)
+ return True
def delete(self, obj, selector):
return None
@@ -112,7 +116,7 @@
def _read_terminator(self, obj, selector):
return None
- def write(self, obj, selector, w_value):
+ def _write_terminator(self, obj, selector, w_value):
obj._get_mapdict_map().add_attr(obj, selector, w_value)
return True
@@ -151,10 +155,10 @@
class NoDictTerminator(Terminator):
- def write(self, obj, selector, w_value):
+ def _write_terminator(self, obj, selector, w_value):
if selector[1] == DICT:
return False
- return Terminator.write(self, obj, selector, w_value)
+ return Terminator._write_terminator(self, obj, selector, w_value)
class DevolvedDictTerminator(Terminator):
@@ -165,13 +169,13 @@
return space.finditem_str(w_dict, selector[0])
return Terminator._read_terminator(self, obj, selector)
- def write(self, obj, selector, w_value):
+ def _write_terminator(self, obj, selector, w_value):
if selector[1] == DICT:
w_dict = obj.getdict()
space = self.space
space.setitem_str(w_dict, selector[0], w_value)
return True
- return Terminator.write(self, obj, selector, w_value)
+ return Terminator._write_terminator(self, obj, selector, w_value)
def delete(self, obj, selector):
from pypy.interpreter.error import OperationError
@@ -208,12 +212,6 @@
w_value = self.read(obj, self.selector)
new_obj._get_mapdict_map().add_attr(new_obj, self.selector, w_value)
- def write(self, obj, selector, w_value):
- if selector == self.selector:
- obj._mapdict_write_storage(self.position, w_value)
- return True
- return self.back.write(obj, selector, w_value)
-
def delete(self, obj, selector):
if selector == self.selector:
# ok, attribute is deleted
From fijal at codespeak.net Wed Oct 27 17:17:34 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Wed, 27 Oct 2010 17:17:34 +0200 (CEST)
Subject: [pypy-svn] r78368 - pypy/extradoc/talk/pycon2011
Message-ID: <20101027151734.41AB25080C@codespeak.net>
Author: fijal
Date: Wed Oct 27 17:17:31 2010
New Revision: 78368
Added:
pypy/extradoc/talk/pycon2011/
pypy/extradoc/talk/pycon2011/pypy-optimizations.txt
- copied unchanged from r78367, user/agaynor/pycon-2011/pypy-optimizations.txt
Log:
(agaynor, fijal) create dir and import draft from agaynor's home
From arigo at codespeak.net Wed Oct 27 17:47:57 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 17:47:57 +0200 (CEST)
Subject: [pypy-svn] r78370 - in pypy/branch/mapdict-without-jit/pypy:
module/__pypy__ module/gc module/gc/test objspace/std
objspace/std/test
Message-ID: <20101027154757.EBDFC282C1D@codespeak.net>
Author: arigo
Date: Wed Oct 27 17:47:56 2010
New Revision: 78370
Modified:
pypy/branch/mapdict-without-jit/pypy/module/__pypy__/__init__.py
pypy/branch/mapdict-without-jit/pypy/module/__pypy__/interp_magic.py
pypy/branch/mapdict-without-jit/pypy/module/gc/interp_gc.py
pypy/branch/mapdict-without-jit/pypy/module/gc/test/test_gc.py
pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py
pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_dictmultiobject.py
pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py
Log:
(cfbolz, arigo)
Add the IndexCache to mapdict, similar to MethodCache.
Modified: pypy/branch/mapdict-without-jit/pypy/module/__pypy__/__init__.py
==============================================================================
--- pypy/branch/mapdict-without-jit/pypy/module/__pypy__/__init__.py (original)
+++ pypy/branch/mapdict-without-jit/pypy/module/__pypy__/__init__.py Wed Oct 27 17:47:56 2010
@@ -23,5 +23,8 @@
'interp_magic.method_cache_counter')
self.extra_interpdef('reset_method_cache_counter',
'interp_magic.reset_method_cache_counter')
+ if self.space.config.objspace.std.withmapdict:
+ self.extra_interpdef('mapdict_cache_counter',
+ 'interp_magic.mapdict_cache_counter')
PYC_MAGIC = get_pyc_magic(self.space)
self.extra_interpdef('PYC_MAGIC', 'space.wrap(%d)' % PYC_MAGIC)
Modified: pypy/branch/mapdict-without-jit/pypy/module/__pypy__/interp_magic.py
==============================================================================
--- pypy/branch/mapdict-without-jit/pypy/module/__pypy__/interp_magic.py (original)
+++ pypy/branch/mapdict-without-jit/pypy/module/__pypy__/interp_magic.py Wed Oct 27 17:47:56 2010
@@ -2,6 +2,7 @@
from pypy.interpreter.gateway import ObjSpace
from pypy.rlib.objectmodel import we_are_translated
from pypy.objspace.std.typeobject import MethodCache
+from pypy.objspace.std.mapdict import IndexCache
def internal_repr(space, w_object):
return space.wrap('%r' % (w_object,))
@@ -36,4 +37,17 @@
cache = space.fromcache(MethodCache)
cache.misses = {}
cache.hits = {}
-
+ if space.config.objspace.std.withmapdict:
+ cache = space.fromcache(IndexCache)
+ cache.misses = {}
+ cache.hits = {}
+
+def mapdict_cache_counter(space, name):
+ """Return a tuple (index_cache_hits, index_cache_misses) for lookups
+ in the mapdict cache with the given attribute name."""
+ assert space.config.objspace.std.withmethodcachecounter
+ assert space.config.objspace.std.withmapdict
+ cache = space.fromcache(IndexCache)
+ return space.newtuple([space.newint(cache.hits.get(name, 0)),
+ space.newint(cache.misses.get(name, 0))])
+mapdict_cache_counter.unwrap_spec = [ObjSpace, str]
Modified: pypy/branch/mapdict-without-jit/pypy/module/gc/interp_gc.py
==============================================================================
--- pypy/branch/mapdict-without-jit/pypy/module/gc/interp_gc.py (original)
+++ pypy/branch/mapdict-without-jit/pypy/module/gc/interp_gc.py Wed Oct 27 17:47:56 2010
@@ -10,6 +10,10 @@
from pypy.objspace.std.typeobject import MethodCache
cache = space.fromcache(MethodCache)
cache.clear()
+ if space.config.objspace.std.withmapdict:
+ from pypy.objspace.std.mapdict import IndexCache
+ cache = space.fromcache(IndexCache)
+ cache.clear()
rgc.collect()
return space.wrap(0)
Modified: pypy/branch/mapdict-without-jit/pypy/module/gc/test/test_gc.py
==============================================================================
--- pypy/branch/mapdict-without-jit/pypy/module/gc/test/test_gc.py (original)
+++ pypy/branch/mapdict-without-jit/pypy/module/gc/test/test_gc.py Wed Oct 27 17:47:56 2010
@@ -122,3 +122,8 @@
gc.collect() # the classes C should all go away here
for r in rlist:
assert r() is None
+
+class AppTestGcMapDictIndexCache(AppTestGcMethodCache):
+ def setup_class(cls):
+ cls.space = gettestobjspace(**{"objspace.std.withmethodcache": True,
+ "objspace.std.withmapdict": True})
Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py (original)
+++ pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py Wed Oct 27 17:47:56 2010
@@ -1,4 +1,5 @@
from pypy.rlib import jit, objectmodel, debug
+from pypy.rlib.rarithmetic import intmask, r_uint
from pypy.interpreter.baseobjspace import W_Root
from pypy.objspace.std.dictmultiobject import W_DictMultiObject
@@ -40,7 +41,44 @@
def delete(self, obj, selector):
return None
+ @jit.purefunction
def index(self, selector):
+ if self.space.config.objspace.std.withmethodcache:
+ return self._index_cache(selector)
+ else:
+ return self._index(selector)
+
+ def _index_cache(self, selector):
+ space = self.space
+ cache = space.fromcache(IndexCache)
+ SHIFT2 = r_uint.BITS - space.config.objspace.std.methodcachesizeexp
+ SHIFT1 = SHIFT2 - 5
+ attrs_as_int = objectmodel.current_object_addr_as_int(self)
+ # ^^^Note: see comment in typeobject.py for
+ # _pure_lookup_where_with_method_cache()
+ hash_selector = objectmodel.compute_hash(selector)
+ product = intmask(attrs_as_int * hash_selector)
+ index_hash = (r_uint(product) ^ (r_uint(product) << SHIFT1)) >> SHIFT2
+ # ^^^Note2: same comment too
+ cached_attr = cache.attrs[index_hash]
+ if cached_attr is self:
+ cached_selector = cache.selectors[index_hash]
+ if cached_selector == selector:
+ index = cache.indices[index_hash]
+ if space.config.objspace.std.withmethodcachecounter:
+ name = selector[0]
+ cache.hits[name] = cache.hits.get(name, 0) + 1
+ return index
+ index = self._index(selector)
+ cache.attrs[index_hash] = self
+ cache.selectors[index_hash] = selector
+ cache.indices[index_hash] = index
+ if space.config.objspace.std.withmethodcachecounter:
+ name = selector[0]
+ cache.misses[name] = cache.misses.get(name, 0) + 1
+ return index
+
+ def _index(self, selector):
return -1
def copy(self, obj):
@@ -221,10 +259,10 @@
self._copy_attr(obj, new_obj)
return new_obj
- def index(self, selector):
+ def _index(self, selector):
if selector == self.selector:
return self.position
- return self.back.index(selector)
+ return self.back._index(selector)
def copy(self, obj):
new_obj = self.back.copy(obj)
@@ -267,6 +305,23 @@
# RPython reasons
w_obj._set_mapdict_storage_and_map(new_obj.storage, new_obj.map)
+class IndexCache(object):
+ def __init__(self, space):
+ assert space.config.objspace.std.withmethodcache
+ SIZE = 1 << space.config.objspace.std.methodcachesizeexp
+ self.attrs = [None] * SIZE
+ self.selectors = [None] * SIZE
+ self.indices = [0] * SIZE
+ if space.config.objspace.std.withmethodcachecounter:
+ self.hits = {}
+ self.misses = {}
+
+ def clear(self):
+ for i in range(len(self.attrs)):
+ self.attrs[i] = None
+ for i in range(len(self.selectors)):
+ self.selectors[i] = None
+
# ____________________________________________________________
# object implementation
Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_dictmultiobject.py
==============================================================================
--- pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_dictmultiobject.py (original)
+++ pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_dictmultiobject.py Wed Oct 27 17:47:56 2010
@@ -607,6 +607,7 @@
withdictmeasurement = False
withsmalldicts = False
withcelldict = False
+ withmethodcache = False
class opcodes:
CALL_LIKELY_BUILTIN = False
Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py
==============================================================================
--- pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py (original)
+++ pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py Wed Oct 27 17:47:56 2010
@@ -24,7 +24,6 @@
hasdict = False
def test_plain_attribute():
- space = " "
w_cls = "class"
aa = PlainAttribute(("b", DICT),
PlainAttribute(("a", DICT),
@@ -778,3 +777,52 @@
assert res == (0, 0, 1)
res = self.check(f, 'x')
assert res == (0, 0, 1)
+
+class AppTestCaching(AppTestWithMapDict):
+ def setup_class(cls):
+ cls.space = gettestobjspace(
+ **{"objspace.std.withmethodcachecounter": True,
+ "objspace.std.withmapdict": True,
+ "objspace.opcodes.CALL_METHOD": True})
+
+ def test_mix_classes(self):
+ import __pypy__
+ class A(object):
+ def f(self):
+ return 42
+ class B(object):
+ def f(self):
+ return 43
+ class C(object):
+ def f(self):
+ return 44
+ l = [A(), B(), C()] * 10
+ __pypy__.reset_method_cache_counter()
+ # 'exec' to make sure that a.f() is compiled with CALL_METHOD
+ exec """for i, a in enumerate(l):
+ assert a.f() == 42 + i % 3
+"""
+ cache_counter = __pypy__.mapdict_cache_counter("f")
+ assert cache_counter[0] >= 15
+ assert cache_counter[1] >= 3 # should be (27, 3)
+ assert sum(cache_counter) == 30
+
+ def test_mix_classes_attribute(self):
+ import __pypy__
+ class A(object):
+ def __init__(self):
+ self.x = 42
+ class B(object):
+ def __init__(self):
+ self.x = 43
+ class C(object):
+ def __init__(self):
+ self.x = 44
+ l = [A(), B(), C()] * 10
+ __pypy__.reset_method_cache_counter()
+ for i, a in enumerate(l):
+ assert a.x == 42 + i % 3
+ cache_counter = __pypy__.mapdict_cache_counter("x")
+ assert cache_counter[0] >= 15
+ assert cache_counter[1] >= 3 # should be (27, 3)
+ assert sum(cache_counter) == 30
From arigo at codespeak.net Wed Oct 27 18:05:51 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 18:05:51 +0200 (CEST)
Subject: [pypy-svn] r78372 -
pypy/branch/mapdict-without-jit/pypy/objspace/std
Message-ID: <20101027160551.26FCA5080C@codespeak.net>
Author: arigo
Date: Wed Oct 27 18:05:48 2010
New Revision: 78372
Modified:
pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py
Log:
Translation fix
Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py (original)
+++ pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py Wed Oct 27 18:05:48 2010
@@ -310,7 +310,8 @@
assert space.config.objspace.std.withmethodcache
SIZE = 1 << space.config.objspace.std.methodcachesizeexp
self.attrs = [None] * SIZE
- self.selectors = [None] * SIZE
+ self._empty_selector = (None, INVALID)
+ self.selectors = [self._empty_selector] * SIZE
self.indices = [0] * SIZE
if space.config.objspace.std.withmethodcachecounter:
self.hits = {}
@@ -320,7 +321,7 @@
for i in range(len(self.attrs)):
self.attrs[i] = None
for i in range(len(self.selectors)):
- self.selectors[i] = None
+ self.selectors[i] = self._empty_selector
# ____________________________________________________________
# object implementation
From fijal at codespeak.net Wed Oct 27 19:18:01 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Wed, 27 Oct 2010 19:18:01 +0200 (CEST)
Subject: [pypy-svn] r78373 - pypy/extradoc/talk/pycon2011
Message-ID: <20101027171801.E0F23282C1D@codespeak.net>
Author: fijal
Date: Wed Oct 27 19:17:59 2010
New Revision: 78373
Modified:
pypy/extradoc/talk/pycon2011/pypy-optimizations.txt
Log:
(agaynor, fijal) improve
Modified: pypy/extradoc/talk/pycon2011/pypy-optimizations.txt
==============================================================================
--- pypy/extradoc/talk/pycon2011/pypy-optimizations.txt (original)
+++ pypy/extradoc/talk/pycon2011/pypy-optimizations.txt Wed Oct 27 19:17:59 2010
@@ -7,10 +7,37 @@
Classification: Discuss in depth
Abstract:
-This talk is 3 parts, what's hard to optimize about Python, how PyPy is organized, and what optimizations our JIT can do.
+The talk would contain a list of features that are hard to optimize
+in Python, why and how we're fixing it in PyPy. This also might be
+used as a rough guide what features in Python are optimizable right
+now and what are not.
+
+The list of mentioned features is not exhaustive, however we will try to
+focus at least on the following:
+
+* Dynamic language - In Python code we have no known types, like a
+ static language. Even operations like "a + b" can do anything,
+ unless we know more about the code.
+
+* Frame introspection - Frame objects need to be allocated for every
+ function call, and all local variables are stored on the frame, and
+ must be accessible from further down the call stack.
+* Object model - All user defined Python objects have a dictionary which
+ stores their attributes, as does every type. When Python does an
+ attribute lookup this requires 2 dictionary lookups.
+
+ In PyPy we use an approach similar to one used by v8 with hidden classes
+ (except more pypy specific) called shared dictionaries.
+
+* FFI calls - calling C from Python is costly and hard to optimize.
+ In PyPy we decided to go via ctypes, this part will explain how are
+ we going to handle ctypes calls to be fast.
+
+This talk is 3 parts, what's hard to optimize about Python, how PyPy is organized, and what optimizations our JIT can do.
+
I. Why Python is hard (8 minutes)
- A) Dynamic language
+ A) Dynamic language - No known types, even operations such as "+" can
B) Frame introspection.
C) Object model.
D) CPython
From afa at codespeak.net Wed Oct 27 19:53:31 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 27 Oct 2010 19:53:31 +0200 (CEST)
Subject: [pypy-svn] r78374 - in pypy/branch/fast-forward/pypy/objspace/std:
. test
Message-ID: <20101027175331.42054282BE0@codespeak.net>
Author: afa
Date: Wed Oct 27 19:53:29 2010
New Revision: 78374
Modified:
pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py
Log:
Slice assignment cannot change the size in RPython.
unfortunately, this is only true when translated...
Modified: pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py Wed Oct 27 19:53:29 2010
@@ -397,9 +397,31 @@
if step != 1:
raise OperationError(space.w_NotImplementedError,
space.wrap("fixme: only step=1 for the moment"))
+ _setitem_helper(w_bytearray, start, stop, slicelength,
+ space.str_w(w_other))
+
+def _setitem_helper(w_bytearray, start, stop, slicelength, data):
assert start >= 0
assert stop >= 0
- w_bytearray.data[start:stop] = [c for c in space.str_w(w_other)]
+ step = 1
+ len2 = len(data)
+ delta = slicelength - len2
+ if delta < 0:
+ delta = -delta
+ newsize = len(w_bytearray.data) + delta
+ w_bytearray.data += ['\0'] * delta
+ lim = start + len2
+ i = newsize - 1
+ while i >= lim:
+ w_bytearray.data[i] = w_bytearray.data[i-delta]
+ i -= 1
+ elif start >= 0:
+ del w_bytearray.data[start:start+delta]
+ else:
+ assert delta == 0
+ for i in range(len2):
+ w_bytearray.data[start] = data[i]
+ start += step
# __________________________________________________________
# Buffer interface
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py Wed Oct 27 19:53:29 2010
@@ -175,6 +175,8 @@
assert b == 'ABC...defghi'
b[3:6] = '()'
assert b == 'ABC()defghi'
+ b[6:6] = '<<'
+ assert b == 'ABC()d<
Author: afa
Date: Wed Oct 27 19:54:43 2010
New Revision: 78375
Added:
pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py (contents, props changed)
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py
Log:
Start to implement _io.BytesIO
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Wed Oct 27 19:54:43 2010
@@ -17,6 +17,29 @@
pass
class W_BufferedIOBase(W_IOBase):
+ def _unsupportedoperation(self, space, message):
+ w_exc = space.getattr(space.getbuiltinmodule('_io'),
+ space.wrap('UnsupportedOperation'))
+ raise OperationError(w_exc, space.wrap(message))
+
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def read_w(self, space, w_size=None):
+ self._unsupportedoperation(space, "read")
+
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def write_w(self, space, w_data):
+ self._unsupportedoperation(space, "write")
+
+W_BufferedIOBase.typedef = TypeDef(
+ '_BufferedIOBase', W_IOBase.typedef,
+ __new__ = generic_new_descr(W_BufferedIOBase),
+ read = interp2app(W_BufferedIOBase.read_w),
+ write = interp2app(W_BufferedIOBase.write_w),
+ )
+
+class BufferedMixin:
+ _mixin_ = True
+
def __init__(self, space):
W_IOBase.__init__(self, space)
self.state = STATE_ZERO
@@ -37,19 +60,6 @@
self.readable = False
self.writable = False
- def _unsupportedoperation(self, space, message):
- w_exc = space.getattr(space.getbuiltinmodule('_io'),
- space.wrap('UnsupportedOperation'))
- raise OperationError(w_exc, space.wrap(message))
-
- @unwrap_spec('self', ObjSpace, W_Root)
- def read_w(self, space, w_size=None):
- self._unsupportedoperation(space, "read")
-
- @unwrap_spec('self', ObjSpace, W_Root)
- def write_w(self, space, w_size=None):
- self._unsupportedoperation(space, "write")
-
def _reader_reset_buf(self):
self.read_end = -1
@@ -57,16 +67,6 @@
self.write_pos = 0
self.write_end = -1
-W_BufferedIOBase.typedef = TypeDef(
- '_BufferedIOBase', W_IOBase.typedef,
- __new__ = generic_new_descr(W_BufferedIOBase),
- read = interp2app(W_BufferedIOBase.read_w),
- write = interp2app(W_BufferedIOBase.write_w),
- )
-
-class BufferedMixin:
- _mixin_ = True
-
def _init(self, space):
if self.buffer_size <= 0:
raise OperationError(space.w_ValueError, space.wrap(
@@ -245,11 +245,6 @@
return self._write(space, ''.join(l))
class W_BufferedReader(BufferedMixin, W_BufferedIOBase):
- def __init__(self, space):
- W_BufferedIOBase.__init__(self, space)
- self.ok = False
- self.detached = False
-
@unwrap_spec('self', ObjSpace, W_Root, int)
def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE):
self.state = STATE_ZERO
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py Wed Oct 27 19:54:43 2010
@@ -1,11 +1,97 @@
-from pypy.module._io.interp_bufferedio import W_BufferedIOBase
from pypy.interpreter.typedef import (
TypeDef, generic_new_descr)
+from pypy.interpreter.gateway import interp2app, unwrap_spec
+from pypy.interpreter.baseobjspace import ObjSpace, W_Root
+from pypy.module._io.interp_bufferedio import W_BufferedIOBase
+from pypy.module._io.interp_iobase import convert_size
+
+def buffer2string(buffer, start, end):
+ from pypy.rlib.rstring import StringBuilder
+ builder = StringBuilder(end - start)
+ for i in range(start, end):
+ builder.append(buffer[i])
+ return builder.build()
class W_BytesIO(W_BufferedIOBase):
- pass
+ def __init__(self, space):
+ W_BufferedIOBase.__init__(self, space)
+ self.pos = 0
+ self.string_size = 0
+ self.buf = []
+
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def descr_init(self, space, w_initvalue=None):
+ # In case __init__ is called multiple times
+ self.string_size = 0
+ self.pos = 0
+
+ if not space.is_w(w_initvalue, space.w_None):
+ self.write_w(space, w_initvalue)
+ self.pos = 0
+
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def read_w(self, space, w_size=None):
+ self._check_closed(space)
+ size = convert_size(space, w_size)
+
+ # adjust invalid sizes
+ n = self.string_size - self.pos
+ if not 0 <= n <= size:
+ size = n
+ if size < 0:
+ size = 0
+
+ output = buffer2string(self.buf, self.pos, self.pos + size)
+ self.pos += size
+ return space.wrap(output)
+
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def write_w(self, space, w_data):
+ self._check_closed(space)
+ buf = space.buffer_w(w_data)
+ length = buf.getlength()
+ if length == 0:
+ return
+
+ if self.pos + length > len(self.buf):
+ self.buf.extend(['\0'] * (self.pos + length - len(self.buf)))
+
+ if self.pos > self.string_size:
+ # In case of overseek, pad with null bytes the buffer region
+ # between the end of stream and the current position.
+ #
+ # 0 lo string_size hi
+ # | |<---used--->|<----------available----------->|
+ # | | <--to pad-->|<---to write---> |
+ # 0 buf position
+ for i in range(self.string_size, self.pos):
+ self.buf[i] = '\0'
+
+ # Copy the data to the internal buffer, overwriting some of the
+ # existing data if self->pos < self->string_size.
+ for i in range(length):
+ self.buf[self.pos + i] = buf.getitem(i)
+ self.pos += length
+
+ # Set the new length of the internal string if it has changed
+ if self.string_size < self.pos:
+ self.string_size = self.pos
+
+ return space.wrap(length)
+
+ @unwrap_spec('self', ObjSpace)
+ def getvalue_w(self, space):
+ self._check_closed(space)
+ return space.wrap(buffer2string(self.buf, 0, self.string_size))
+
+
W_BytesIO.typedef = TypeDef(
'BytesIO', W_BufferedIOBase.typedef,
__new__ = generic_new_descr(W_BytesIO),
+ __init__ = interp2app(W_BytesIO.descr_init),
+
+ read = interp2app(W_BytesIO.read_w),
+ write = interp2app(W_BytesIO.write_w),
+ getvalue = interp2app(W_BytesIO.getvalue_w),
)
Added: pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py
==============================================================================
--- (empty file)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py Wed Oct 27 19:54:43 2010
@@ -0,0 +1,25 @@
+from pypy.conftest import gettestobjspace
+
+class AppTestBytesIO:
+ def setup_class(cls):
+ cls.space = gettestobjspace(usemodules=['_io'])
+
+ def test_write(self):
+ import _io
+ f = _io.BytesIO()
+ assert f.write("hello") == 5
+ import gc; gc.collect()
+ assert f.getvalue() == "hello"
+ f.close()
+
+ def test_read(self):
+ import _io
+ f = _io.BytesIO("hello")
+ assert f.read() == "hello"
+ import gc; gc.collect()
+ f.close()
+
+ ## def test_seek(self):
+ ## import _io
+ ## f = _io.BytesIO("hello")
+ ## assert f.seek(-1, 2) == 4
From afa at codespeak.net Wed Oct 27 19:56:19 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 27 Oct 2010 19:56:19 +0200 (CEST)
Subject: [pypy-svn] r78376 - pypy/branch/fast-forward/pypy/translator/c/src
Message-ID: <20101027175619.9DA945080C@codespeak.net>
Author: afa
Date: Wed Oct 27 19:56:18 2010
New Revision: 78376
Modified:
pypy/branch/fast-forward/pypy/translator/c/src/debug_alloc.h
Log:
Fix compilation on Windows, when debug_alloc is enabled
Modified: pypy/branch/fast-forward/pypy/translator/c/src/debug_alloc.h
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/c/src/debug_alloc.h (original)
+++ pypy/branch/fast-forward/pypy/translator/c/src/debug_alloc.h Wed Oct 27 19:56:18 2010
@@ -62,8 +62,8 @@
count++;
if (count > 0)
{
- fprintf(stderr, "debug_alloc.h: %ld mallocs left", count);
char *env = getenv("PYPY_ALLOC");
+ fprintf(stderr, "debug_alloc.h: %ld mallocs left", count);
if (env && *env)
{
fprintf(stderr, " (most recent first):\n");
From arigo at codespeak.net Wed Oct 27 19:57:18 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Wed, 27 Oct 2010 19:57:18 +0200 (CEST)
Subject: [pypy-svn] r78377 - pypy/trunk/pypy/objspace/std
Message-ID: <20101027175718.7EE4E282BE0@codespeak.net>
Author: arigo
Date: Wed Oct 27 19:57:16 2010
New Revision: 78377
Modified:
pypy/trunk/pypy/objspace/std/listobject.py
Log:
Document this assert (thanks amaury).
Modified: pypy/trunk/pypy/objspace/std/listobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/listobject.py (original)
+++ pypy/trunk/pypy/objspace/std/listobject.py Wed Oct 27 19:57:16 2010
@@ -283,7 +283,7 @@
elif start >= 0:
del items[start:start+delta]
else:
- assert delta==0
+ assert delta==0 # start<0 is only possible with slicelength==0
elif len2 != slicelength: # No resize for extended slices
raise operationerrfmt(space.w_ValueError, "attempt to "
"assign sequence of size %d to extended slice of size %d",
From afa at codespeak.net Wed Oct 27 23:20:51 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Wed, 27 Oct 2010 23:20:51 +0200 (CEST)
Subject: [pypy-svn] r78380 -
pypy/branch/fast-forward/pypy/translator/goal/test2
Message-ID: <20101027212051.6E52A5080C@codespeak.net>
Author: afa
Date: Wed Oct 27 23:20:49 2010
New Revision: 78380
Modified:
pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py
Log:
Fix test when PYTHONPATH is not set
Modified: pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py (original)
+++ pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py Wed Oct 27 23:20:49 2010
@@ -489,7 +489,8 @@
yield
finally:
old_cwd.chdir()
- os.putenv('PYTHONPATH', old_pythonpath)
+ if old_pythonpath is not None:
+ os.putenv('PYTHONPATH', old_pythonpath)
tmpdir.join('site.py').write('print "SHOULD NOT RUN"')
runme_py = tmpdir.join('runme.py')
From afa at codespeak.net Thu Oct 28 00:10:10 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 00:10:10 +0200 (CEST)
Subject: [pypy-svn] r78381 - in
pypy/branch/fast-forward/pypy/translator/goal: . test2
Message-ID: <20101027221010.AD52B36C1FD@codespeak.net>
Author: afa
Date: Thu Oct 28 00:10:04 2010
New Revision: 78381
Modified:
pypy/branch/fast-forward/pypy/translator/goal/app_main.py
pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py
Log:
The command::
python -i -c "import sys; sys.exit(1)"
- must display the SystemExit traceback
- must enter interactive mode
Modified: pypy/branch/fast-forward/pypy/translator/goal/app_main.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/goal/app_main.py (original)
+++ pypy/branch/fast-forward/pypy/translator/goal/app_main.py Thu Oct 28 00:10:04 2010
@@ -21,6 +21,25 @@
originalexcepthook = sys.__excepthook__
+def handle_sys_exit(e):
+ # exit if we catch a w_SystemExit
+ exitcode = e.code
+ if exitcode is None:
+ exitcode = 0
+ else:
+ try:
+ exitcode = int(exitcode)
+ except:
+ # not an integer: print it to stderr
+ try:
+ stderr = sys.stderr
+ except AttributeError:
+ pass # too bad
+ else:
+ print >> stderr, exitcode
+ exitcode = 1
+ raise SystemExit(exitcode)
+
def run_toplevel(f, *fargs, **fkwds):
"""Calls f() and handles all OperationErrors.
Intended use is to run the main program or one interactive statement.
@@ -44,67 +63,53 @@
stdout.write('\n')
except SystemExit, e:
- # exit if we catch a w_SystemExit
- exitcode = e.code
- if exitcode is None:
- exitcode = 0
- else:
- try:
- exitcode = int(exitcode)
- except:
- # not an integer: print it to stderr
- try:
- stderr = sys.stderr
- except AttributeError:
- pass # too bad
- else:
- print >> stderr, exitcode
- exitcode = 1
- raise SystemExit(exitcode)
+ handle_sys_exit(e)
+ except:
+ display_exception()
+ return False
+ return True # success
+
+def display_exception():
+ etype, evalue, etraceback = sys.exc_info()
+ try:
+ # extra debugging info in case the code below goes very wrong
+ if DEBUG and hasattr(sys, 'stderr'):
+ s = getattr(etype, '__name__', repr(etype))
+ print >> sys.stderr, "debug: exception-type: ", s
+ print >> sys.stderr, "debug: exception-value:", str(evalue)
+ tbentry = etraceback
+ if tbentry:
+ while tbentry.tb_next:
+ tbentry = tbentry.tb_next
+ lineno = tbentry.tb_lineno
+ filename = tbentry.tb_frame.f_code.co_filename
+ print >> sys.stderr, "debug: exception-tb: %s:%d" % (
+ filename, lineno)
+
+ # set the sys.last_xxx attributes
+ sys.last_type = etype
+ sys.last_value = evalue
+ sys.last_traceback = etraceback
+
+ # call sys.excepthook
+ hook = getattr(sys, 'excepthook', originalexcepthook)
+ hook(etype, evalue, etraceback)
+ return # done
except:
- etype, evalue, etraceback = sys.exc_info()
try:
- # extra debugging info in case the code below goes very wrong
- if DEBUG and hasattr(sys, 'stderr'):
- s = getattr(etype, '__name__', repr(etype))
- print >> sys.stderr, "debug: exception-type: ", s
- print >> sys.stderr, "debug: exception-value:", str(evalue)
- tbentry = etraceback
- if tbentry:
- while tbentry.tb_next:
- tbentry = tbentry.tb_next
- lineno = tbentry.tb_lineno
- filename = tbentry.tb_frame.f_code.co_filename
- print >> sys.stderr, "debug: exception-tb: %s:%d" % (
- filename, lineno)
-
- # set the sys.last_xxx attributes
- sys.last_type = etype
- sys.last_value = evalue
- sys.last_traceback = etraceback
-
- # call sys.excepthook
- hook = getattr(sys, 'excepthook', originalexcepthook)
- hook(etype, evalue, etraceback)
- return False # done
-
- except:
- try:
- stderr = sys.stderr
- except AttributeError:
- pass # too bad
- else:
- print >> stderr, 'Error calling sys.excepthook:'
- originalexcepthook(*sys.exc_info())
- print >> stderr
- print >> stderr, 'Original exception was:'
+ stderr = sys.stderr
+ except AttributeError:
+ pass # too bad
+ else:
+ print >> stderr, 'Error calling sys.excepthook:'
+ originalexcepthook(*sys.exc_info())
+ print >> stderr
+ print >> stderr, 'Original exception was:'
- # we only get here if sys.excepthook didn't do its job
- originalexcepthook(etype, evalue, etraceback)
- return False
+ # we only get here if sys.excepthook didn't do its job
+ originalexcepthook(etype, evalue, etraceback)
- return True # success
# ____________________________________________________________
# Option parsing
@@ -481,14 +486,25 @@
sys.path.insert(0, scriptdir)
success = run_toplevel(execfile, sys.argv[0], mainmodule.__dict__)
- # start a prompt if requested
+ except SystemExit, e:
+ status = e.code
if inspect_requested():
+ display_exception()
+ else:
+ status = not success
+
+ # start a prompt if requested
+ if inspect_requested():
+ inteactive = False
+ try:
from _pypy_interact import interactive_console
success = run_toplevel(interactive_console, mainmodule)
- except SystemExit, e:
- return e.code
- else:
- return not success
+ except SystemExit, e:
+ status = e.code
+ else:
+ status = not success
+
+ return status
def resolvedirof(filename):
try:
Modified: pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py
==============================================================================
--- pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py (original)
+++ pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py Thu Oct 28 00:10:04 2010
@@ -241,6 +241,11 @@
child.sendline('"pypy.translator.goal.test2.mymodule" in sys.modules')
child.expect('False')
+ def test_option_i_noexit(self):
+ child = self.spawn(['-i', '-c', 'import sys; sys.exit(1)'])
+ child.expect('Traceback')
+ child.expect('SystemExit: 1')
+
def test_options_u_i(self):
if sys.platform == "win32":
skip("close_fds is not supported on Windows platforms")
From afa at codespeak.net Thu Oct 28 01:54:14 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 01:54:14 +0200 (CEST)
Subject: [pypy-svn] r78382 - in pypy/branch/fast-forward/pypy/module/rctime:
. test
Message-ID: <20101027235414.4C310282BE0@codespeak.net>
Author: afa
Date: Thu Oct 28 01:54:11 2010
New Revision: 78382
Modified:
pypy/branch/fast-forward/pypy/module/rctime/app_time.py
pypy/branch/fast-forward/pypy/module/rctime/test/test_rctime.py
Log:
Ensure that time.strptime() returns a time structure
(the _strptime module changed in 2.7 to return an additional value)
Modified: pypy/branch/fast-forward/pypy/module/rctime/app_time.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/rctime/app_time.py (original)
+++ pypy/branch/fast-forward/pypy/module/rctime/app_time.py Thu Oct 28 01:54:11 2010
@@ -25,7 +25,7 @@
(same as strftime())."""
import _strptime # from the CPython standard library
- return _strptime._strptime(string, format)
+ return _strptime._strptime(string, format)[0]
__doc__ = """This module provides various functions to manipulate time values.
Modified: pypy/branch/fast-forward/pypy/module/rctime/test/test_rctime.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/rctime/test/test_rctime.py (original)
+++ pypy/branch/fast-forward/pypy/module/rctime/test/test_rctime.py Thu Oct 28 01:54:11 2010
@@ -270,9 +270,11 @@
def test_strptime(self):
import time as rctime
-
+
t = rctime.time()
tt = rctime.gmtime(t)
+ assert isinstance(rctime.strptime("", ""), type(tt))
+
for directive in ('a', 'A', 'b', 'B', 'c', 'd', 'H', 'I',
'j', 'm', 'M', 'p', 'S',
'U', 'w', 'W', 'x', 'X', 'y', 'Y', 'Z', '%'):
From afa at codespeak.net Thu Oct 28 02:02:54 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 02:02:54 +0200 (CEST)
Subject: [pypy-svn] r78383 - pypy/branch/fast-forward/lib_pypy
Message-ID: <20101028000254.94026282BE0@codespeak.net>
Author: afa
Date: Thu Oct 28 02:02:52 2010
New Revision: 78383
Modified:
pypy/branch/fast-forward/lib_pypy/datetime.py
Log:
Update datetime module to use the raw _strptime function
Modified: pypy/branch/fast-forward/lib_pypy/datetime.py
==============================================================================
--- pypy/branch/fast-forward/lib_pypy/datetime.py (original)
+++ pypy/branch/fast-forward/lib_pypy/datetime.py Thu Oct 28 02:02:52 2010
@@ -1612,10 +1612,11 @@
@classmethod
def strptime(cls, date_string, format):
'string, format -> new datetime parsed from a string (like time.strptime()).'
+ from _strptime import _strptime
# _strptime._strptime returns a two-element tuple. The first
# element is a time.struct_time object. The second is the
# microseconds (which are not defined for time.struct_time).
- struct, micros = _time.strptime(date_string, format)
+ struct, micros = _strptime(date_string, format)
return cls(*(struct[0:6] + (micros,)))
def utcoffset(self):
From afa at codespeak.net Thu Oct 28 09:00:29 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 09:00:29 +0200 (CEST)
Subject: [pypy-svn] r78384 - in pypy/branch/fast-forward:
lib-python/modified-2.7.0 pypy/interpreter pypy/interpreter/test
Message-ID: <20101028070029.EB507282BE0@codespeak.net>
Author: afa
Date: Thu Oct 28 09:00:27 2010
New Revision: 78384
Modified:
pypy/branch/fast-forward/lib-python/modified-2.7.0/inspect.py
pypy/branch/fast-forward/pypy/interpreter/argument.py
pypy/branch/fast-forward/pypy/interpreter/test/test_argument.py
Log:
CPython removed the "non-keyword" fragment from error messages.
Also fix a typo in inspect.py
Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/inspect.py
==============================================================================
--- pypy/branch/fast-forward/lib-python/modified-2.7.0/inspect.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/inspect.py Thu Oct 28 09:00:27 2010
@@ -952,7 +952,7 @@
f_name, 'at most' if defaults else 'exactly', num_args,
'arguments' if num_args > 1 else 'argument', num_total))
elif num_args == 0 and num_total:
- raise TypeError('%s() takes no arguments (%d given)' %
+ raise TypeError('%s() takes no argument (%d given)' %
(f_name, num_total))
for arg in args:
if isinstance(arg, str) and arg in named:
Modified: pypy/branch/fast-forward/pypy/interpreter/argument.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/argument.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/argument.py Thu Oct 28 09:00:27 2010
@@ -574,20 +574,17 @@
args = None
#args_w, kwds_w = args.unpack()
if self.has_kwarg or (self.num_kwds and self.num_defaults):
- msg2 = "non-keyword "
if self.missing_args:
required_args = self.expected_nargs - self.num_defaults
nargs = required_args - self.missing_args
else:
nargs = self.num_args
else:
- msg2 = ""
nargs = self.num_args + self.num_kwds
n = self.expected_nargs
if n == 0:
- msg = "%s() takes no %sargument (%d given)" % (
+ msg = "%s() takes no argument (%d given)" % (
fnname,
- msg2,
nargs)
else:
defcount = self.num_defaults
@@ -598,17 +595,14 @@
else:
msg1 = "at least"
n -= defcount
- if not self.num_kwds: # msg "f() takes at least X non-keyword args"
- msg2 = "" # is confusing if no kwd arg actually provided
if n == 1:
plural = ""
else:
plural = "s"
- msg = "%s() takes %s %d %sargument%s (%d given)" % (
+ msg = "%s() takes %s %d argument%s (%d given)" % (
fnname,
msg1,
n,
- msg2,
plural,
nargs)
return msg
Modified: pypy/branch/fast-forward/pypy/interpreter/test/test_argument.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/test/test_argument.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/test/test_argument.py Thu Oct 28 09:00:27 2010
@@ -486,10 +486,10 @@
assert s == "foo() takes at most 2 arguments (3 given)"
err = ArgErrCount(0, 1, 2, True, False, ['a'], 1)
s = err.getmsg('foo')
- assert s == "foo() takes at least 1 non-keyword argument (0 given)"
+ assert s == "foo() takes at least 1 argument (0 given)"
err = ArgErrCount(2, 1, 1, False, True, [], 0)
s = err.getmsg('foo')
- assert s == "foo() takes exactly 1 non-keyword argument (2 given)"
+ assert s == "foo() takes exactly 1 argument (2 given)"
def test_unknown_keywords(self):
From afa at codespeak.net Thu Oct 28 10:19:35 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 10:19:35 +0200 (CEST)
Subject: [pypy-svn] r78385 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101028081935.5D387282BE0@codespeak.net>
Author: afa
Date: Thu Oct 28 10:19:33 2010
New Revision: 78385
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
Log:
file.read(n) may return a shorter string at end of file.
This also fixes readlines() when there is no \n at the end.
Too much code, and not enough tests :-(
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Thu Oct 28 10:19:33 2010
@@ -371,12 +371,14 @@
self.read_end = 0
while remaining > 0 and self.read_end < self.buffer_size:
- # Read until EOF or until read() would block
try:
size = self._fill_buffer(space)
except BlockingIOError:
+ # EOF or read() would block
if written == 0:
return None
+ size = 0
+ if size == 0:
break
if remaining > 0:
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Thu Oct 28 10:19:33 2010
@@ -22,6 +22,16 @@
assert r == "a\nb\n"
f.close()
+ def test_read_pieces(self):
+ import _io
+ raw = _io.FileIO(self.tmpfile)
+ f = _io.BufferedReader(raw)
+ assert f.read(3) == "a\nb"
+ assert f.read(3) == "\nc"
+ assert f.read(3) == ""
+ assert f.read(3) == ""
+ f.close()
+
def test_seek(self):
import _io
raw = _io.FileIO(self.tmpfile)
@@ -33,6 +43,12 @@
assert f.read() == "\nc"
f.close()
+ def test_readlines(self):
+ import _io
+ raw = _io.FileIO(self.tmpfile)
+ f = _io.BufferedReader(raw)
+ assert f.readlines() == ['a\n', 'b\n', 'c']
+
class AppTestBufferedWriter:
def setup_class(cls):
cls.space = gettestobjspace(usemodules=['_io'])
From arigo at codespeak.net Thu Oct 28 10:42:07 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 28 Oct 2010 10:42:07 +0200 (CEST)
Subject: [pypy-svn] r78386 - in
pypy/branch/bisect-module/pypy/module/_bisect: . test
Message-ID: <20101028084207.1136B282BE0@codespeak.net>
Author: arigo
Date: Thu Oct 28 10:42:05 2010
New Revision: 78386
Added:
pypy/branch/bisect-module/pypy/module/_bisect/ (props changed)
pypy/branch/bisect-module/pypy/module/_bisect/__init__.py (contents, props changed)
pypy/branch/bisect-module/pypy/module/_bisect/interp_bisect.py (contents, props changed)
pypy/branch/bisect-module/pypy/module/_bisect/test/ (props changed)
pypy/branch/bisect-module/pypy/module/_bisect/test/__init__.py (contents, props changed)
pypy/branch/bisect-module/pypy/module/_bisect/test/test_bisect.py (contents, props changed)
Log:
bisect_left, bisect_right
Added: pypy/branch/bisect-module/pypy/module/_bisect/__init__.py
==============================================================================
--- (empty file)
+++ pypy/branch/bisect-module/pypy/module/_bisect/__init__.py Thu Oct 28 10:42:05 2010
@@ -0,0 +1,27 @@
+"""
+Mixed-module definition for the _bisect module.
+This is an optional module; if not present, bisect.py uses the
+pure Python version of these functions.
+"""
+
+from pypy.interpreter.mixedmodule import MixedModule
+
+
+class Module(MixedModule):
+ """\
+This module provides support for maintaining a list in sorted order without
+having to sort the list after each insertion. For long lists of items with
+expensive comparison operations, this can be an improvement over the more
+common approach."""
+
+ appleveldefs = {
+ }
+
+ interpleveldefs = {
+ 'bisect': 'interp_bisect.bisect_right',
+ 'bisect_left': 'interp_bisect.bisect_left',
+ 'bisect_right': 'interp_bisect.bisect_right',
+ 'insort': 'interp_bisect.insort_right',
+ 'insort_left': 'interp_bisect.insort_left',
+ 'insort_right': 'interp_bisect.insort_right',
+ }
Added: pypy/branch/bisect-module/pypy/module/_bisect/interp_bisect.py
==============================================================================
--- (empty file)
+++ pypy/branch/bisect-module/pypy/module/_bisect/interp_bisect.py Thu Oct 28 10:42:05 2010
@@ -0,0 +1,45 @@
+from pypy.interpreter.gateway import ObjSpace, W_Root
+
+
+def bisect_left(space, w_a, w_x, lo=0, hi=-1):
+ """Return the index where to insert item x in list a, assuming a is sorted.
+
+The return value i is such that all e in a[:i] have e < x, and all e in
+a[i:] have e >= x. So if x already appears in the list, i points just
+before the leftmost x already there.
+
+Optional args lo (default 0) and hi (default len(a)) bound the
+slice of a to be searched."""
+ if hi == -1:
+ hi = space.int_w(space.len(w_a))
+ while lo < hi:
+ mid = (lo + hi) >> 1
+ w_litem = space.getitem(w_a, space.wrap(mid))
+ if space.is_true(space.lt(w_litem, w_x)):
+ lo = mid + 1
+ else:
+ hi = mid
+ return space.wrap(lo)
+bisect_left.unwrap_spec = [ObjSpace, W_Root, W_Root, int, int]
+
+
+def bisect_right(space, w_a, w_x, lo=0, hi=-1):
+ """Return the index where to insert item x in list a, assuming a is sorted.
+
+The return value i is such that all e in a[:i] have e <= x, and all e in
+a[i:] have e > x. So if x already appears in the list, i points just
+beyond the rightmost x already there
+
+Optional args lo (default 0) and hi (default len(a)) bound the
+slice of a to be searched."""
+ if hi == -1:
+ hi = space.int_w(space.len(w_a))
+ while lo < hi:
+ mid = (lo + hi) >> 1
+ w_litem = space.getitem(w_a, space.wrap(mid))
+ if space.is_true(space.lt(w_x, w_litem)):
+ hi = mid
+ else:
+ lo = mid + 1
+ return space.wrap(lo)
+bisect_right.unwrap_spec = [ObjSpace, W_Root, W_Root, int, int]
Added: pypy/branch/bisect-module/pypy/module/_bisect/test/__init__.py
==============================================================================
Added: pypy/branch/bisect-module/pypy/module/_bisect/test/test_bisect.py
==============================================================================
--- (empty file)
+++ pypy/branch/bisect-module/pypy/module/_bisect/test/test_bisect.py Thu Oct 28 10:42:05 2010
@@ -0,0 +1,79 @@
+from pypy.conftest import gettestobjspace
+
+
+class AppTestBisect:
+
+ def setup_class(cls):
+ cls.space = gettestobjspace(usemodules=['_bisect'])
+
+ def test_bisect_left(self):
+ from _bisect import bisect_left
+ a = [0, 5, 6, 6, 6, 7]
+ assert bisect_left(a, None) == 0
+ assert bisect_left(a, -3) == 0
+ assert bisect_left(a, 0) == 0
+ assert bisect_left(a, 3) == 1
+ assert bisect_left(a, 5) == 1
+ assert bisect_left(a, 5.5) == 2
+ assert bisect_left(a, 6) == 2
+ assert bisect_left(a, 6.0) == 2
+ assert bisect_left(a, 6.1) == 5
+ assert bisect_left(a, 7) == 5
+ assert bisect_left(a, 8) == 6
+ a = []
+ assert bisect_left(a, 123) == 0
+ a = [9]
+ assert bisect_left(a, -123) == 0
+ assert bisect_left(a, 9) == 0
+ assert bisect_left(a, 123) == 1
+ a = [9, 9]
+ assert bisect_left(a, -123) == 0
+ assert bisect_left(a, 9) == 0
+ assert bisect_left(a, 123) == 2
+ a = [4, 6, 6, 9]
+ assert bisect_left(a, 6, 0) == 1
+ assert bisect_left(a, 6, 1) == 1
+ assert bisect_left(a, 6, 2) == 2
+ assert bisect_left(a, 6, 3) == 3
+ assert bisect_left(a, 6, 4) == 4
+ assert bisect_left(a, 6, 0, 0) == 0
+ assert bisect_left(a, 6, 0, 1) == 1
+ assert bisect_left(a, 6, 0, 2) == 1
+ assert bisect_left(a, 6, 0, 3) == 1
+ assert bisect_left(a, 6, 0, 4) == 1
+
+ def test_bisect_right(self):
+ from _bisect import bisect_right
+ a = [0, 5, 6, 6, 6, 7]
+ assert bisect_right(a, None) == 0
+ assert bisect_right(a, -3) == 0
+ assert bisect_right(a, 0) == 1
+ assert bisect_right(a, 3) == 1
+ assert bisect_right(a, 5) == 2
+ assert bisect_right(a, 5.5) == 2
+ assert bisect_right(a, 6) == 5
+ assert bisect_right(a, 6.0) == 5
+ assert bisect_right(a, 6.1) == 5
+ assert bisect_right(a, 7) == 6
+ assert bisect_right(a, 8) == 6
+ a = []
+ assert bisect_right(a, 123) == 0
+ a = [9]
+ assert bisect_right(a, -123) == 0
+ assert bisect_right(a, 9) == 1
+ assert bisect_right(a, 123) == 1
+ a = [9, 9]
+ assert bisect_right(a, -123) == 0
+ assert bisect_right(a, 9) == 2
+ assert bisect_right(a, 123) == 2
+ a = [4, 6, 6, 9]
+ assert bisect_right(a, 6, 0) == 3
+ assert bisect_right(a, 6, 1) == 3
+ assert bisect_right(a, 6, 2) == 3
+ assert bisect_right(a, 6, 3) == 3
+ assert bisect_right(a, 6, 4) == 4
+ assert bisect_right(a, 6, 0, 0) == 0
+ assert bisect_right(a, 6, 0, 1) == 1
+ assert bisect_right(a, 6, 0, 2) == 2
+ assert bisect_right(a, 6, 0, 3) == 3
+ assert bisect_right(a, 6, 0, 4) == 3
From arigo at codespeak.net Thu Oct 28 10:48:42 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 28 Oct 2010 10:48:42 +0200 (CEST)
Subject: [pypy-svn] r78387 - in
pypy/branch/bisect-module/pypy/module/_bisect: . test
Message-ID: <20101028084842.9D4745080C@codespeak.net>
Author: arigo
Date: Thu Oct 28 10:48:41 2010
New Revision: 78387
Added:
pypy/branch/bisect-module/pypy/module/_bisect/app_bisect.py
Modified:
pypy/branch/bisect-module/pypy/module/_bisect/__init__.py
pypy/branch/bisect-module/pypy/module/_bisect/test/test_bisect.py
Log:
Insort.
Modified: pypy/branch/bisect-module/pypy/module/_bisect/__init__.py
==============================================================================
--- pypy/branch/bisect-module/pypy/module/_bisect/__init__.py (original)
+++ pypy/branch/bisect-module/pypy/module/_bisect/__init__.py Thu Oct 28 10:48:41 2010
@@ -15,13 +15,13 @@
common approach."""
appleveldefs = {
+ 'insort': 'app_bisect.insort_right',
+ 'insort_left': 'app_bisect.insort_left',
+ 'insort_right': 'app_bisect.insort_right',
}
interpleveldefs = {
'bisect': 'interp_bisect.bisect_right',
'bisect_left': 'interp_bisect.bisect_left',
'bisect_right': 'interp_bisect.bisect_right',
- 'insort': 'interp_bisect.insort_right',
- 'insort_left': 'interp_bisect.insort_left',
- 'insort_right': 'interp_bisect.insort_right',
}
Added: pypy/branch/bisect-module/pypy/module/_bisect/app_bisect.py
==============================================================================
--- (empty file)
+++ pypy/branch/bisect-module/pypy/module/_bisect/app_bisect.py Thu Oct 28 10:48:41 2010
@@ -0,0 +1,21 @@
+from _bisect import bisect_left, bisect_right
+
+
+def insort_left(a, x, lo=0, hi=-1):
+ """Insert item x in list a, and keep it sorted assuming a is sorted.
+
+If x is already in a, insert it to the left of the leftmost x.
+
+Optional args lo (default 0) and hi (default len(a)) bound the
+slice of a to be searched."""
+ a.insert(bisect_left(a, x, lo, hi), x)
+
+
+def insort_right(a, x, lo=0, hi=-1):
+ """Insert item x in list a, and keep it sorted assuming a is sorted.
+
+If x is already in a, insert it to the right of the rightmost x.
+
+Optional args lo (default 0) and hi (default len(a)) bound the
+slice of a to be searched."""
+ a.insert(bisect_right(a, x, lo, hi), x)
Modified: pypy/branch/bisect-module/pypy/module/_bisect/test/test_bisect.py
==============================================================================
--- pypy/branch/bisect-module/pypy/module/_bisect/test/test_bisect.py (original)
+++ pypy/branch/bisect-module/pypy/module/_bisect/test/test_bisect.py Thu Oct 28 10:48:41 2010
@@ -77,3 +77,17 @@
assert bisect_right(a, 6, 0, 2) == 2
assert bisect_right(a, 6, 0, 3) == 3
assert bisect_right(a, 6, 0, 4) == 3
+
+ def test_insort_left(self):
+ from _bisect import insort_left
+ a = [0, 5, 6, 6, 6, 7]
+ insort_left(a, 6.0)
+ assert a == [0, 5, 6.0, 6, 6, 6, 7]
+ assert map(type, a) == [int, int, float, int, int, int, int]
+
+ def test_insort_right(self):
+ from _bisect import insort_right
+ a = [0, 5, 6, 6, 6, 7]
+ insort_right(a, 6.0)
+ assert a == [0, 5, 6, 6, 6, 6.0, 7]
+ assert map(type, a) == [int, int, int, int, int, float, int]
From arigo at codespeak.net Thu Oct 28 10:49:23 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 28 Oct 2010 10:49:23 +0200 (CEST)
Subject: [pypy-svn] r78388 - pypy/branch/bisect-module/pypy/config
Message-ID: <20101028084923.29058282BE0@codespeak.net>
Author: arigo
Date: Thu Oct 28 10:49:21 2010
New Revision: 78388
Modified:
pypy/branch/bisect-module/pypy/config/pypyoption.py
Log:
Enable this small module.
Modified: pypy/branch/bisect-module/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/bisect-module/pypy/config/pypyoption.py (original)
+++ pypy/branch/bisect-module/pypy/config/pypyoption.py Thu Oct 28 10:49:21 2010
@@ -30,7 +30,8 @@
"rctime" , "select", "zipimport", "_lsprof",
"crypt", "signal", "_rawffi", "termios", "zlib",
"struct", "md5", "sha", "bz2", "_minimal_curses", "cStringIO",
- "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array"]
+ "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array",
+ "_bisect"]
))
working_oo_modules = default_modules.copy()
From afa at codespeak.net Thu Oct 28 11:31:13 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 11:31:13 +0200 (CEST)
Subject: [pypy-svn] r78389 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101028093113.8DCD85080C@codespeak.net>
Author: afa
Date: Thu Oct 28 11:31:11 2010
New Revision: 78389
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
Log:
Implement .fileno() and .detach()
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Thu Oct 28 11:31:11 2010
@@ -30,11 +30,16 @@
def write_w(self, space, w_data):
self._unsupportedoperation(space, "write")
+ @unwrap_spec('self', ObjSpace)
+ def detach_w(self, space):
+ self._unsupportedoperation(space, "detach")
+
W_BufferedIOBase.typedef = TypeDef(
'_BufferedIOBase', W_IOBase.typedef,
__new__ = generic_new_descr(W_BufferedIOBase),
read = interp2app(W_BufferedIOBase.read_w),
write = interp2app(W_BufferedIOBase.write_w),
+ detach = interp2app(W_BufferedIOBase.detach_w),
)
class BufferedMixin:
@@ -127,6 +132,7 @@
@unwrap_spec('self', ObjSpace, r_longlong, int)
def seek_w(self, space, pos, whence=0):
+ self._check_init(space)
if whence not in (0, 1, 2):
raise operationerrfmt(space.w_ValueError,
"whence must be between 0 and 2, not %d", whence)
@@ -244,6 +250,20 @@
l.append(self.buffer[i])
return self._write(space, ''.join(l))
+ @unwrap_spec('self', ObjSpace)
+ def detach_w(self, space):
+ self._check_init(space)
+ space.call_method(self, "flush")
+ raw = self.raw
+ self.raw = None
+ self.state = STATE_DETACHED
+ return space.wrap(raw)
+
+ @unwrap_spec('self', ObjSpace)
+ def fileno_w(self, space):
+ self._check_init(space)
+ return space.call_method(self.raw, "fileno")
+
class W_BufferedReader(BufferedMixin, W_BufferedIOBase):
@unwrap_spec('self', ObjSpace, W_Root, int)
def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE):
@@ -261,6 +281,7 @@
@unwrap_spec('self', ObjSpace, W_Root)
def read_w(self, space, w_size=None):
+ self._check_init(space)
self._check_closed(space, "read of closed file")
size = convert_size(space, w_size)
@@ -417,6 +438,8 @@
seek = interp2app(W_BufferedReader.seek_w),
close = interp2app(W_BufferedReader.close_w),
flush = interp2app(W_BufferedReader.flush_w),
+ detach = interp2app(W_BufferedReader.detach_w),
+ fileno = interp2app(W_BufferedReader.fileno_w),
closed = GetSetProperty(W_BufferedReader.closed_get_w),
)
@@ -442,6 +465,7 @@
@unwrap_spec('self', ObjSpace, W_Root)
def write_w(self, space, w_data):
+ self._check_init(space)
self._check_closed(space, "write to closed file")
data = space.str_w(w_data)
size = len(data)
@@ -548,6 +572,7 @@
@unwrap_spec('self', ObjSpace)
def flush_w(self, space):
+ self._check_init(space)
self._check_closed(space, "flush of closed file")
with self.lock:
self._writer_flush_unlocked(space)
@@ -568,6 +593,8 @@
# from the mixin class
seek = interp2app(W_BufferedWriter.seek_w),
close = interp2app(W_BufferedWriter.close_w),
+ fileno = interp2app(W_BufferedWriter.fileno_w),
+ detach = interp2app(W_BufferedWriter.detach_w),
closed = GetSetProperty(W_BufferedWriter.closed_get_w),
)
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Thu Oct 28 11:31:11 2010
@@ -49,6 +49,19 @@
f = _io.BufferedReader(raw)
assert f.readlines() == ['a\n', 'b\n', 'c']
+ def test_detach(self):
+ import _io
+ raw = _io.FileIO(self.tmpfile)
+ f = _io.BufferedReader(raw)
+ assert f.fileno() == raw.fileno()
+ assert f.detach() is raw
+ raises(ValueError, f.fileno)
+ raises(ValueError, f.close)
+ raises(ValueError, f.detach)
+ raises(ValueError, f.flush)
+ assert not raw.closed
+ raw.close()
+
class AppTestBufferedWriter:
def setup_class(cls):
cls.space = gettestobjspace(usemodules=['_io'])
From afa at codespeak.net Thu Oct 28 12:11:51 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 12:11:51 +0200 (CEST)
Subject: [pypy-svn] r78390 - in pypy/trunk/pypy/translator/c/gcc: . test
test/darwin64
Message-ID: <20101028101151.6B81A282C1E@codespeak.net>
Author: afa
Date: Thu Oct 28 12:11:49 2010
New Revision: 78390
Added:
pypy/trunk/pypy/translator/c/gcc/test/darwin64/
pypy/trunk/pypy/translator/c/gcc/test/darwin64/track0.s
Modified:
pypy/trunk/pypy/translator/c/gcc/test/test_trackgcroot.py
pypy/trunk/pypy/translator/c/gcc/trackgcroot.py
Log:
Add trackgcroot support for 64bit darwin
Added: pypy/trunk/pypy/translator/c/gcc/test/darwin64/track0.s
==============================================================================
--- (empty file)
+++ pypy/trunk/pypy/translator/c/gcc/test/darwin64/track0.s Thu Oct 28 12:11:49 2010
@@ -0,0 +1,159 @@
+_PyObject_Realloc:
+LFB118:
+ movq %rbx, -32(%rsp)
+LCFI86:
+ movq %rbp, -24(%rsp)
+LCFI87:
+ movq %r12, -16(%rsp)
+LCFI88:
+ movq %r13, -8(%rsp)
+LCFI89:
+ subq $40, %rsp
+LCFI90:
+ movq %rdi, %rbx
+ testq %rdi, %rdi
+ je L384
+ movq %rdi, %rbp
+ andq $-4096, %rbp
+ movl 32(%rbp), %edx
+ movl _narenas(%rip), %eax
+ cmpl %eax, %edx
+ jae L360
+ movq _arenas(%rip), %rax
+ mov %edx, %edx
+ movq %rdi, %rcx
+ subq (%rax,%rdx,8), %rcx
+ cmpq $262143, %rcx
+ ja L360
+ movl 36(%rbp), %ecx
+ incl %ecx
+ leal 0(,%rcx,8), %r12d
+ mov %r12d, %eax
+ cmpq %rax, %rsi
+ ja L363
+ leaq 0(,%rsi,4), %rdx
+ sall $4, %ecx
+ leal (%rcx,%r12), %eax
+ mov %eax, %eax
+ cmpq %rax, %rdx
+ ja L365
+ movl %esi, %r12d
+L363:
+ movq %rsi, %rdi
+ call _PyObject_Malloc
+ ;; expected {40(%rsp) | 8(%rsp), 24(%rsp), 32(%rsp), %r14, %r15, 16(%rsp) | }
+ testq %rax, %rax
+ je L367
+ mov %r12d, %edx
+ movq %rbx, %rsi
+ movq %rax, %rdi
+ call _memcpy
+ ;; expected {40(%rsp) | 8(%rsp), 24(%rsp), 32(%rsp), %r14, %r15, 16(%rsp) | }
+ movl 32(%rbp), %edx
+ movl _narenas(%rip), %eax
+ cmpl %eax, %edx
+ jae L369
+ movq _arenas(%rip), %rax
+ mov %edx, %edx
+ movq %rbx, %rcx
+ subq (%rax,%rdx,8), %rcx
+ cmpq $262143, %rcx
+ jbe L385
+L369:
+ movq %rbx, %rdi
+ call _free
+ ;; expected {40(%rsp) | 8(%rsp), 24(%rsp), 32(%rsp), %r14, %r15, 16(%rsp) | }
+ movq %r13, %rbx
+ jmp L365
+ .align 4,0x90
+L360:
+ testq %rsi, %rsi
+ jne L386
+ movl $1, %esi
+ movq %rbx, %rdi
+ call _realloc
+ ;; expected {40(%rsp) | 8(%rsp), 24(%rsp), 32(%rsp), %r14, %r15, 16(%rsp) | }
+ testq %rax, %rax
+ cmovne %rax, %rbx
+L365:
+ movq %rbx, %rax
+ movq 8(%rsp), %rbx
+ movq 16(%rsp), %rbp
+ movq 24(%rsp), %r12
+ movq 32(%rsp), %r13
+ addq $40, %rsp
+ ret
+ .align 4,0x90
+L386:
+ movq %rbx, %rdi
+ movq 8(%rsp), %rbx
+ movq 16(%rsp), %rbp
+ movq 24(%rsp), %r12
+ movq 32(%rsp), %r13
+ addq $40, %rsp
+ jmp _realloc
+L384:
+ movq %rsi, %rdi
+ movq 8(%rsp), %rbx
+ movq 16(%rsp), %rbp
+ movq 24(%rsp), %r12
+ movq 32(%rsp), %r13
+ addq $40, %rsp
+ jmp _PyObject_Malloc
+L367:
+ movq %r13, %rbx
+ jmp L365
+L385:
+ movl (%rbp), %esi
+ testl %esi, %esi
+ je L387
+ movq 8(%rbp), %rax
+ movq %rax, (%rbx)
+ movq %rbx, 8(%rbp)
+ testq %rax, %rax
+ je L374
+ movl (%rbp), %eax
+ decl %eax
+ movl %eax, (%rbp)
+ testl %eax, %eax
+ jne L367
+ movq 16(%rbp), %rdx
+ movq 24(%rbp), %rax
+ movq %rax, 24(%rdx)
+ movq %rdx, 16(%rax)
+ movq _freepools(%rip), %rax
+ movq %rax, 16(%rbp)
+ movq %rbp, _freepools(%rip)
+ movq %r13, %rbx
+ jmp L365
+L374:
+ movl (%rbp), %eax
+ decl %eax
+ movl %eax, (%rbp)
+ testl %eax, %eax
+ je L388
+ movl 36(%rbp), %eax
+ addl %eax, %eax
+ mov %eax, %eax
+ leaq _usedpools(%rip), %rdx
+ movq (%rdx,%rax,8), %rax
+ movq 24(%rax), %rdx
+ movq %rax, 16(%rbp)
+ movq %rdx, 24(%rbp)
+ movq %rbp, 24(%rax)
+ movq %rbp, 16(%rdx)
+ movq %r13, %rbx
+ jmp L365
+L387:
+ leaq LC6(%rip), %rcx
+ movl $744, %edx
+ leaq LC7(%rip), %rsi
+ leaq ___func__.207211(%rip), %rdi
+ call ___assert_rtn
+L388:
+ leaq LC6(%rip), %rcx
+ movl $783, %edx
+ leaq LC7(%rip), %rsi
+ leaq ___func__.207211(%rip), %rdi
+ call ___assert_rtn
+LFE118:
Modified: pypy/trunk/pypy/translator/c/gcc/test/test_trackgcroot.py
==============================================================================
--- pypy/trunk/pypy/translator/c/gcc/test/test_trackgcroot.py (original)
+++ pypy/trunk/pypy/translator/c/gcc/test/test_trackgcroot.py Thu Oct 28 12:11:49 2010
@@ -109,7 +109,7 @@
def test_computegcmaptable():
tests = []
- for format in ('elf', 'darwin', 'msvc', 'elf64'):
+ for format in ('elf', 'elf64', 'darwin', 'darwin64', 'msvc'):
for path in this_dir.join(format).listdir("track*.s"):
n = path.purebasename[5:]
try:
Modified: pypy/trunk/pypy/translator/c/gcc/trackgcroot.py
==============================================================================
--- pypy/trunk/pypy/translator/c/gcc/trackgcroot.py (original)
+++ pypy/trunk/pypy/translator/c/gcc/trackgcroot.py Thu Oct 28 12:11:49 2010
@@ -1090,7 +1090,7 @@
ElfFunctionGcRootTracker64.init_regexp()
-class DarwinFunctionGcRootTracker(ElfFunctionGcRootTracker32):
+class DarwinFunctionGcRootTracker32(ElfFunctionGcRootTracker32):
format = 'darwin'
function_names_prefix = '_'
@@ -1102,7 +1102,19 @@
funcname = '_' + match.group(1)
FunctionGcRootTracker32.__init__(self, funcname, lines, filetag)
-class Mingw32FunctionGcRootTracker(DarwinFunctionGcRootTracker):
+class DarwinFunctionGcRootTracker64(ElfFunctionGcRootTracker64):
+ format = 'darwin64'
+ function_names_prefix = '_'
+
+ r_functionstart = re.compile(r"_(\w+):\s*$")
+ OFFSET_LABELS = 0
+
+ def __init__(self, lines, filetag=0):
+ match = self.r_functionstart.match(lines[0])
+ funcname = '_' + match.group(1)
+ FunctionGcRootTracker64.__init__(self, funcname, lines, filetag)
+
+class Mingw32FunctionGcRootTracker(DarwinFunctionGcRootTracker32):
format = 'mingw32'
function_names_prefix = '_'
@@ -1373,7 +1385,7 @@
class DarwinAssemblerParser(AssemblerParser):
format = "darwin"
- FunctionGcRootTracker = DarwinFunctionGcRootTracker
+ FunctionGcRootTracker = DarwinFunctionGcRootTracker32
r_textstart = re.compile(r"\t.text\s*$")
@@ -1419,6 +1431,10 @@
return super(DarwinAssemblerParser, self).process_function(
lines, entrypoint, filename)
+class DarwinAssemblerParser64(DarwinAssemblerParser):
+ format = "darwin64"
+ FunctionGcRootTracker = DarwinFunctionGcRootTracker64
+
class Mingw32AssemblerParser(DarwinAssemblerParser):
format = "mingw32"
FunctionGcRootTracker = Mingw32FunctionGcRootTracker
@@ -1542,6 +1558,7 @@
'elf': ElfAssemblerParser,
'elf64': ElfAssemblerParser64,
'darwin': DarwinAssemblerParser,
+ 'darwin64': DarwinAssemblerParser64,
'mingw32': Mingw32AssemblerParser,
'msvc': MsvcAssemblerParser,
}
@@ -1885,7 +1902,10 @@
shuffle = False
output_raw_table = False
if sys.platform == 'darwin':
- format = 'darwin'
+ if sys.maxint > 2147483647:
+ format = 'darwin64'
+ else:
+ format = 'darwin'
elif sys.platform == 'win32':
format = 'mingw32'
else:
From arigo at codespeak.net Thu Oct 28 12:17:39 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 28 Oct 2010 12:17:39 +0200 (CEST)
Subject: [pypy-svn] r78391 - pypy/branch/bisect-module/pypy/module/_bisect
Message-ID: <20101028101739.1B6EC5080C@codespeak.net>
Author: arigo
Date: Thu Oct 28 12:17:38 2010
New Revision: 78391
Modified:
pypy/branch/bisect-module/pypy/module/_bisect/app_bisect.py
Log:
Recorder things that way. Should fix CPython's test_bisect.py.
Modified: pypy/branch/bisect-module/pypy/module/_bisect/app_bisect.py
==============================================================================
--- pypy/branch/bisect-module/pypy/module/_bisect/app_bisect.py (original)
+++ pypy/branch/bisect-module/pypy/module/_bisect/app_bisect.py Thu Oct 28 12:17:38 2010
@@ -8,7 +8,8 @@
Optional args lo (default 0) and hi (default len(a)) bound the
slice of a to be searched."""
- a.insert(bisect_left(a, x, lo, hi), x)
+ n = bisect_left(a, x, lo, hi)
+ a.insert(n, x)
def insort_right(a, x, lo=0, hi=-1):
@@ -18,4 +19,5 @@
Optional args lo (default 0) and hi (default len(a)) bound the
slice of a to be searched."""
- a.insert(bisect_right(a, x, lo, hi), x)
+ n = bisect_right(a, x, lo, hi)
+ a.insert(n, x)
From arigo at codespeak.net Thu Oct 28 12:17:47 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 28 Oct 2010 12:17:47 +0200 (CEST)
Subject: [pypy-svn] r78392 - pypy/branch/bisect-module/lib-python
Message-ID: <20101028101747.7DE20282C1E@codespeak.net>
Author: arigo
Date: Thu Oct 28 12:17:46 2010
New Revision: 78392
Modified:
pypy/branch/bisect-module/lib-python/conftest.py
Log:
Add a usemodules line.
Modified: pypy/branch/bisect-module/lib-python/conftest.py
==============================================================================
--- pypy/branch/bisect-module/lib-python/conftest.py (original)
+++ pypy/branch/bisect-module/lib-python/conftest.py Thu Oct 28 12:17:46 2010
@@ -144,7 +144,7 @@
RegrTest('test_binhex.py'),
RegrTest('test_binop.py', core=True),
- RegrTest('test_bisect.py', core=True),
+ RegrTest('test_bisect.py', core=True, usemodules='_bisect'),
RegrTest('test_bool.py', core=True),
RegrTest('test_bsddb.py', skip="unsupported extension module"),
RegrTest('test_bsddb185.py', skip="unsupported extension module"),
From arigo at codespeak.net Thu Oct 28 12:18:55 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 28 Oct 2010 12:18:55 +0200 (CEST)
Subject: [pypy-svn] r78393 - in pypy/trunk: lib-python pypy/config
pypy/module/_bisect pypy/module/_bisect/test
Message-ID: <20101028101855.4A9F6282C1E@codespeak.net>
Author: arigo
Date: Thu Oct 28 12:18:53 2010
New Revision: 78393
Added:
pypy/trunk/pypy/module/_bisect/ (props changed)
- copied from r78392, pypy/branch/bisect-module/pypy/module/_bisect/
Modified:
pypy/trunk/lib-python/conftest.py
pypy/trunk/pypy/config/pypyoption.py
pypy/trunk/pypy/module/_bisect/test/ (props changed)
Log:
Merge branch/bisect-module, writing the '_bisect' module like CPython.
Modified: pypy/trunk/lib-python/conftest.py
==============================================================================
--- pypy/trunk/lib-python/conftest.py (original)
+++ pypy/trunk/lib-python/conftest.py Thu Oct 28 12:18:53 2010
@@ -144,7 +144,7 @@
RegrTest('test_binhex.py'),
RegrTest('test_binop.py', core=True),
- RegrTest('test_bisect.py', core=True),
+ RegrTest('test_bisect.py', core=True, usemodules='_bisect'),
RegrTest('test_bool.py', core=True),
RegrTest('test_bsddb.py', skip="unsupported extension module"),
RegrTest('test_bsddb185.py', skip="unsupported extension module"),
Modified: pypy/trunk/pypy/config/pypyoption.py
==============================================================================
--- pypy/trunk/pypy/config/pypyoption.py (original)
+++ pypy/trunk/pypy/config/pypyoption.py Thu Oct 28 12:18:53 2010
@@ -30,7 +30,8 @@
"rctime" , "select", "zipimport", "_lsprof",
"crypt", "signal", "_rawffi", "termios", "zlib",
"struct", "md5", "sha", "bz2", "_minimal_curses", "cStringIO",
- "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array"]
+ "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array",
+ "_bisect"]
))
working_oo_modules = default_modules.copy()
From arigo at codespeak.net Thu Oct 28 12:19:19 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 28 Oct 2010 12:19:19 +0200 (CEST)
Subject: [pypy-svn] r78394 - pypy/branch/bisect-module
Message-ID: <20101028101919.1A1865080C@codespeak.net>
Author: arigo
Date: Thu Oct 28 12:19:17 2010
New Revision: 78394
Removed:
pypy/branch/bisect-module/
Log:
Remove merged branch.
From arigo at codespeak.net Thu Oct 28 12:39:15 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 28 Oct 2010 12:39:15 +0200 (CEST)
Subject: [pypy-svn] r78395 - pypy/trunk/pypy/module/pypyjit/test
Message-ID: <20101028103915.57DC0282C1D@codespeak.net>
Author: arigo
Date: Thu Oct 28 12:39:13 2010
New Revision: 78395
Modified:
pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
Log:
Fix and document this test.
Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Thu Oct 28 12:39:13 2010
@@ -294,7 +294,11 @@
ops = self.get_by_bytecode("LOAD_ATTR")
assert len(ops) == 2
- assert ops[0].get_opnames() == ["getfield_gc", "getarrayitem_gc",
+ # With mapdict, we get fast access to (so far) the 5 first
+ # attributes, which means it is done with only the following
+ # operations. (For the other attributes there is additionally
+ # a getarrayitem_gc.)
+ assert ops[0].get_opnames() == ["getfield_gc",
"guard_nonnull_class"]
assert not ops[1] # second LOAD_ATTR folded away
From arigo at codespeak.net Thu Oct 28 13:32:07 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Thu, 28 Oct 2010 13:32:07 +0200 (CEST)
Subject: [pypy-svn] r78397 - in pypy/trunk/pypy/jit/codewriter: . test
Message-ID: <20101028113207.0DFE05080C@codespeak.net>
Author: arigo
Date: Thu Oct 28 13:32:06 2010
New Revision: 78397
Modified:
pypy/trunk/pypy/jit/codewriter/format.py
pypy/trunk/pypy/jit/codewriter/jtransform.py
pypy/trunk/pypy/jit/codewriter/test/test_list.py
Log:
Minimal support for void lists: remove the operations that are no-ops.
Modified: pypy/trunk/pypy/jit/codewriter/format.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/format.py (original)
+++ pypy/trunk/pypy/jit/codewriter/format.py Thu Oct 28 13:32:06 2010
@@ -80,7 +80,8 @@
def assert_format(ssarepr, expected):
asm = format_assembler(ssarepr)
- expected = str(py.code.Source(expected)).strip() + '\n'
+ if expected != '':
+ expected = str(py.code.Source(expected)).strip() + '\n'
asmlines = asm.split("\n")
explines = expected.split("\n")
for asm, exp in zip(asmlines, explines):
Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/trunk/pypy/jit/codewriter/jtransform.py Thu Oct 28 13:32:06 2010
@@ -894,17 +894,21 @@
prefix = 'do_resizable_'
ARRAY = LIST.items.TO
if self._array_of_voids(ARRAY):
- raise NotSupported("resizable lists of voids")
- descrs = (self.cpu.arraydescrof(ARRAY),
- self.cpu.fielddescrof(LIST, 'length'),
- self.cpu.fielddescrof(LIST, 'items'),
- self.cpu.sizeof(LIST))
+ prefix += 'void_'
+ descrs = ()
+ else:
+ descrs = (self.cpu.arraydescrof(ARRAY),
+ self.cpu.fielddescrof(LIST, 'length'),
+ self.cpu.fielddescrof(LIST, 'items'),
+ self.cpu.sizeof(LIST))
else:
prefix = 'do_fixed_'
if self._array_of_voids(LIST):
- raise NotSupported("fixed lists of voids")
- arraydescr = self.cpu.arraydescrof(LIST)
- descrs = (arraydescr,)
+ prefix += 'void_'
+ descrs = ()
+ else:
+ arraydescr = self.cpu.arraydescrof(LIST)
+ descrs = (arraydescr,)
#
try:
meth = getattr(self, prefix + oopspec_name.replace('.', '_'))
@@ -943,6 +947,11 @@
descr, args[1]], v_posindex)
return v_posindex, [op0, op1]
+ def _prepare_void_list_getset(self, op):
+ non_negative, can_raise = self._get_list_nonneg_canraise_flags(op)
+ if can_raise:
+ raise NotSupported("list operation can raise")
+
def _get_initial_newlist_length(self, op, args):
# normalize number of arguments to the 'newlist' function
if len(args) > 1:
@@ -1014,6 +1023,12 @@
def do_fixed_list_ll_arraycopy(self, op, args, arraydescr):
return self._handle_oopspec_call(op, args, EffectInfo.OS_ARRAYCOPY)
+ def do_fixed_void_list_getitem(self, op, args):
+ self._prepare_void_list_getset(op)
+ return []
+ do_fixed_void_list_getitem_foldable = do_fixed_void_list_getitem
+ do_fixed_void_list_setitem = do_fixed_void_list_getitem
+
# ---------- resizable lists ----------
def do_resizable_newlist(self, op, args, arraydescr, lengthdescr,
@@ -1049,6 +1064,12 @@
return SpaceOperation('getfield_gc_i',
[args[0], lengthdescr], op.result)
+ def do_resizable_void_list_getitem(self, op, args):
+ self._prepare_void_list_getset(op)
+ return []
+ do_resizable_void_list_getitem_foldable = do_resizable_void_list_getitem
+ do_resizable_void_list_setitem = do_resizable_void_list_getitem
+
# ----------
# Strings and Unicodes.
Modified: pypy/trunk/pypy/jit/codewriter/test/test_list.py
==============================================================================
--- pypy/trunk/pypy/jit/codewriter/test/test_list.py (original)
+++ pypy/trunk/pypy/jit/codewriter/test/test_list.py Thu Oct 28 13:32:06 2010
@@ -19,6 +19,7 @@
class FakeCPU:
class arraydescrof(AbstractDescr):
def __init__(self, ARRAY):
+ assert ARRAY.OF != lltype.Void
self.ARRAY = ARRAY
def __repr__(self):
return ''
From afa at codespeak.net Thu Oct 28 14:59:11 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 14:59:11 +0200 (CEST)
Subject: [pypy-svn] r78398 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101028125911.25630282C1E@codespeak.net>
Author: afa
Date: Thu Oct 28 14:59:09 2010
New Revision: 78398
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
Log:
BufferedReader.read1
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Thu Oct 28 14:59:09 2010
@@ -27,6 +27,10 @@
self._unsupportedoperation(space, "read")
@unwrap_spec('self', ObjSpace, W_Root)
+ def read1_w(self, space, w_size):
+ self._unsupportedoperation(space, "read1")
+
+ @unwrap_spec('self', ObjSpace, W_Root)
def write_w(self, space, w_data):
self._unsupportedoperation(space, "write")
@@ -38,6 +42,7 @@
'_BufferedIOBase', W_IOBase.typedef,
__new__ = generic_new_descr(W_BufferedIOBase),
read = interp2app(W_BufferedIOBase.read_w),
+ read1 = interp2app(W_BufferedIOBase.read1_w),
write = interp2app(W_BufferedIOBase.write_w),
detach = interp2app(W_BufferedIOBase.detach_w),
)
@@ -296,6 +301,45 @@
res = self._read_generic(space, size)
return space.wrap(res)
+ @unwrap_spec('self', ObjSpace, int)
+ def read1_w(self, space, size):
+ self._check_init(space)
+ self._check_closed(space, "read of closed file")
+
+ if size < 0:
+ raise OperationError(space.w_ValueError, space.wrap(
+ "read length must be positive"))
+ if size == 0:
+ return space.wrap("")
+
+ with self.lock:
+ if self.writable:
+ self._writer_flush_unlocked(space, restore_pos=True)
+
+ # Return up to n bytes. If at least one byte is buffered, we only
+ # return buffered bytes. Otherwise, we do one raw read.
+
+ # XXX: this mimicks the io.py implementation but is probably
+ # wrong. If we need to read from the raw stream, then we could
+ # actually read all `n` bytes asked by the caller (and possibly
+ # more, so as to fill our buffer for the next reads).
+
+ have = self._readahead()
+ if have == 0:
+ # Fill the buffer from the raw stream
+ self._reader_reset_buf()
+ self.pos = 0
+ try:
+ have = self._fill_buffer(space)
+ except BlockingIOError:
+ have = 0
+ if size > have:
+ size = have
+ data = rffi.charpsize2str(rffi.ptradd(self.buffer, self.pos),
+ size)
+ self.pos += size
+ return space.wrap(data)
+
def _read_all(self, space):
"Read all the file, don't update the cache"
builder = StringBuilder()
@@ -433,6 +477,7 @@
__init__ = interp2app(W_BufferedReader.descr_init),
read = interp2app(W_BufferedReader.read_w),
+ read1 = interp2app(W_BufferedReader.read1_w),
# from the mixin class
seek = interp2app(W_BufferedReader.seek_w),
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Thu Oct 28 14:59:09 2010
@@ -32,6 +32,29 @@
assert f.read(3) == ""
f.close()
+ def test_read1(self):
+ import _io
+ class RecordingFileIO(_io.FileIO):
+ def read(self, size=-1):
+ self.nbreads += 1
+ return _io.FileIO.read(self, size)
+ def readinto(self, buf):
+ self.nbreads += 1
+ return _io.FileIO.readinto(self, buf)
+ raw = RecordingFileIO(self.tmpfile)
+ raw.nbreads = 0
+ f = _io.BufferedReader(raw, buffer_size=3)
+ assert f.read(1) == 'a'
+ assert f.read1(1) == '\n'
+ assert raw.nbreads == 1
+ assert f.read1(100) == 'b'
+ assert raw.nbreads == 1
+ assert f.read1(100) == '\nc'
+ assert raw.nbreads == 2
+ assert f.read1(100) == ''
+ assert raw.nbreads == 3
+ f.close()
+
def test_seek(self):
import _io
raw = _io.FileIO(self.tmpfile)
From afa at codespeak.net Thu Oct 28 15:15:38 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 15:15:38 +0200 (CEST)
Subject: [pypy-svn] r78399 - in pypy/trunk/pypy/translator/c/gcc: .
test/darwin64
Message-ID: <20101028131538.3D443282C1E@codespeak.net>
Author: afa
Date: Thu Oct 28 15:15:36 2010
New Revision: 78399
Added:
pypy/trunk/pypy/translator/c/gcc/test/darwin64/track1.s
Modified:
pypy/trunk/pypy/translator/c/gcc/trackgcroot.py
Log:
On darwin 64bit, fix pattern to recognise items of table-based switches
Added: pypy/trunk/pypy/translator/c/gcc/test/darwin64/track1.s
==============================================================================
--- (empty file)
+++ pypy/trunk/pypy/translator/c/gcc/test/darwin64/track1.s Thu Oct 28 15:15:36 2010
@@ -0,0 +1,785 @@
+_pypy_g_PyObject_RichCompare:
+LFB302:
+ movq %rbx, -48(%rsp)
+LCFI297:
+ movq %rbp, -40(%rsp)
+LCFI298:
+ movq %r12, -32(%rsp)
+LCFI299:
+ movq %r13, -24(%rsp)
+LCFI300:
+ movq %r14, -16(%rsp)
+LCFI301:
+ movq %r15, -8(%rsp)
+LCFI302:
+ subq $88, %rsp
+LCFI303:
+ movq %rdi, %rbx
+ movq %rsi, %rbp
+L970:
+ cmpl $5, %edx
+ jbe L1056
+L971:
+ movq _pypy_g_pypy_rpython_memory_gc_minimark_MiniMarkGC at GOTPCREL(%rip), %rdi
+ movq 16(%rdi), %rdx
+ leaq 16(%rdx), %rax
+ movq %rax, 16(%rdi)
+ cmpq 24(%rdi), %rax
+ ja L1034
+ movq %rdx, %rbx
+ movq _pypy_g_ExcData at GOTPCREL(%rip), %r11
+L1036:
+ movq $61224, (%rbx)
+ movq $0, 8(%rdx)
+L1037:
+ movq %r11, (%rsp)
+ call _pypy_g_PyErr_BadInternalCall
+ ;; expected {88(%rsp) | 40(%rsp), 56(%rsp), 64(%rsp), 72(%rsp), 80(%rsp), 48(%rsp) | %rbx}
+ movq ___gcmapend at GOTPCREL(%rip), %rax
+ /* GCROOT %rbx */
+ movq (%rsp), %r11
+ movq (%r11), %rbp
+ testq %rbp, %rbp
+ je L1057
+L1038:
+ movq 8(%r11), %r12
+ movq _pypydtcount at GOTPCREL(%rip), %r10
+ movl (%r10), %eax
+ movq _pypy_debug_tracebacks at GOTPCREL(%rip), %r9
+ movslq %eax,%rdx
+ salq $4, %rdx
+ leaq _loc.235055(%rip), %rcx
+ movq %rcx, (%rdx,%r9)
+ movq %rbp, 8(%rdx,%r9)
+ incl %eax
+ andl $127, %eax
+ movl %eax, (%r10)
+ movq _pypy_g_typeinfo at GOTPCREL(%rip), %r8
+ leaq 59688(%r8), %rax
+ cmpq %rax, %rbp
+ je L1040
+ leaq 59544(%r8), %rax
+ cmpq %rax, %rbp
+ je L1040
+L1042:
+ movq $0, 8(%r11)
+ movq $0, (%r11)
+ /* keepalive %r12 */
+ /* keepalive %rbx */
+L1043:
+L1044:
+ /* GC_NOCOLLECT pypy_g_ll_issubclass */
+L1046:
+L1047:
+ /* keepalive %rbx */
+ cmpq $0, 8(%rbx)
+ jle L1048
+L1049:
+ /* keepalive %rbx */
+L1050:
+ leaq 59688(%r8), %rcx
+ movq %rcx, (%r11)
+ movq _pypy_g_exceptions_AssertionError_6 at GOTPCREL(%rip), %rax
+ movq %rax, 8(%r11)
+ movl (%r10), %edx
+ movslq %edx,%rax
+ salq $4, %rax
+ movq $0, (%rax,%r9)
+ movq %rcx, 8(%rax,%r9)
+ incl %edx
+ andl $127, %edx
+ /* GC_NOCOLLECT pypy_g_RPyRaiseException */
+ movslq %edx,%rax
+ salq $4, %rax
+ leaq _loc.235061(%rip), %rbx
+ movq %rbx, (%rax,%r9)
+ movq $0, 8(%rax,%r9)
+ incl %edx
+ andl $127, %edx
+ movl %edx, (%r10)
+ xorl %eax, %eax
+ .align 4,0x90
+L981:
+ movq 40(%rsp), %rbx
+ movq 48(%rsp), %rbp
+ movq 56(%rsp), %r12
+ movq 64(%rsp), %r13
+ movq 72(%rsp), %r14
+ movq 80(%rsp), %r15
+ addq $88, %rsp
+ ret
+ .align 4,0x90
+L1056:
+ mov %edx, %eax
+ leaq L978(%rip), %rdx
+ movslq (%rdx,%rax,4),%rax
+ addq %rdx, %rax
+ jmp *%rax
+ .align 2,0x90
+L978:
+ .long L972-L978
+ .long L973-L978
+ .long L974-L978
+ .long L975-L978
+ .long L976-L978
+ .long L977-L978
+L972:
+ /* keepalive %rdi */
+ /* keepalive %rsi */
+ call _pypy_g___mm_lt_0_perform_call
+ ;; expected {88(%rsp) | 40(%rsp), 56(%rsp), 64(%rsp), 72(%rsp), 80(%rsp), 48(%rsp) | %rbx, %rbp}
+ movq ___gcmapend at GOTPCREL(%rip), %rdx
+ movq %rbx, %r15
+ /* GCROOT %r15 */
+ movq %rbp, %r14
+ /* GCROOT %r14 */
+ movq %rbx, %r12
+ /* GCROOT %r12 */
+ /* GCROOT %rbp */
+ movq _pypy_g_ExcData at GOTPCREL(%rip), %r11
+ movq (%r11), %rbx
+ testq %rbx, %rbx
+ je L981
+L979:
+ movq 8(%r11), %r13
+ movq _pypydtcount at GOTPCREL(%rip), %r10
+ movl (%r10), %eax
+ movq _pypy_debug_tracebacks at GOTPCREL(%rip), %r9
+ movslq %eax,%rdx
+ salq $4, %rdx
+ leaq _loc.234970(%rip), %rcx
+ movq %rcx, (%rdx,%r9)
+ movq %rbx, 8(%rdx,%r9)
+ incl %eax
+ andl $127, %eax
+ movl %eax, (%r10)
+ movq _pypy_g_typeinfo at GOTPCREL(%rip), %r8
+ leaq 59688(%r8), %rax
+ cmpq %rax, %rbx
+ je L982
+ leaq 59544(%r8), %rax
+ cmpq %rax, %rbx
+ je L982
+L984:
+ movq $0, 8(%r11)
+ movq $0, (%r11)
+ /* keepalive %r13 */
+ /* keepalive %r15 */
+ /* keepalive %r14 */
+ /* keepalive %r12 */
+ /* keepalive %rbp */
+L985:
+ movq 1712(%r8), %rcx
+ movq (%rbx), %rdx
+ subq %rcx, %rdx
+ movq 1720(%r8), %rax
+L986:
+ /* GC_NOCOLLECT pypy_g_ll_issubclass */
+ subq %rcx, %rax
+ cmpq %rax, %rdx
+ jae L1058
+L987:
+ /* keepalive %r12 */
+ /* keepalive %rbp */
+ movq %rbp, %rsi
+ movq %r12, %rdi
+ movq 40(%rsp), %rbx
+ movq 48(%rsp), %rbp
+ movq 56(%rsp), %r12
+ movq 64(%rsp), %r13
+ movq 72(%rsp), %r14
+ movq 80(%rsp), %r15
+ addq $88, %rsp
+ jmp _pypy_g_comparison_lt_impl
+L973:
+ /* keepalive %rdi */
+ /* keepalive %rsi */
+ call _pypy_g___mm_le_0_perform_call
+ ;; expected {88(%rsp) | 40(%rsp), 56(%rsp), 64(%rsp), 72(%rsp), 80(%rsp), 48(%rsp) | %rbx, %rbp}
+ movq ___gcmapend at GOTPCREL(%rip), %rdx
+ movq %rbx, %r15
+ /* GCROOT %r15 */
+ movq %rbp, %r14
+ /* GCROOT %r14 */
+ movq %rbx, %r12
+ /* GCROOT %r12 */
+ /* GCROOT %rbp */
+ movq _pypy_g_ExcData at GOTPCREL(%rip), %r11
+ movq (%r11), %rbx
+ testq %rbx, %rbx
+ je L981
+L989:
+ movq 8(%r11), %r13
+ movq _pypydtcount at GOTPCREL(%rip), %r10
+ movl (%r10), %eax
+ movq _pypy_debug_tracebacks at GOTPCREL(%rip), %r9
+ movslq %eax,%rdx
+ salq $4, %rdx
+ leaq _loc.234982(%rip), %rcx
+ movq %rcx, (%rdx,%r9)
+ movq %rbx, 8(%rdx,%r9)
+ incl %eax
+ andl $127, %eax
+ movl %eax, (%r10)
+ movq _pypy_g_typeinfo at GOTPCREL(%rip), %r8
+ leaq 59688(%r8), %rax
+ cmpq %rax, %rbx
+ je L991
+ leaq 59544(%r8), %rax
+ cmpq %rax, %rbx
+ je L991
+L993:
+ movq $0, 8(%r11)
+ movq $0, (%r11)
+ /* keepalive %r13 */
+ /* keepalive %r15 */
+ /* keepalive %r14 */
+ /* keepalive %r12 */
+ /* keepalive %rbp */
+L994:
+ movq 1712(%r8), %rcx
+ movq (%rbx), %rdx
+ subq %rcx, %rdx
+ movq 1720(%r8), %rax
+L995:
+ /* GC_NOCOLLECT pypy_g_ll_issubclass */
+ subq %rcx, %rax
+ cmpq %rax, %rdx
+ jae L1059
+L996:
+ /* keepalive %r12 */
+ /* keepalive %rbp */
+ movq %rbp, %rsi
+ movq %r12, %rdi
+ movq 40(%rsp), %rbx
+ movq 48(%rsp), %rbp
+ movq 56(%rsp), %r12
+ movq 64(%rsp), %r13
+ movq 72(%rsp), %r14
+ movq 80(%rsp), %r15
+ addq $88, %rsp
+ jmp _pypy_g_comparison_le_impl
+L974:
+ /* keepalive %rdi */
+ /* keepalive %rsi */
+ call _pypy_g___mm_eq_0_perform_call
+ ;; expected {88(%rsp) | 40(%rsp), 56(%rsp), 64(%rsp), 72(%rsp), 80(%rsp), 48(%rsp) | %rbx, %rbp}
+ movq ___gcmapend at GOTPCREL(%rip), %rdx
+ movq %rbx, %r15
+ /* GCROOT %r15 */
+ movq %rbp, %r14
+ /* GCROOT %r14 */
+ movq %rbx, %r12
+ /* GCROOT %r12 */
+ /* GCROOT %rbp */
+ movq _pypy_g_ExcData at GOTPCREL(%rip), %r11
+ movq (%r11), %rbx
+ testq %rbx, %rbx
+ je L981
+L998:
+ movq 8(%r11), %r13
+ movq _pypydtcount at GOTPCREL(%rip), %r10
+ movl (%r10), %eax
+ movq _pypy_debug_tracebacks at GOTPCREL(%rip), %r9
+ movslq %eax,%rdx
+ salq $4, %rdx
+ leaq _loc.234994(%rip), %rcx
+ movq %rcx, (%rdx,%r9)
+ movq %rbx, 8(%rdx,%r9)
+ incl %eax
+ andl $127, %eax
+ movl %eax, (%r10)
+ movq _pypy_g_typeinfo at GOTPCREL(%rip), %r8
+ leaq 59688(%r8), %rax
+ cmpq %rax, %rbx
+ je L1000
+ leaq 59544(%r8), %rax
+ cmpq %rax, %rbx
+ je L1000
+L1002:
+ movq $0, 8(%r11)
+ movq $0, (%r11)
+ /* keepalive %r13 */
+ /* keepalive %r15 */
+ /* keepalive %r14 */
+ /* keepalive %rbp */
+ /* keepalive %r12 */
+L1003:
+ movq 1712(%r8), %rcx
+ movq (%rbx), %rdx
+ subq %rcx, %rdx
+ movq 1720(%r8), %rax
+L1004:
+ /* GC_NOCOLLECT pypy_g_ll_issubclass */
+ subq %rcx, %rax
+ cmpq %rax, %rdx
+ jae L1060
+L1005:
+ /* keepalive %r12 */
+ /* keepalive %rbp */
+ movq %rbp, %rsi
+ movq %r12, %rdi
+ movq 40(%rsp), %rbx
+ movq 48(%rsp), %rbp
+ movq 56(%rsp), %r12
+ movq 64(%rsp), %r13
+ movq 72(%rsp), %r14
+ movq 80(%rsp), %r15
+ addq $88, %rsp
+ jmp _pypy_g_comparison_eq_impl
+L975:
+ /* keepalive %rdi */
+ /* keepalive %rsi */
+ call _pypy_g___mm_ne_0_perform_call
+ ;; expected {88(%rsp) | 40(%rsp), 56(%rsp), 64(%rsp), 72(%rsp), 80(%rsp), 48(%rsp) | %rbx, %rbp}
+ movq ___gcmapend at GOTPCREL(%rip), %rdx
+ movq %rbx, %r15
+ /* GCROOT %r15 */
+ movq %rbp, %r14
+ /* GCROOT %r14 */
+ movq %rbx, %r12
+ /* GCROOT %r12 */
+ /* GCROOT %rbp */
+ movq _pypy_g_ExcData at GOTPCREL(%rip), %r11
+ movq (%r11), %rbx
+ testq %rbx, %rbx
+ je L981
+L1007:
+ movq 8(%r11), %r13
+ movq _pypydtcount at GOTPCREL(%rip), %r10
+ movl (%r10), %eax
+ movq _pypy_debug_tracebacks at GOTPCREL(%rip), %r9
+ movslq %eax,%rdx
+ salq $4, %rdx
+ leaq _loc.235006(%rip), %rcx
+ movq %rcx, (%rdx,%r9)
+ movq %rbx, 8(%rdx,%r9)
+ incl %eax
+ andl $127, %eax
+ movl %eax, (%r10)
+ movq _pypy_g_typeinfo at GOTPCREL(%rip), %r8
+ leaq 59688(%r8), %rax
+ cmpq %rax, %rbx
+ je L1009
+ leaq 59544(%r8), %rax
+ cmpq %rax, %rbx
+ je L1009
+L1011:
+ movq $0, 8(%r11)
+ movq $0, (%r11)
+ /* keepalive %r13 */
+ /* keepalive %r15 */
+ /* keepalive %r14 */
+ /* keepalive %r12 */
+ /* keepalive %rbp */
+L1012:
+ movq 1712(%r8), %rcx
+ movq (%rbx), %rdx
+ subq %rcx, %rdx
+ movq 1720(%r8), %rax
+L1013:
+ /* GC_NOCOLLECT pypy_g_ll_issubclass */
+ subq %rcx, %rax
+ cmpq %rax, %rdx
+ jae L1061
+L1014:
+ /* keepalive %r12 */
+ /* keepalive %rbp */
+ movq %rbp, %rsi
+ movq %r12, %rdi
+ movq 40(%rsp), %rbx
+ movq 48(%rsp), %rbp
+ movq 56(%rsp), %r12
+ movq 64(%rsp), %r13
+ movq 72(%rsp), %r14
+ movq 80(%rsp), %r15
+ addq $88, %rsp
+ jmp _pypy_g_comparison_ne_impl
+L976:
+ /* keepalive %rdi */
+ /* keepalive %rsi */
+ call _pypy_g___mm_gt_0_perform_call
+ ;; expected {88(%rsp) | 40(%rsp), 56(%rsp), 64(%rsp), 72(%rsp), 80(%rsp), 48(%rsp) | %rbx, %rbp}
+ movq ___gcmapend at GOTPCREL(%rip), %rdx
+ movq %rbx, %r15
+ /* GCROOT %r15 */
+ movq %rbp, %r14
+ /* GCROOT %r14 */
+ movq %rbx, %r12
+ /* GCROOT %r12 */
+ /* GCROOT %rbp */
+ movq _pypy_g_ExcData at GOTPCREL(%rip), %r11
+ movq (%r11), %rbx
+ testq %rbx, %rbx
+ je L981
+L1016:
+ movq 8(%r11), %r13
+ movq _pypydtcount at GOTPCREL(%rip), %r10
+ movl (%r10), %eax
+ movq _pypy_debug_tracebacks at GOTPCREL(%rip), %r9
+ movslq %eax,%rdx
+ salq $4, %rdx
+ leaq _loc.235018(%rip), %rcx
+ movq %rcx, (%rdx,%r9)
+ movq %rbx, 8(%rdx,%r9)
+ incl %eax
+ andl $127, %eax
+ movl %eax, (%r10)
+ movq _pypy_g_typeinfo at GOTPCREL(%rip), %r8
+ leaq 59688(%r8), %rax
+ cmpq %rax, %rbx
+ je L1018
+ leaq 59544(%r8), %rax
+ cmpq %rax, %rbx
+ je L1018
+L1020:
+ movq $0, 8(%r11)
+ movq $0, (%r11)
+ /* keepalive %r13 */
+ /* keepalive %r15 */
+ /* keepalive %r14 */
+ /* keepalive %rbp */
+ /* keepalive %r12 */
+L1021:
+ movq 1712(%r8), %rcx
+ movq (%rbx), %rdx
+ subq %rcx, %rdx
+ movq 1720(%r8), %rax
+L1022:
+ /* GC_NOCOLLECT pypy_g_ll_issubclass */
+ subq %rcx, %rax
+ cmpq %rax, %rdx
+ jae L1062
+L1023:
+ /* keepalive %r12 */
+ /* keepalive %rbp */
+ movq %rbp, %rsi
+ movq %r12, %rdi
+ movq 40(%rsp), %rbx
+ movq 48(%rsp), %rbp
+ movq 56(%rsp), %r12
+ movq 64(%rsp), %r13
+ movq 72(%rsp), %r14
+ movq 80(%rsp), %r15
+ addq $88, %rsp
+ jmp _pypy_g_comparison_gt_impl
+L977:
+ /* keepalive %rdi */
+ /* keepalive %rsi */
+ call _pypy_g___mm_ge_0_perform_call
+ ;; expected {88(%rsp) | 40(%rsp), 56(%rsp), 64(%rsp), 72(%rsp), 80(%rsp), 48(%rsp) | %rbx, %rbp}
+ movq ___gcmapend at GOTPCREL(%rip), %rdx
+ movq %rbx, %r15
+ /* GCROOT %r15 */
+ movq %rbp, %r14
+ /* GCROOT %r14 */
+ movq %rbx, %r12
+ /* GCROOT %r12 */
+ /* GCROOT %rbp */
+ movq _pypy_g_ExcData at GOTPCREL(%rip), %r11
+ movq (%r11), %rbx
+ testq %rbx, %rbx
+ je L981
+L1025:
+ movq 8(%r11), %r13
+ movq _pypydtcount at GOTPCREL(%rip), %r10
+ movl (%r10), %eax
+ movq _pypy_debug_tracebacks at GOTPCREL(%rip), %r9
+ movslq %eax,%rdx
+ salq $4, %rdx
+ leaq _loc.235030(%rip), %rcx
+ movq %rcx, (%rdx,%r9)
+ movq %rbx, 8(%rdx,%r9)
+ incl %eax
+ andl $127, %eax
+ movl %eax, (%r10)
+ movq _pypy_g_typeinfo at GOTPCREL(%rip), %r8
+ leaq 59688(%r8), %rax
+ cmpq %rax, %rbx
+ je L1027
+ leaq 59544(%r8), %rax
+ cmpq %rax, %rbx
+ je L1027
+L1029:
+ movq $0, 8(%r11)
+ movq $0, (%r11)
+ /* keepalive %r13 */
+ /* keepalive %r15 */
+ /* keepalive %r14 */
+ /* keepalive %r12 */
+ /* keepalive %rbp */
+L1030:
+ movq 1712(%r8), %rcx
+ movq (%rbx), %rdx
+ subq %rcx, %rdx
+ movq 1720(%r8), %rax
+L1031:
+ /* GC_NOCOLLECT pypy_g_ll_issubclass */
+ subq %rcx, %rax
+ cmpq %rax, %rdx
+ jae L1063
+L1032:
+ /* keepalive %r12 */
+ /* keepalive %rbp */
+ movq %rbp, %rsi
+ movq %r12, %rdi
+ movq 40(%rsp), %rbx
+ movq 48(%rsp), %rbp
+ movq 56(%rsp), %r12
+ movq 64(%rsp), %r13
+ movq 72(%rsp), %r14
+ movq 80(%rsp), %r15
+ addq $88, %rsp
+ jmp _pypy_g_comparison_ge_impl
+L1045:
+L1034:
+ movl $16, %esi
+ call _pypy_g_MiniMarkGC_collect_and_reserve
+ ;; expected {88(%rsp) | 40(%rsp), 56(%rsp), 64(%rsp), 72(%rsp), 80(%rsp), 48(%rsp) | }
+ movq _pypy_g_ExcData at GOTPCREL(%rip), %r11
+ movq %rax, %rdx
+ movq %rax, %rbx
+ cmpq $0, (%r11)
+ je L1036
+L1053:
+ movq _pypydtcount at GOTPCREL(%rip), %rsi
+ movl (%rsi), %edx
+ movq _pypy_debug_tracebacks at GOTPCREL(%rip), %rcx
+ movslq %edx,%rax
+ salq $4, %rax
+ leaq _loc.235064(%rip), %rbx
+ movq %rbx, (%rax,%rcx)
+ movq $0, 8(%rcx,%rax)
+ incl %edx
+ andl $127, %edx
+L1052:
+ movslq %edx,%rax
+ salq $4, %rax
+ leaq _loc.235062(%rip), %rbx
+ movq %rbx, (%rax,%rcx)
+ movq $0, 8(%rax,%rcx)
+ incl %edx
+ andl $127, %edx
+ movl %edx, (%rsi)
+ xorl %eax, %eax
+ jmp L981
+L1027:
+ movq %r8, 24(%rsp)
+ movq %r9, 16(%rsp)
+ movq %r10, 8(%rsp)
+ movq %r11, (%rsp)
+ call _pypy_debug_catch_fatal_exception
+ ;; expected {88(%rsp) | 40(%rsp), 56(%rsp), 64(%rsp), 72(%rsp), 80(%rsp), 48(%rsp) | }
+ movq (%rsp), %r11
+ movq 8(%rsp), %r10
+ movq 16(%rsp), %r9
+ movq 24(%rsp), %r8
+ jmp L1029
+L1018:
+ movq %r8, 24(%rsp)
+ movq %r9, 16(%rsp)
+ movq %r10, 8(%rsp)
+ movq %r11, (%rsp)
+ call _pypy_debug_catch_fatal_exception
+ ;; expected {88(%rsp) | 40(%rsp), 56(%rsp), 64(%rsp), 72(%rsp), 80(%rsp), 48(%rsp) | }
+ movq (%rsp), %r11
+ movq 8(%rsp), %r10
+ movq 16(%rsp), %r9
+ movq 24(%rsp), %r8
+ jmp L1020
+L1009:
+ movq %r8, 24(%rsp)
+ movq %r9, 16(%rsp)
+ movq %r10, 8(%rsp)
+ movq %r11, (%rsp)
+ call _pypy_debug_catch_fatal_exception
+ ;; expected {88(%rsp) | 40(%rsp), 56(%rsp), 64(%rsp), 72(%rsp), 80(%rsp), 48(%rsp) | }
+ movq (%rsp), %r11
+ movq 8(%rsp), %r10
+ movq 16(%rsp), %r9
+ movq 24(%rsp), %r8
+ jmp L1011
+L1000:
+ movq %r8, 24(%rsp)
+ movq %r9, 16(%rsp)
+ movq %r10, 8(%rsp)
+ movq %r11, (%rsp)
+ call _pypy_debug_catch_fatal_exception
+ ;; expected {88(%rsp) | 40(%rsp), 56(%rsp), 64(%rsp), 72(%rsp), 80(%rsp), 48(%rsp) | }
+ movq (%rsp), %r11
+ movq 8(%rsp), %r10
+ movq 16(%rsp), %r9
+ movq 24(%rsp), %r8
+ jmp L1002
+L991:
+ movq %r8, 24(%rsp)
+ movq %r9, 16(%rsp)
+ movq %r10, 8(%rsp)
+ movq %r11, (%rsp)
+ call _pypy_debug_catch_fatal_exception
+ ;; expected {88(%rsp) | 40(%rsp), 56(%rsp), 64(%rsp), 72(%rsp), 80(%rsp), 48(%rsp) | }
+ movq (%rsp), %r11
+ movq 8(%rsp), %r10
+ movq 16(%rsp), %r9
+ movq 24(%rsp), %r8
+ jmp L993
+L982:
+ movq %r8, 24(%rsp)
+ movq %r9, 16(%rsp)
+ movq %r10, 8(%rsp)
+ movq %r11, (%rsp)
+ call _pypy_debug_catch_fatal_exception
+ ;; expected {88(%rsp) | 40(%rsp), 56(%rsp), 64(%rsp), 72(%rsp), 80(%rsp), 48(%rsp) | }
+ movq (%rsp), %r11
+ movq 8(%rsp), %r10
+ movq 16(%rsp), %r9
+ movq 24(%rsp), %r8
+ jmp L984
+L1059:
+L997:
+ movq %rbx, (%r11)
+ movq %r13, 8(%r11)
+ movl (%r10), %edx
+ movslq %edx,%rax
+ salq $4, %rax
+ movq $-1, (%rax,%r9)
+ movq %rbx, 8(%rax,%r9)
+ incl %edx
+ andl $127, %edx
+ movl %edx, (%r10)
+ /* GC_NOCOLLECT pypy_g_RPyReRaiseException */
+ xorl %eax, %eax
+ jmp L981
+L1058:
+L988:
+ movq %rbx, (%r11)
+ movq %r13, 8(%r11)
+ movl (%r10), %edx
+ movslq %edx,%rax
+ salq $4, %rax
+ movq $-1, (%rax,%r9)
+ movq %rbx, 8(%rax,%r9)
+ incl %edx
+ andl $127, %edx
+ movl %edx, (%r10)
+ /* GC_NOCOLLECT pypy_g_RPyReRaiseException */
+ xorl %eax, %eax
+ jmp L981
+L1061:
+L1015:
+ movq %rbx, (%r11)
+ movq %r13, 8(%r11)
+ movl (%r10), %edx
+ movslq %edx,%rax
+ salq $4, %rax
+ movq $-1, (%rax,%r9)
+ movq %rbx, 8(%rax,%r9)
+ incl %edx
+ andl $127, %edx
+ movl %edx, (%r10)
+ /* GC_NOCOLLECT pypy_g_RPyReRaiseException */
+ xorl %eax, %eax
+ jmp L981
+L1060:
+L1006:
+ movq %rbx, (%r11)
+ movq %r13, 8(%r11)
+ movl (%r10), %edx
+ movslq %edx,%rax
+ salq $4, %rax
+ movq $-1, (%rax,%r9)
+ movq %rbx, 8(%rax,%r9)
+ incl %edx
+ andl $127, %edx
+ movl %edx, (%r10)
+ /* GC_NOCOLLECT pypy_g_RPyReRaiseException */
+ xorl %eax, %eax
+ jmp L981
+L1063:
+L1033:
+ movq %rbx, (%r11)
+ movq %r13, 8(%r11)
+ movl (%r10), %edx
+ movslq %edx,%rax
+ salq $4, %rax
+ movq $-1, (%rax,%r9)
+ movq %rbx, 8(%rax,%r9)
+ incl %edx
+ andl $127, %edx
+ movl %edx, (%r10)
+ /* GC_NOCOLLECT pypy_g_RPyReRaiseException */
+ xorl %eax, %eax
+ jmp L981
+L1062:
+L1024:
+ movq %rbx, (%r11)
+ movq %r13, 8(%r11)
+ movl (%r10), %edx
+ movslq %edx,%rax
+ salq $4, %rax
+ movq $-1, (%rax,%r9)
+ movq %rbx, 8(%rax,%r9)
+ incl %edx
+ andl $127, %edx
+ movl %edx, (%r10)
+ /* GC_NOCOLLECT pypy_g_RPyReRaiseException */
+ xorl %eax, %eax
+ jmp L981
+L1040:
+ movq %r8, 24(%rsp)
+ movq %r9, 16(%rsp)
+ movq %r10, 8(%rsp)
+ movq %r11, (%rsp)
+ call _pypy_debug_catch_fatal_exception
+ ;; expected {88(%rsp) | 40(%rsp), 56(%rsp), 64(%rsp), 72(%rsp), 80(%rsp), 48(%rsp) | }
+ movq (%rsp), %r11
+ movq 8(%rsp), %r10
+ movq 16(%rsp), %r9
+ movq 24(%rsp), %r8
+ jmp L1042
+L1048:
+ /* keepalive %rbx */
+ /* keepalive %r12 */
+L1051:
+ movq %rbp, (%r11)
+ movq %r12, 8(%r11)
+ movl (%r10), %edx
+ movslq %edx,%rax
+ salq $4, %rax
+ movq $-1, (%rax,%r9)
+ movq %rbp, 8(%rax,%r9)
+ incl %edx
+ andl $127, %edx
+ movl %edx, (%r10)
+ /* GC_NOCOLLECT pypy_g_RPyReRaiseException */
+ xorl %eax, %eax
+ jmp L981
+L1057:
+L1039:
+ movq _pypy_g_typeinfo at GOTPCREL(%rip), %rsi
+ addq $59688, %rsi
+ movq %rsi, (%r11)
+ movq _pypy_g_exceptions_AssertionError_5 at GOTPCREL(%rip), %rax
+ movq %rax, 8(%r11)
+ movq _pypydtcount at GOTPCREL(%rip), %rdi
+ movl (%rdi), %edx
+ movq _pypy_debug_tracebacks at GOTPCREL(%rip), %rcx
+ movslq %edx,%rax
+ salq $4, %rax
+ movq $0, (%rax,%rcx)
+ movq %rsi, 8(%rcx,%rax)
+ incl %edx
+ andl $127, %edx
+ /* GC_NOCOLLECT pypy_g_RPyRaiseException */
+ movslq %edx,%rax
+ salq $4, %rax
+ leaq _loc.235054(%rip), %rbx
+ movq %rbx, (%rax,%rcx)
+ movq $0, 8(%rcx,%rax)
+ incl %edx
+ andl $127, %edx
+ movl %edx, (%rdi)
+ xorl %eax, %eax
+ jmp L981
+LFE302:
+ .align 4,0x90
Modified: pypy/trunk/pypy/translator/c/gcc/trackgcroot.py
==============================================================================
--- pypy/trunk/pypy/translator/c/gcc/trackgcroot.py (original)
+++ pypy/trunk/pypy/translator/c/gcc/trackgcroot.py Thu Oct 28 15:15:36 2010
@@ -1106,6 +1106,9 @@
format = 'darwin64'
function_names_prefix = '_'
+ LABEL = ElfFunctionGcRootTracker32.LABEL
+ r_jmptable_item = re.compile(r"\t.(?:long|quad)\t"+LABEL+"(-\"?[A-Za-z0-9$]+\"?)?\s*$")
+
r_functionstart = re.compile(r"_(\w+):\s*$")
OFFSET_LABELS = 0
From afa at codespeak.net Thu Oct 28 15:51:56 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 15:51:56 +0200 (CEST)
Subject: [pypy-svn] r78401 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101028135156.6F5DC282C1E@codespeak.net>
Author: afa
Date: Thu Oct 28 15:51:55 2010
New Revision: 78401
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_io.py
pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
Log:
Test and fix for BlockingIOError
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_io.py Thu Oct 28 15:51:55 2010
@@ -18,7 +18,7 @@
self.written = written
W_BlockingIOError.typedef = TypeDef(
- 'BlockingIOError',
+ 'BlockingIOError', W_IOError.typedef,
__doc__ = ("Exception raised when I/O would block "
"on a non-blocking I/O stream"),
__new__ = generic_new_descr(W_BlockingIOError),
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Thu Oct 28 15:51:55 2010
@@ -42,6 +42,16 @@
import _io
e = _io.UnsupportedOperation("seek")
+ def test_blockingerror(self):
+ import _io
+ try:
+ raise _io.BlockingIOError(42, "test blocking", 123)
+ except IOError, e:
+ assert isinstance(e, _io.BlockingIOError)
+ assert e.errno == 42
+ assert e.strerror == "test blocking"
+ assert e.characters_written == 123
+
def test_destructor(self):
import io
io.IOBase()
From afa at codespeak.net Thu Oct 28 16:04:20 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 16:04:20 +0200 (CEST)
Subject: [pypy-svn] r78402 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101028140420.B603836E076@codespeak.net>
Author: afa
Date: Thu Oct 28 16:04:19 2010
New Revision: 78402
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
Log:
BufferedReader.readinto()
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Thu Oct 28 16:04:19 2010
@@ -38,6 +38,19 @@
def detach_w(self, space):
self._unsupportedoperation(space, "detach")
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def readinto_w(self, space, w_buffer):
+ rwbuffer = space.rwbuffer_w(w_buffer)
+ length = rwbuffer.getlength()
+ w_data = space.call_method(self, "read", space.wrap(length))
+
+ if not space.isinstance_w(w_data, space.w_str):
+ raise OperationError(space.w_TypeError, space.wrap(
+ "read() should return bytes"))
+ data = space.str_w(w_data)
+ rwbuffer.setslice(0, data)
+ return space.wrap(len(data))
+
W_BufferedIOBase.typedef = TypeDef(
'_BufferedIOBase', W_IOBase.typedef,
__new__ = generic_new_descr(W_BufferedIOBase),
@@ -45,6 +58,7 @@
read1 = interp2app(W_BufferedIOBase.read1_w),
write = interp2app(W_BufferedIOBase.write_w),
detach = interp2app(W_BufferedIOBase.detach_w),
+ readinto = interp2app(W_BufferedIOBase.readinto_w),
)
class BufferedMixin:
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Thu Oct 28 16:04:19 2010
@@ -55,6 +55,15 @@
assert raw.nbreads == 3
f.close()
+ def test_readinto(self):
+ import _io
+ a = bytearray('x' * 10)
+ raw = _io.FileIO(self.tmpfile)
+ f = _io.BufferedReader(raw)
+ assert f.readinto(a) == 5
+ f.close()
+ assert a == 'a\nb\ncxxxxx'
+
def test_seek(self):
import _io
raw = _io.FileIO(self.tmpfile)
From afa at codespeak.net Thu Oct 28 16:15:44 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 16:15:44 +0200 (CEST)
Subject: [pypy-svn] r78403 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101028141544.BC367282C1E@codespeak.net>
Author: afa
Date: Thu Oct 28 16:15:43 2010
New Revision: 78403
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
Log:
Deprecate BufferedWriter.max_buffer_size
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Thu Oct 28 16:15:43 2010
@@ -123,6 +123,10 @@
self._check_init(space)
W_IOBase._check_closed(self, space, message)
+ def _deprecated_max_buffer_size(self, space):
+ space.warn("max_buffer_size is deprecated",
+ space.w_DeprecationWarning)
+
def _raw_tell(self, space):
w_pos = space.call_method(self.raw, "tell")
pos = space.r_longlong_w(w_pos)
@@ -503,8 +507,12 @@
)
class W_BufferedWriter(BufferedMixin, W_BufferedIOBase):
- @unwrap_spec('self', ObjSpace, W_Root, int)
- def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE):
+ @unwrap_spec('self', ObjSpace, W_Root, int, int)
+ def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE,
+ max_buffer_size=-234):
+ if max_buffer_size != -234:
+ self._deprecated_max_buffer_size(space)
+
self.state = STATE_ZERO
raw = space.interp_w(W_IOBase, w_raw)
raw.check_writable_w(space)
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Thu Oct 28 16:15:43 2010
@@ -131,6 +131,17 @@
raises(ValueError, b.flush)
raises(ValueError, b.close)
+ def test_deprecated_max_buffer_size(self):
+ import _io, warnings
+ raw = _io.FileIO(self.tmpfile, 'w')
+ with warnings.catch_warnings(record=True) as w:
+ warnings.simplefilter("always")
+ f = _io.BufferedWriter(raw, max_buffer_size=8192)
+ f.close()
+ assert len(w) == 1
+ assert str(w[0].message) == "max_buffer_size is deprecated"
+ assert w[0].category is DeprecationWarning
+
def test_check_several_writes(self):
import _io
raw = _io.FileIO(self.tmpfile, 'w')
From afa at codespeak.net Thu Oct 28 16:36:56 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 16:36:56 +0200 (CEST)
Subject: [pypy-svn] r78404 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101028143656.9759836E076@codespeak.net>
Author: afa
Date: Thu Oct 28 16:36:55 2010
New Revision: 78404
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
Log:
implement tell().
Fix an issue when seek() crosses the buffer boundaries
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Thu Oct 28 16:36:55 2010
@@ -153,6 +153,12 @@
return self.raw_pos - self.pos
return 0
+ @unwrap_spec('self', ObjSpace)
+ def tell_w(self, space):
+ self._check_init(space)
+ pos = self._raw_tell(space) - self._raw_offset()
+ return space.wrap(pos)
+
@unwrap_spec('self', ObjSpace, r_longlong, int)
def seek_w(self, space, pos, whence=0):
self._check_init(space)
@@ -425,6 +431,8 @@
data = rffi.charpsize2str(rffi.ptradd(self.buffer, self.pos),
current_size)
builder.append(data)
+ remaining -= len(data)
+ written += len(data)
self._reader_reset_buf()
# XXX potential bug in CPython? The following is not enabled.
@@ -446,6 +454,7 @@
size = len(data)
if size == 0:
return builder.build()
+ builder.append(data)
remaining -= size
written += size
@@ -499,6 +508,7 @@
# from the mixin class
seek = interp2app(W_BufferedReader.seek_w),
+ tell = interp2app(W_BufferedReader.tell_w),
close = interp2app(W_BufferedReader.close_w),
flush = interp2app(W_BufferedReader.flush_w),
detach = interp2app(W_BufferedReader.detach_w),
@@ -659,6 +669,7 @@
# from the mixin class
seek = interp2app(W_BufferedWriter.seek_w),
+ tell = interp2app(W_BufferedWriter.tell_w),
close = interp2app(W_BufferedWriter.close_w),
fileno = interp2app(W_BufferedWriter.fileno_w),
detach = interp2app(W_BufferedWriter.detach_w),
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Thu Oct 28 16:36:55 2010
@@ -94,6 +94,22 @@
assert not raw.closed
raw.close()
+ def test_tell(self):
+ import _io
+ raw = _io.FileIO(self.tmpfile)
+ f = _io.BufferedReader(raw, buffer_size=2)
+ assert f.tell() == 0
+ d1 = f.read(1)
+ assert f.tell() == 1
+ d2 = f.read(2)
+ assert f.tell() == 3
+ assert f.seek(0) == 0
+ assert f.tell() == 0
+ d3 = f.read(3)
+ assert f.tell() == 3
+ assert d1 + d2 == d3
+ f.close()
+
class AppTestBufferedWriter:
def setup_class(cls):
cls.space = gettestobjspace(usemodules=['_io'])
From afa at codespeak.net Thu Oct 28 17:03:15 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 17:03:15 +0200 (CEST)
Subject: [pypy-svn] r78409 - pypy/trunk/pypy/translator/c/gcc
Message-ID: <20101028150315.94620282C1E@codespeak.net>
Author: afa
Date: Thu Oct 28 17:03:14 2010
New Revision: 78409
Modified:
pypy/trunk/pypy/translator/c/gcc/trackgcroot.py
Log:
Add darwin64 variant for the pypy_asm_stackwalk() function
Modified: pypy/trunk/pypy/translator/c/gcc/trackgcroot.py
==============================================================================
--- pypy/trunk/pypy/translator/c/gcc/trackgcroot.py (original)
+++ pypy/trunk/pypy/translator/c/gcc/trackgcroot.py Thu Oct 28 17:03:14 2010
@@ -1655,7 +1655,8 @@
elif self.format == 'elf64':
print >> output, "\t.text"
print >> output, "\t.globl %s" % _globalname('pypy_asm_stackwalk')
- print >> output, "\t.type pypy_asm_stackwalk, @function"
+ _variant(elf64='.type pypy_asm_stackwalk, @function',
+ darwin64='')
print >> output, "%s:" % _globalname('pypy_asm_stackwalk')
print >> output, """\
@@ -1700,8 +1701,9 @@
/* the return value is the one of the 'call' above, */
/* because %rax (and possibly %rdx) are unmodified */
ret
- .size pypy_asm_stackwalk, .-pypy_asm_stackwalk
"""
+ _variant(elf64='.size pypy_asm_stackwalk, .-pypy_asm_stackwalk',
+ darwin64='')
else:
print >> output, "\t.text"
print >> output, "\t.globl %s" % _globalname('pypy_asm_stackwalk')
@@ -1828,6 +1830,7 @@
_variant(elf='.section\t.rodata',
elf64='.section\t.rodata',
darwin='.const',
+ darwin64='.const',
mingw32='')
print >> output, """\
From afa at codespeak.net Thu Oct 28 17:12:25 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 17:12:25 +0200 (CEST)
Subject: [pypy-svn] r78410 - pypy/trunk/pypy/translator/c/gcc
Message-ID: <20101028151225.1741436E076@codespeak.net>
Author: afa
Date: Thu Oct 28 17:12:24 2010
New Revision: 78410
Modified:
pypy/trunk/pypy/translator/c/gcc/trackgcroot.py
Log:
Fix pypy_asm_stackwalk() for darwin64, again
Modified: pypy/trunk/pypy/translator/c/gcc/trackgcroot.py
==============================================================================
--- pypy/trunk/pypy/translator/c/gcc/trackgcroot.py (original)
+++ pypy/trunk/pypy/translator/c/gcc/trackgcroot.py Thu Oct 28 17:12:24 2010
@@ -1652,7 +1652,7 @@
}
}
"""
- elif self.format == 'elf64':
+ elif self.format in ('elf64', 'darwin64'):
print >> output, "\t.text"
print >> output, "\t.globl %s" % _globalname('pypy_asm_stackwalk')
_variant(elf64='.type pypy_asm_stackwalk, @function',
From afa at codespeak.net Thu Oct 28 17:24:24 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 17:24:24 +0200 (CEST)
Subject: [pypy-svn] r78412 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101028152424.6FE5D282BE8@codespeak.net>
Author: afa
Date: Thu Oct 28 17:24:23 2010
New Revision: 78412
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
Log:
implement truncate() on buffered files
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Thu Oct 28 17:24:23 2010
@@ -170,7 +170,9 @@
# Check if seeking leaves us inside the current buffer, so as to
# return quickly if possible. Also, we needn't take the lock in
# this fast path.
- current = self._raw_tell(space)
+ if self.abs_pos == -1:
+ self._raw_tell(space)
+ current = self.abs_pos
available = self._readahead()
if available > 0:
if whence == 0:
@@ -293,6 +295,23 @@
self._check_init(space)
return space.call_method(self.raw, "fileno")
+ @unwrap_spec('self', ObjSpace, W_Root)
+ def truncate_w(self, space, w_size=None):
+ self._check_init(space)
+ with self.lock:
+ if self.writable:
+ self._writer_flush_unlocked(space)
+ if self.readable:
+ if space.is_w(w_size, space.w_None):
+ # Rewind the raw stream so that its position corresponds
+ # to the current logical position
+ self._raw_seek(space, -self._raw_offset(), 1)
+ self._reader_reset_buf()
+ # invalidate cached position
+ self.abs_pos = -1
+
+ return space.call_method(self.raw, "truncate", w_size)
+
class W_BufferedReader(BufferedMixin, W_BufferedIOBase):
@unwrap_spec('self', ObjSpace, W_Root, int)
def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE):
@@ -512,6 +531,7 @@
close = interp2app(W_BufferedReader.close_w),
flush = interp2app(W_BufferedReader.flush_w),
detach = interp2app(W_BufferedReader.detach_w),
+ truncate = interp2app(W_BufferedReader.truncate_w),
fileno = interp2app(W_BufferedReader.fileno_w),
closed = GetSetProperty(W_BufferedReader.closed_get_w),
)
@@ -673,6 +693,7 @@
close = interp2app(W_BufferedWriter.close_w),
fileno = interp2app(W_BufferedWriter.fileno_w),
detach = interp2app(W_BufferedWriter.detach_w),
+ truncate = interp2app(W_BufferedWriter.truncate_w),
closed = GetSetProperty(W_BufferedWriter.closed_get_w),
)
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Thu Oct 28 17:24:23 2010
@@ -187,3 +187,11 @@
import gc; gc.collect()
assert record == [1, 2, 3]
+ def test_truncate(self):
+ import _io
+ raw = _io.FileIO(self.tmpfile, 'w+')
+ raw.write('x' * 20)
+ b = _io.BufferedReader(raw)
+ assert b.seek(8) == 8
+ assert b.truncate() == 8
+ assert b.tell() == 8
From afa at codespeak.net Thu Oct 28 18:35:45 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 18:35:45 +0200 (CEST)
Subject: [pypy-svn] r78418 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101028163545.DDE2736E076@codespeak.net>
Author: afa
Date: Thu Oct 28 18:35:44 2010
New Revision: 78418
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py
Log:
BytesIO seek() and tell()
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py Thu Oct 28 18:35:44 2010
@@ -2,8 +2,11 @@
TypeDef, generic_new_descr)
from pypy.interpreter.gateway import interp2app, unwrap_spec
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
+from pypy.interpreter.error import OperationError, operationerrfmt
+from pypy.rlib.rarithmetic import r_longlong
from pypy.module._io.interp_bufferedio import W_BufferedIOBase
from pypy.module._io.interp_iobase import convert_size
+import sys
def buffer2string(buffer, start, end):
from pypy.rlib.rstring import StringBuilder
@@ -84,6 +87,38 @@
self._check_closed(space)
return space.wrap(buffer2string(self.buf, 0, self.string_size))
+ @unwrap_spec('self', ObjSpace)
+ def tell_w(self, space):
+ self._check_closed(space)
+ return space.wrap(self.pos)
+
+ @unwrap_spec('self', ObjSpace, r_longlong, int)
+ def seek_w(self, space, pos, whence=0):
+ self._check_closed(space)
+
+ if whence == 0:
+ if pos < 0:
+ raise OperationError(space.w_ValueError, space.wrap(
+ "negative seek value"))
+ if whence == 1:
+ if pos > sys.maxint - self.pos:
+ raise OperationError(space.w_OverflowError, space.wrap(
+ "new position too large"))
+ pos += self.pos
+ elif whence == 2:
+ if pos > sys.maxint - self.string_size:
+ raise OperationError(space.w_OverflowError, space.wrap(
+ "new position too large"))
+ pos += self.string_size
+ else:
+ raise operationerrfmt(space.w_ValueError,
+ "whence must be between 0 and 2, not %d", whence)
+
+ if pos >= 0:
+ self.pos = pos
+ else:
+ self.pos = 0
+ return space.wrap(self.pos)
W_BytesIO.typedef = TypeDef(
'BytesIO', W_BufferedIOBase.typedef,
@@ -93,5 +128,7 @@
read = interp2app(W_BytesIO.read_w),
write = interp2app(W_BytesIO.write_w),
getvalue = interp2app(W_BytesIO.getvalue_w),
+ seek = interp2app(W_BytesIO.seek_w),
+ tell = interp2app(W_BytesIO.tell_w),
)
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py Thu Oct 28 18:35:44 2010
@@ -19,7 +19,9 @@
import gc; gc.collect()
f.close()
- ## def test_seek(self):
- ## import _io
- ## f = _io.BytesIO("hello")
- ## assert f.seek(-1, 2) == 4
+ def test_seek(self):
+ import _io
+ f = _io.BytesIO("hello")
+ assert f.tell() == 0
+ assert f.seek(-1, 2) == 4
+ assert f.tell() == 4
From afa at codespeak.net Thu Oct 28 21:41:14 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 21:41:14 +0200 (CEST)
Subject: [pypy-svn] r78420 -
pypy/branch/fast-forward/lib-python/modified-2.7.0/test
Message-ID: <20101028194114.030E0282BE8@codespeak.net>
Author: afa
Date: Thu Oct 28 21:41:12 2010
New Revision: 78420
Added:
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_file.py
- copied, changed from r78419, pypy/branch/fast-forward/lib-python/2.7.0/test/test_file.py
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_file2k.py
- copied, changed from r78419, pypy/branch/fast-forward/lib-python/2.7.0/test/test_file2k.py
Log:
call gc.collect() before checking dead weakrefs
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_file.py (from r78419, pypy/branch/fast-forward/lib-python/2.7.0/test/test_file.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/test/test_file.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_file.py Thu Oct 28 21:41:12 2010
@@ -12,7 +12,7 @@
import io
import _pyio as pyio
-from test.test_support import TESTFN, run_unittest
+from test.test_support import TESTFN, run_unittest, gc_collect
from UserList import UserList
class AutoFileTests(unittest.TestCase):
@@ -33,6 +33,7 @@
self.assertEquals(self.f.tell(), p.tell())
self.f.close()
self.f = None
+ gc_collect()
self.assertRaises(ReferenceError, getattr, p, 'tell')
def testAttributes(self):
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_file2k.py (from r78419, pypy/branch/fast-forward/lib-python/2.7.0/test/test_file2k.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/test/test_file2k.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_file2k.py Thu Oct 28 21:41:12 2010
@@ -11,7 +11,7 @@
threading = None
from test import test_support
-from test.test_support import TESTFN, run_unittest
+from test.test_support import TESTFN, run_unittest, gc_collect
from UserList import UserList
class AutoFileTests(unittest.TestCase):
@@ -32,6 +32,7 @@
self.assertEquals(self.f.tell(), p.tell())
self.f.close()
self.f = None
+ gc_collect()
self.assertRaises(ReferenceError, getattr, p, 'tell')
def testAttributes(self):
From afa at codespeak.net Thu Oct 28 22:13:29 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 22:13:29 +0200 (CEST)
Subject: [pypy-svn] r78421 - in pypy/branch/fast-forward/pypy: interpreter
interpreter/astcompiler objspace/std
Message-ID: <20101028201329.71066282BE8@codespeak.net>
Author: afa
Date: Thu Oct 28 22:13:27 2010
New Revision: 78421
Modified:
pypy/branch/fast-forward/pypy/interpreter/astcompiler/codegen.py
pypy/branch/fast-forward/pypy/interpreter/pyopcode.py
pypy/branch/fast-forward/pypy/objspace/std/frame.py
Log:
Generate the same LIST_APPEND opcode as cpython.
Also remove a duplicated function.
Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/codegen.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/astcompiler/codegen.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/astcompiler/codegen.py Thu Oct 28 22:13:27 2010
@@ -1060,7 +1060,7 @@
self._listcomp_generator(gens, gen_index, elt)
else:
elt.walkabout(self)
- self.emit_op_arg(ops.LIST_APPEND, gen_index)
+ self.emit_op_arg(ops.LIST_APPEND, gen_index + 1)
self.use_next_block(skip)
self.use_next_block(if_cleanup)
self.emit_jump(ops.JUMP_ABSOLUTE, start, True)
Modified: pypy/branch/fast-forward/pypy/interpreter/pyopcode.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyopcode.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyopcode.py Thu Oct 28 22:13:27 2010
@@ -1037,7 +1037,7 @@
def LIST_APPEND(self, oparg, next_instr):
w = self.popvalue()
- v = self.peekvalue(oparg)
+ v = self.peekvalue(oparg - 1)
self.space.call_method(v, 'append', w)
def SET_ADD(self, oparg, next_instr):
@@ -1124,11 +1124,6 @@
w_dict = self.space.newdict()
self.pushvalue(w_dict)
- def LIST_APPEND(self, oparg, next_instr):
- w_value = self.popvalue()
- w_list = self.peekvalue(oparg + 1)
- self.space.call_method(w_list, "append", w_value)
-
def STORE_MAP(self, zero, next_instr):
if sys.version_info >= (2, 6):
w_key = self.popvalue()
Modified: pypy/branch/fast-forward/pypy/objspace/std/frame.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/frame.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/frame.py Thu Oct 28 22:13:27 2010
@@ -18,7 +18,7 @@
def LIST_APPEND(f, oparg, next_instr):
w = f.popvalue()
- v = f.peekvalue(oparg)
+ v = f.peekvalue(oparg - 1)
if type(v) is W_ListObject:
v.append(w)
else:
From afa at codespeak.net Thu Oct 28 23:05:53 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 23:05:53 +0200 (CEST)
Subject: [pypy-svn] r78422 -
pypy/branch/fast-forward/pypy/interpreter/pyparser/test
Message-ID: <20101028210553.D050A282C1E@codespeak.net>
Author: afa
Date: Thu Oct 28 23:05:52 2010
New Revision: 78422
Modified:
pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py
Log:
Fix tests after I changed the error message
Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/pyparser/test/test_pyparse.py Thu Oct 28 23:05:52 2010
@@ -69,9 +69,9 @@
assert exc.offset == 5
assert exc.text.startswith("name another for")
exc = py.test.raises(SyntaxError, parse, "\"blah").value
- assert exc.msg == "EOL while scanning single-quoted string"
+ assert exc.msg == "EOL while scanning string literal"
exc = py.test.raises(SyntaxError, parse, "'''\n").value
- assert exc.msg == "EOF while scanning triple-quoted string"
+ assert exc.msg == "EOF while scanning triple-quoted string literal"
for input in ("())", "(()", "((", "))"):
py.test.raises(SyntaxError, parse, input)
From afa at codespeak.net Thu Oct 28 23:33:49 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 23:33:49 +0200 (CEST)
Subject: [pypy-svn] r78424 - pypy/branch/fast-forward/pypy/objspace/std
Message-ID: <20101028213349.95DCC282C1E@codespeak.net>
Author: afa
Date: Thu Oct 28 23:33:48 2010
New Revision: 78424
Modified:
pypy/branch/fast-forward/pypy/objspace/std/formatting.py
Log:
Implement the same kind of limit as cpython,
(except that cpython limit is 117)
this avoids a memory error with the '%.123456789d' format
Modified: pypy/branch/fast-forward/pypy/objspace/std/formatting.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/formatting.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/formatting.py Thu Oct 28 23:33:48 2010
@@ -41,6 +41,10 @@
def std_wp_int(self, r, prefix='', keep_zero=False):
# use self.prec to add some '0' on the left of the number
if self.prec >= 0:
+ if self.prec > 1000:
+ raise OperationError(
+ self.space.w_OverflowError, self.space.wrap(
+ 'formatted integer is too long (precision too large?)'))
sign = r[0] == '-'
padding = self.prec - (len(r)-int(sign))
if padding > 0:
From afa at codespeak.net Thu Oct 28 23:41:23 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 23:41:23 +0200 (CEST)
Subject: [pypy-svn] r78425 - in pypy/branch/fast-forward/pypy/objspace/std:
. test
Message-ID: <20101028214123.786F036E076@codespeak.net>
Author: afa
Date: Thu Oct 28 23:41:22 2010
New Revision: 78425
Modified:
pypy/branch/fast-forward/pypy/objspace/std/strutil.py
pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py
Log:
'0x' is not a correct hexadecimal number.
Modified: pypy/branch/fast-forward/pypy/objspace/std/strutil.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/strutil.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/strutil.py Thu Oct 28 23:41:22 2010
@@ -62,14 +62,14 @@
raise ParseStringError, "%s() base must be >= 2 and <= 36" % (fname,)
self.base = base
- if not s:
- self.error()
if base == 16 and (s.startswith('0x') or s.startswith('0X')):
s = s[2:]
if base == 8 and (s.startswith('0o') or s.startswith('0O')):
s = s[2:]
if base == 2 and (s.startswith('0b') or s.startswith('0B')):
s = s[2:]
+ if not s:
+ self.error()
self.s = s
self.n = len(s)
self.i = 0
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py Thu Oct 28 23:41:22 2010
@@ -88,6 +88,8 @@
raises(ParseStringError, string_to_int, s+' ')
raises(ParseStringError, string_to_int, '+'+s)
raises(ParseStringError, string_to_int, '-'+s)
+ raises(ParseStringError, string_to_int, '0x', 16)
+ raises(ParseStringError, string_to_int, '-0x', 16)
def test_string_to_int_overflow(self):
import sys
From afa at codespeak.net Thu Oct 28 23:42:49 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Thu, 28 Oct 2010 23:42:49 +0200 (CEST)
Subject: [pypy-svn] r78426 - pypy/branch/fast-forward/pypy/objspace/std/test
Message-ID: <20101028214249.A0D32282C1E@codespeak.net>
Author: afa
Date: Thu Oct 28 23:42:48 2010
New Revision: 78426
Modified:
pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py
Log:
update: '0x' is *no more* a correct hexadecimal number
Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py
==============================================================================
--- pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py (original)
+++ pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py Thu Oct 28 23:42:48 2010
@@ -48,10 +48,6 @@
('0x10', 0, 16),
('0XE', 0, 14),
('0', 0, 0),
- ('0x', 0, 0), # according to CPython so far
- ('0X', 0, 0), # " "
- ('0x', 16, 0), # " "
- ('0X', 16, 0), # " "
('0b11', 2, 3),
('0B10', 2, 2),
('0o77', 8, 63),
From afa at codespeak.net Fri Oct 29 00:35:44 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 00:35:44 +0200 (CEST)
Subject: [pypy-svn] r78427 -
pypy/branch/fast-forward/lib-python/modified-2.7.0/test
Message-ID: <20101028223544.004CB36E076@codespeak.net>
Author: afa
Date: Fri Oct 29 00:35:43 2010
New Revision: 78427
Added:
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_weakref.py
- copied, changed from r78426, pypy/branch/fast-forward/lib-python/2.7.0/test/test_weakref.py
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_weakset.py
- copied, changed from r78426, pypy/branch/fast-forward/lib-python/2.7.0/test/test_weakset.py
Log:
Fix many weakref tests by forcing garbage collection when needed
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_weakref.py (from r78426, pypy/branch/fast-forward/lib-python/2.7.0/test/test_weakref.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/test/test_weakref.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_weakref.py Fri Oct 29 00:35:43 2010
@@ -1,4 +1,3 @@
-import gc
import sys
import unittest
import UserList
@@ -6,6 +5,7 @@
import operator
from test import test_support
+from test.test_support import gc_collect
# Used in ReferencesTestCase.test_ref_created_during_del() .
ref_from_del = None
@@ -70,6 +70,7 @@
ref1 = weakref.ref(o, self.callback)
ref2 = weakref.ref(o, self.callback)
del o
+ gc_collect()
self.assertTrue(ref1() is None,
"expected reference to be invalidated")
self.assertTrue(ref2() is None,
@@ -101,13 +102,16 @@
ref1 = weakref.proxy(o, self.callback)
ref2 = weakref.proxy(o, self.callback)
del o
+ gc_collect()
def check(proxy):
proxy.bar
self.assertRaises(weakref.ReferenceError, check, ref1)
self.assertRaises(weakref.ReferenceError, check, ref2)
- self.assertRaises(weakref.ReferenceError, bool, weakref.proxy(C()))
+ ref3 = weakref.proxy(C())
+ gc_collect()
+ self.assertRaises(weakref.ReferenceError, bool, ref3)
self.assertTrue(self.cbcalled == 2)
def check_basic_ref(self, factory):
@@ -124,6 +128,7 @@
o = factory()
ref = weakref.ref(o, self.callback)
del o
+ gc_collect()
self.assertTrue(self.cbcalled == 1,
"callback did not properly set 'cbcalled'")
self.assertTrue(ref() is None,
@@ -148,6 +153,7 @@
self.assertTrue(weakref.getweakrefcount(o) == 2,
"wrong weak ref count for object")
del proxy
+ gc_collect()
self.assertTrue(weakref.getweakrefcount(o) == 1,
"wrong weak ref count for object after deleting proxy")
@@ -325,6 +331,7 @@
"got wrong number of weak reference objects")
del ref1, ref2, proxy1, proxy2
+ gc_collect()
self.assertTrue(weakref.getweakrefcount(o) == 0,
"weak reference objects not unlinked from"
" referent when discarded.")
@@ -338,6 +345,7 @@
ref1 = weakref.ref(o, self.callback)
ref2 = weakref.ref(o, self.callback)
del ref1
+ gc_collect()
self.assertTrue(weakref.getweakrefs(o) == [ref2],
"list of refs does not match")
@@ -345,10 +353,12 @@
ref1 = weakref.ref(o, self.callback)
ref2 = weakref.ref(o, self.callback)
del ref2
+ gc_collect()
self.assertTrue(weakref.getweakrefs(o) == [ref1],
"list of refs does not match")
del ref1
+ gc_collect()
self.assertTrue(weakref.getweakrefs(o) == [],
"list of refs not cleared")
@@ -400,13 +410,11 @@
# when the second attempt to remove the instance from the "list
# of all objects" occurs.
- import gc
-
class C(object):
pass
c = C()
- wr = weakref.ref(c, lambda ignore: gc.collect())
+ wr = weakref.ref(c, lambda ignore: gc_collect())
del c
# There endeth the first part. It gets worse.
@@ -414,7 +422,7 @@
c1 = C()
c1.i = C()
- wr = weakref.ref(c1.i, lambda ignore: gc.collect())
+ wr = weakref.ref(c1.i, lambda ignore: gc_collect())
c2 = C()
c2.c1 = c1
@@ -430,8 +438,6 @@
del c2
def test_callback_in_cycle_1(self):
- import gc
-
class J(object):
pass
@@ -467,11 +473,9 @@
# search II.__mro__, but that's NULL. The result was a segfault in
# a release build, and an assert failure in a debug build.
del I, J, II
- gc.collect()
+ gc_collect()
def test_callback_in_cycle_2(self):
- import gc
-
# This is just like test_callback_in_cycle_1, except that II is an
# old-style class. The symptom is different then: an instance of an
# old-style class looks in its own __dict__ first. 'J' happens to
@@ -496,11 +500,9 @@
I.wr = weakref.ref(J, I.acallback)
del I, J, II
- gc.collect()
+ gc_collect()
def test_callback_in_cycle_3(self):
- import gc
-
# This one broke the first patch that fixed the last two. In this
# case, the objects reachable from the callback aren't also reachable
# from the object (c1) *triggering* the callback: you can get to
@@ -520,11 +522,9 @@
c2.wr = weakref.ref(c1, c2.cb)
del c1, c2
- gc.collect()
+ gc_collect()
def test_callback_in_cycle_4(self):
- import gc
-
# Like test_callback_in_cycle_3, except c2 and c1 have different
# classes. c2's class (C) isn't reachable from c1 then, so protecting
# objects reachable from the dying object (c1) isn't enough to stop
@@ -548,11 +548,9 @@
c2.wr = weakref.ref(c1, c2.cb)
del c1, c2, C, D
- gc.collect()
+ gc_collect()
def test_callback_in_cycle_resurrection(self):
- import gc
-
# Do something nasty in a weakref callback: resurrect objects
# from dead cycles. For this to be attempted, the weakref and
# its callback must also be part of the cyclic trash (else the
@@ -583,7 +581,7 @@
del c1, c2, C # make them all trash
self.assertEqual(alist, []) # del isn't enough to reclaim anything
- gc.collect()
+ gc_collect()
# c1.wr and c2.wr were part of the cyclic trash, so should have
# been cleared without their callbacks executing. OTOH, the weakref
# to C is bound to a function local (wr), and wasn't trash, so that
@@ -593,12 +591,10 @@
self.assertEqual(wr(), None)
del alist[:]
- gc.collect()
+ gc_collect()
self.assertEqual(alist, [])
def test_callbacks_on_callback(self):
- import gc
-
# Set up weakref callbacks *on* weakref callbacks.
alist = []
def safe_callback(ignore):
@@ -626,12 +622,12 @@
del callback, c, d, C
self.assertEqual(alist, []) # del isn't enough to clean up cycles
- gc.collect()
+ gc_collect()
self.assertEqual(alist, ["safe_callback called"])
self.assertEqual(external_wr(), None)
del alist[:]
- gc.collect()
+ gc_collect()
self.assertEqual(alist, [])
def test_gc_during_ref_creation(self):
@@ -641,9 +637,11 @@
self.check_gc_during_creation(weakref.proxy)
def check_gc_during_creation(self, makeref):
- thresholds = gc.get_threshold()
- gc.set_threshold(1, 1, 1)
- gc.collect()
+ if test_support.check_impl_detail():
+ import gc
+ thresholds = gc.get_threshold()
+ gc.set_threshold(1, 1, 1)
+ gc_collect()
class A:
pass
@@ -663,7 +661,8 @@
weakref.ref(referenced, callback)
finally:
- gc.set_threshold(*thresholds)
+ if test_support.check_impl_detail():
+ gc.set_threshold(*thresholds)
def test_ref_created_during_del(self):
# Bug #1377858
@@ -683,7 +682,7 @@
r = weakref.ref(Exception)
self.assertRaises(TypeError, r.__init__, 0, 0, 0, 0, 0)
# No exception should be raised here
- gc.collect()
+ gc_collect()
def test_classes(self):
# Check that both old-style classes and new-style classes
@@ -696,12 +695,12 @@
weakref.ref(int)
a = weakref.ref(A, l.append)
A = None
- gc.collect()
+ gc_collect()
self.assertEqual(a(), None)
self.assertEqual(l, [a])
b = weakref.ref(B, l.append)
B = None
- gc.collect()
+ gc_collect()
self.assertEqual(b(), None)
self.assertEqual(l, [a, b])
@@ -722,6 +721,7 @@
self.assertTrue(mr.called)
self.assertEqual(mr.value, 24)
del o
+ gc_collect()
self.assertTrue(mr() is None)
self.assertTrue(mr.called)
@@ -839,15 +839,18 @@
del items1, items2
self.assertTrue(len(dict) == self.COUNT)
del objects[0]
+ gc_collect()
self.assertTrue(len(dict) == (self.COUNT - 1),
"deleting object did not cause dictionary update")
del objects, o
+ gc_collect()
self.assertTrue(len(dict) == 0,
"deleting the values did not clear the dictionary")
# regression on SF bug #447152:
dict = weakref.WeakValueDictionary()
self.assertRaises(KeyError, dict.__getitem__, 1)
dict[2] = C()
+ gc_collect()
self.assertRaises(KeyError, dict.__getitem__, 2)
def test_weak_keys(self):
@@ -868,9 +871,11 @@
del items1, items2
self.assertTrue(len(dict) == self.COUNT)
del objects[0]
+ gc_collect()
self.assertTrue(len(dict) == (self.COUNT - 1),
"deleting object did not cause dictionary update")
del objects, o
+ gc_collect()
self.assertTrue(len(dict) == 0,
"deleting the keys did not clear the dictionary")
o = Object(42)
@@ -1137,6 +1142,7 @@
for o in objs:
count += 1
del d[o]
+ gc_collect()
self.assertEqual(len(d), 0)
self.assertEqual(count, 2)
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_weakset.py (from r78426, pypy/branch/fast-forward/lib-python/2.7.0/test/test_weakset.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/test/test_weakset.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_weakset.py Fri Oct 29 00:35:43 2010
@@ -57,6 +57,7 @@
self.assertEqual(len(self.s), len(self.d))
self.assertEqual(len(self.fs), 1)
del self.obj
+ test_support.gc_collect()
self.assertEqual(len(self.fs), 0)
def test_contains(self):
@@ -65,6 +66,7 @@
self.assertRaises(TypeError, self.s.__contains__, [[]])
self.assertIn(self.obj, self.fs)
del self.obj
+ test_support.gc_collect()
self.assertNotIn(SomeClass('F'), self.fs)
def test_union(self):
@@ -203,6 +205,7 @@
self.assertEqual(self.s, dup)
self.assertRaises(TypeError, self.s.add, [])
self.fs.add(Foo())
+ test_support.gc_collect()
self.assertTrue(len(self.fs) == 1)
self.fs.add(self.obj)
self.assertTrue(len(self.fs) == 1)
From afa at codespeak.net Fri Oct 29 00:47:54 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 00:47:54 +0200 (CEST)
Subject: [pypy-svn] r78429 -
pypy/branch/fast-forward/lib-python/modified-2.7.0/test
Message-ID: <20101028224754.07E07282BE8@codespeak.net>
Author: afa
Date: Fri Oct 29 00:47:53 2010
New Revision: 78429
Added:
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_tempfile.py
- copied, changed from r78426, pypy/branch/fast-forward/lib-python/2.7.0/test/test_tempfile.py
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/tf_inherit_check.py
- copied unchanged from r78426, pypy/branch/fast-forward/lib-python/2.7.0/test/tf_inherit_check.py
Log:
More fixes with gc.collect()
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_tempfile.py (from r78426, pypy/branch/fast-forward/lib-python/2.7.0/test/test_tempfile.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/test/test_tempfile.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_tempfile.py Fri Oct 29 00:47:53 2010
@@ -244,6 +244,7 @@
dir = tempfile.mkdtemp()
try:
self.do_create(dir=dir).write("blat")
+ test_support.gc_collect()
finally:
os.rmdir(dir)
@@ -528,12 +529,15 @@
self.do_create(suf="b")
self.do_create(pre="a", suf="b")
self.do_create(pre="aa", suf=".txt")
+ test_support.gc_collect()
def test_many(self):
# mktemp can choose many usable file names (stochastic)
extant = range(TEST_FILES)
for i in extant:
extant[i] = self.do_create(pre="aa")
+ del extant
+ test_support.gc_collect()
## def test_warning(self):
## # mktemp issues a warning when used
From afa at codespeak.net Fri Oct 29 01:01:08 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 01:01:08 +0200 (CEST)
Subject: [pypy-svn] r78430 - pypy/branch/fast-forward/pypy/module/pyexpat
Message-ID: <20101028230108.85CD7282BE8@codespeak.net>
Author: afa
Date: Fri Oct 29 01:01:06 2010
New Revision: 78430
Modified:
pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py
Log:
Format pyexpat error messages like CPython
Modified: pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py (original)
+++ pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py Fri Oct 29 01:01:06 2010
@@ -505,7 +505,7 @@
err = rffi.charp2strn(XML_ErrorString(code), 200)
lineno = XML_GetCurrentLineNumber(self.itself)
colno = XML_GetCurrentColumnNumber(self.itself)
- msg = "%s: line: %d, column: %d" % (err, lineno, colno)
+ msg = "%s: line %d, column %d" % (err, lineno, colno)
w_module = space.getbuiltinmodule('pyexpat')
w_errorcls = space.getattr(w_module, space.wrap('error'))
w_error = space.call_function(
From afa at codespeak.net Fri Oct 29 01:15:35 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 01:15:35 +0200 (CEST)
Subject: [pypy-svn] r78431 - in
pypy/branch/fast-forward/pypy/module/pyexpat: . test
Message-ID: <20101028231535.DE65E36E076@codespeak.net>
Author: afa
Date: Fri Oct 29 01:15:34 2010
New Revision: 78431
Modified:
pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py
pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py
Log:
pyexpat: add parser.buffer_size
Modified: pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py (original)
+++ pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py Fri Oct 29 01:15:34 2010
@@ -527,6 +527,16 @@
def descr_ErrorByteIndex(space, self):
return space.wrap(XML_GetErrorByteIndex(self.itself))
+ def get_buffer_size(space, self):
+ return space.wrap(self.buffer_size)
+ def set_buffer_size(space, self, w_value):
+ value = space.int_w(w_value)
+ if value <= 0:
+ raise OperationError(space.w_ValueError, space.wrap(
+ "buffer_size must be greater than zero"))
+ self.flush_character_buffer(space)
+ self.buffer_size = value
+
def get_buffer_text(space, self):
return space.wrap(self.buffer_w is not None)
def set_buffer_text(space, self, w_value):
@@ -561,6 +571,9 @@
ordered_attributes = bool_property('ordered_attributes', W_XMLParserType),
specified_attributes = bool_property('specified_attributes', W_XMLParserType),
intern = GetSetProperty(W_XMLParserType.get_intern, cls=W_XMLParserType),
+ buffer_size = GetSetProperty(W_XMLParserType.get_buffer_size,
+ W_XMLParserType.set_buffer_size,
+ cls=W_XMLParserType),
buffer_text = GetSetProperty(W_XMLParserType.get_buffer_text,
W_XMLParserType.set_buffer_text, cls=W_XMLParserType),
Modified: pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py (original)
+++ pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py Fri Oct 29 01:15:34 2010
@@ -15,3 +15,9 @@
assert res == 1
raises(pyexpat.ExpatError, p.Parse, "3")
+
+ def test_set_buffersize(self):
+ import pyexpat
+ p = pyexpat.ParserCreate()
+ p.buffer_size = 150
+ assert p.buffer_size == 150
From agaynor at codespeak.net Fri Oct 29 06:49:43 2010
From: agaynor at codespeak.net (agaynor at codespeak.net)
Date: Fri, 29 Oct 2010 06:49:43 +0200 (CEST)
Subject: [pypy-svn] r78433 - pypy/trunk/pypy/doc/config
Message-ID: <20101029044943.0513A282BE8@codespeak.net>
Author: agaynor
Date: Fri Oct 29 06:49:41 2010
New Revision: 78433
Added:
pypy/trunk/pypy/doc/config/objspace.usemodules._bisect.txt
Log:
Added docs for the usemodule _bisect option.
Added: pypy/trunk/pypy/doc/config/objspace.usemodules._bisect.txt
==============================================================================
--- (empty file)
+++ pypy/trunk/pypy/doc/config/objspace.usemodules._bisect.txt Fri Oct 29 06:49:41 2010
@@ -0,0 +1,4 @@
+Use the '_bisect' module.
+Used, optionally, by the 'bisect' standard lib module. This module is expected to be working and is included by default.
+
+
From david at codespeak.net Fri Oct 29 09:36:55 2010
From: david at codespeak.net (david at codespeak.net)
Date: Fri, 29 Oct 2010 09:36:55 +0200 (CEST)
Subject: [pypy-svn] r78438 - in
pypy/branch/arm-backend/pypy/jit/backend/arm: . test
Message-ID: <20101029073655.8EA4336E076@codespeak.net>
Author: david
Date: Fri Oct 29 09:36:53 2010
New Revision: 78438
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/arch.py
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py
pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py
Log:
Implement int_mul, int_floordiv and int_mod operations.
- Added support to codebuilder to allocate new memory for instructions
- Implement int_div and int_mod as functions called from assembler, because doesn't provide instructions for them
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/arch.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/arch.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/arch.py Fri Oct 29 09:36:53 2010
@@ -1,2 +1,20 @@
+from pypy.rpython.lltypesystem import lltype
+
FUNC_ALIGN=8
WORD=4
+
+arm_int_div_sign = lltype.Ptr(lltype.FuncType([lltype.Signed, lltype.Signed], lltype.Signed))
+arm_int_mod_sign = arm_int_div_sign
+
+def arm_int_div(a, b):
+ return int(a/float(b))
+
+def arm_int_mod(a, b):
+ sign = 1
+ if a < 0:
+ a = -1 * a
+ sign = -1
+ if b < 0:
+ b = -1 * b
+ res = a % b
+ return sign * res
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Fri Oct 29 09:36:53 2010
@@ -3,7 +3,7 @@
from pypy.jit.backend.arm import registers as r
from pypy.jit.backend.arm.arch import WORD, FUNC_ALIGN
from pypy.jit.backend.arm.codebuilder import ARMv7Builder, ARMv7InMemoryBuilder
-from pypy.jit.backend.arm.regalloc import ARMRegisterManager
+from pypy.jit.backend.arm.regalloc import ARMRegisterManager, ARMFrameManager
from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity
from pypy.jit.metainterp.history import ConstInt, BoxInt, Box, BasicFailDescr
from pypy.jit.metainterp.resoperation import rop
@@ -25,6 +25,7 @@
self.fail_boxes_int = values_array(lltype.Signed, failargs_limit)
self._debug_asm = True
+ self._exit_code_addr = self.mc.curraddr()
self._gen_exit_path()
self.align()
self.mc._start_addr = self.mc.curraddr()
@@ -136,7 +137,7 @@
n = self.cpu.get_fail_descr_number(op.getdescr())
self.encode32(mem, j+1, n)
self.mc.gen_load_int(r.lr.value, memaddr, cond=fcond)
- self.mc.gen_load_int(reg.value, self.mc.baseaddr(), cond=fcond)
+ self.mc.gen_load_int(reg.value, self._exit_code_addr, cond=fcond)
self.mc.MOV_rr(r.pc.value, reg.value, cond=fcond)
# This register is used for patching when assembling a bridge
@@ -168,7 +169,7 @@
# cpu interface
def assemble_loop(self, inputargs, operations, looptoken):
longevity = compute_vars_longevity(inputargs, operations)
- regalloc = ARMRegisterManager(longevity, assembler=self.mc)
+ regalloc = ARMRegisterManager(longevity, assembler=self.mc, frame_manager=ARMFrameManager())
self.align()
loop_start=self.mc.curraddr()
self.gen_func_prolog()
@@ -189,7 +190,7 @@
def assemble_bridge(self, faildescr, inputargs, operations):
enc = rffi.cast(rffi.CCHARP, faildescr._failure_recovery_code)
longevity = compute_vars_longevity(inputargs, operations)
- regalloc = ARMRegisterManager(longevity, assembler=self.mc)
+ regalloc = ARMRegisterManager(longevity, assembler=self.mc, frame_manager=ARMFrameManager())
regalloc.update_bindings(enc, inputargs)
bridge_head = self.mc.curraddr()
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Fri Oct 29 09:36:53 2010
@@ -1,10 +1,13 @@
from pypy.jit.backend.arm import conditions as cond
from pypy.jit.backend.arm import registers as reg
-from pypy.jit.backend.arm.arch import WORD
+from pypy.jit.backend.arm.arch import (WORD, FUNC_ALIGN, arm_int_div,
+ arm_int_div_sign, arm_int_mod_sign, arm_int_mod)
from pypy.jit.backend.arm.instruction_builder import define_instructions
from pypy.rlib.rmmap import alloc, PTR
+from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.lltypesystem import lltype, rffi
+from pypy.jit.metainterp.history import ConstInt, BoxInt, Box, BasicFailDescr
class AbstractARMv7Builder(object):
def _init(self, data, map_size):
@@ -40,6 +43,26 @@
def BKPT(self, cond=cond.AL):
self.write32(cond << 28 | 0x1200070)
+ def DIV(self, cond=cond.AL):
+ """Generates a call to a helper function used for division, takes its
+ arguments in r0 and r1, result is placed in r0"""
+ self.PUSH(range(2, 12), cond=cond)
+ div_addr = rffi.cast(lltype.Signed, llhelper(arm_int_div_sign, arm_int_div))
+ self.gen_load_int(reg.r2.value, div_addr, cond=cond)
+ self.gen_load_int(reg.lr.value, self.curraddr()+self.size_of_gen_load_int+WORD, cond=cond)
+ self.MOV_rr(reg.pc.value, reg.r2.value, cond=cond)
+ self.LDM(reg.sp.value, range(2, 12), w=1, cond=cond) # XXX Replace with POP instr. someday
+
+ def MOD(self, cond=cond.AL):
+ """Generate a call to a helper function used for modulo, takes its
+ arguments in r0 and r1, result is placed in r0"""
+ self.PUSH(range(2, 12), cond=cond)
+ mod_addr = rffi.cast(lltype.Signed, llhelper(arm_int_mod_sign, arm_int_mod))
+ self.gen_load_int(reg.r2.value, mod_addr, cond=cond)
+ self.gen_load_int(reg.lr.value, self.curraddr()+self.size_of_gen_load_int+WORD, cond=cond)
+ self.MOV_rr(reg.pc.value, reg.r2.value, cond=cond)
+ self.LDM(reg.sp.value, range(2, 12), w=1, cond=cond) # XXX Replace with POP instr. someday
+
def _encode_reg_list(self, instr, regs):
for reg in regs:
instr |= 0x1 << reg
@@ -73,16 +96,22 @@
"""r is the register number, value is the value to be loaded to the
register"""
assert r != reg.ip.value, 'ip is used to load int'
- self.MOV_ri(r, (value & 0xFF), cond=cond)
ip = reg.ip.value
+ self.MOV_ri(r, (value & 0xFF), cond=cond)
for offset in range(8, 25, 8):
t = (value >> offset) & 0xFF
- #if t == 0:
- # continue
self.MOV_ri(ip, t, cond=cond)
self.ORR_rr(r, r, ip, offset, cond=cond)
+ # regalloc support
+ def regalloc_mov(self, prev_loc, loc):
+ if isinstance(prev_loc, ConstInt):
+ # XXX check size of imm for current instr
+ self.gen_load_int(loc.value, prev_loc.getint())
+ else:
+ self.MOV_rr(loc.value, prev_loc.value)
+
class ARMv7InMemoryBuilder(AbstractARMv7Builder):
def __init__(self, start, end):
map_size = end - start
@@ -96,5 +125,28 @@
data = alloc(map_size)
self._pos = 0
self._init(data, map_size)
+ self.checks = True
+ self.n_data=0
+
+ _space_for_jump = 10 * WORD
+ def writechar(self, char):
+ if self.checks and not self._pos < self._size - self._space_for_jump:
+ self.checks = False
+ self._add_more_mem()
+ self.checks = True
+ assert self._pos < self._size
+ AbstractARMv7Builder.writechar(self, char)
+
+ def _add_more_mem(self):
+ new_mem = alloc(self._size)
+ new_mem_addr = rffi.cast(lltype.Signed, new_mem)
+ self.PUSH([reg.r0.value])
+ self.gen_load_int(reg.r0.value, new_mem_addr)
+ self.MOV_rr(reg.pc.value, reg.r0.value)
+ self._dump_trace('data%d.asm' % self.n_data)
+ self.n_data+=1
+ self._data = new_mem
+ self._pos = 0
+ self.LDM(reg.sp.value, [reg.r0.value], w=1) # XXX Replace with POP instr. someday
define_instructions(AbstractARMv7Builder)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py Fri Oct 29 09:36:53 2010
@@ -5,3 +5,6 @@
def __init__(self, value):
self.value = value
+
+ def __repr__(self):
+ return 'r%d' % self.value
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Fri Oct 29 09:36:53 2010
@@ -1,7 +1,8 @@
from pypy.jit.backend.arm import conditions as c
from pypy.jit.backend.arm import locations
from pypy.jit.backend.arm import registers as r
-from pypy.jit.backend.arm.arch import WORD, FUNC_ALIGN
+from pypy.jit.backend.arm.arch import (WORD, FUNC_ALIGN, arm_int_div,
+ arm_int_div_sign, arm_int_mod_sign, arm_int_mod)
from pypy.jit.backend.arm.codebuilder import ARMv7Builder, ARMv7InMemoryBuilder
from pypy.jit.backend.arm.regalloc import ARMRegisterManager
from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity
@@ -68,7 +69,44 @@
return fcond
def emit_op_int_mul(self, op, regalloc, fcond):
- import pdb; pdb.set_trace()
+ res = regalloc.try_allocate_reg(op.result)
+ reg1 = self._put_in_reg(op.getarg(0), regalloc)
+ reg2 = self._put_in_reg(op.getarg(1), regalloc)
+ self.mc.MUL(res.value, reg1.value, reg2.value)
+ regalloc.possibly_free_var(reg1)
+ regalloc.possibly_free_var(reg2)
+ return fcond
+
+ def emit_op_int_floordiv(self, op, regalloc, fcond):
+ arg1 = regalloc.make_sure_var_in_reg(op.getarg(0), selected_reg=r.r0)
+ arg2 = regalloc.make_sure_var_in_reg(op.getarg(1), selected_reg=r.r1)
+ assert arg1 == r.r0
+ assert arg2 == r.r1
+ res = regalloc.try_allocate_reg(op.result)
+ self.mc.DIV(fcond)
+ self.mc.MOV_rr(res.value, r.r0.value, cond=fcond)
+ regalloc.possibly_free_vars_for_op(op)
+ return fcond
+
+ def emit_op_int_mod(self, op, regalloc, fcond):
+ arg1 = regalloc.make_sure_var_in_reg(op.getarg(0), selected_reg=r.r0)
+ arg2 = regalloc.make_sure_var_in_reg(op.getarg(1), selected_reg=r.r1)
+ assert arg1 == r.r0
+ assert arg2 == r.r1
+ res = regalloc.try_allocate_reg(op.result)
+ self.mc.MOD(fcond)
+ self.mc.MOV_rr(res.value, r.r0.value, cond=fcond)
+ regalloc.possibly_free_vars_for_op(op)
+ return fcond
+
+ def _put_in_reg(self, box, regalloc):
+ if isinstance(box, ConstInt):
+ t = Box()
+ reg = regalloc.try_allocate_reg(t)
+ self.mc.gen_load_int(reg.value, box.getint())
+ else:
+ reg = regalloc.try_allocate_reg(box)
+ return reg
class GuardOpAssembler(object):
_mixin_ = True
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Fri Oct 29 09:36:53 2010
@@ -19,6 +19,9 @@
self.try_allocate_reg(inputargs[i], r.all_regs[ord(enc[j])])
j += 1
+ def convert_to_imm(self, c):
+ return c
+
class ARMFrameManager(FrameManager):
@staticmethod
def frame_pos(loc, type):
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py Fri Oct 29 09:36:53 2010
@@ -13,6 +13,7 @@
addr = asm.mc._start_addr
assert addr % 8 == 0
func = rffi.cast(lltype.Ptr(BOOTSTRAP_TP), addr)
+ asm.mc._dump_trace('test.asm')
return func()
def skip_unless_arm():
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py Fri Oct 29 09:36:53 2010
@@ -1,3 +1,4 @@
+from pypy.jit.backend.arm.arch import arm_int_div, arm_int_div_sign
from pypy.jit.backend.arm import conditions as c
from pypy.jit.backend.arm import registers as r
from pypy.jit.backend.arm.arch import WORD
@@ -37,6 +38,18 @@
self.a.gen_func_epilog()
assert run_asm(self.a) == -123
+ def test_load_neg_int_to_reg(self):
+ self.a.gen_func_prolog()
+ self.a.mc.gen_load_int(r.r0.value, -110)
+ self.a.gen_func_epilog()
+ assert run_asm(self.a) == -110
+
+ def test_load_neg_int_to_reg2(self):
+ self.a.gen_func_prolog()
+ self.a.mc.gen_load_int(r.r0.value, -3)
+ self.a.gen_func_epilog()
+ assert run_asm(self.a) == -3
+
def test_or(self):
self.a.gen_func_prolog()
@@ -108,6 +121,47 @@
self.a.gen_func_epilog()
assert run_asm(self.a) == 133
+ def test_division(self):
+ self.a.gen_func_prolog()
+ self.a.mc.MOV_ri(r.r0.value, 123)
+ self.a.mc.MOV_ri(r.r1.value, 2)
+
+ # call to div
+ self.a.mc.PUSH(range(2, 12))
+ div_addr = rffi.cast(lltype.Signed, llhelper(arm_int_div_sign, arm_int_div))
+ self.a.mc.gen_load_int(r.r10.value, div_addr)
+ self.a.mc.gen_load_int(r.lr.value, self.a.mc.curraddr()+self.a.mc.size_of_gen_load_int+WORD)
+ self.a.mc.MOV_rr(r.pc.value, r.r10.value)
+ self.a.mc.LDM(r.sp.value, range(2, 12), w=1) # XXX Replace with POP instr. someday
+ self.a.gen_func_epilog()
+ assert run_asm(self.a) == 61
+
+ def test_DIV(self):
+ self.a.gen_func_prolog()
+ self.a.mc.MOV_ri(r.r0.value, 123)
+ self.a.mc.MOV_ri(r.r1.value, 2)
+ self.a.mc.DIV()
+ self.a.gen_func_epilog()
+ assert run_asm(self.a) == 61
+
+ def test_DIV2(self):
+ self.a.gen_func_prolog()
+ self.a.mc.gen_load_int(r.r0.value, -110)
+ self.a.mc.gen_load_int(r.r1.value, 3)
+ self.a.mc.DIV()
+ self.a.gen_func_epilog()
+ assert run_asm(self.a) == -36
+
+ def test_DIV3(self):
+ self.a.gen_func_prolog()
+ self.a.mc.gen_load_int(r.r8.value, 110)
+ self.a.mc.gen_load_int(r.r9.value, -3)
+ self.a.mc.MOV_rr(r.r0.value, r.r8.value)
+ self.a.mc.MOV_rr(r.r1.value, r.r9.value)
+ self.a.mc.DIV()
+ self.a.gen_func_epilog()
+ assert run_asm(self.a) == -36
+
def callme(inp):
i = inp + 10
return i
From afa at codespeak.net Fri Oct 29 10:16:24 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 10:16:24 +0200 (CEST)
Subject: [pypy-svn] r78442 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101029081624.996F436E0C1@codespeak.net>
Author: afa
Date: Fri Oct 29 10:16:23 2010
New Revision: 78442
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py
pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py
Log:
Bah. Test and fix
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py Fri Oct 29 10:16:23 2010
@@ -100,7 +100,7 @@
if pos < 0:
raise OperationError(space.w_ValueError, space.wrap(
"negative seek value"))
- if whence == 1:
+ elif whence == 1:
if pos > sys.maxint - self.pos:
raise OperationError(space.w_OverflowError, space.wrap(
"new position too large"))
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py Fri Oct 29 10:16:23 2010
@@ -25,3 +25,4 @@
assert f.tell() == 0
assert f.seek(-1, 2) == 4
assert f.tell() == 4
+ assert f.seek(0) == 0
From cfbolz at codespeak.net Fri Oct 29 10:18:37 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 29 Oct 2010 10:18:37 +0200 (CEST)
Subject: [pypy-svn] r78443 - pypy/extradoc/sprintinfo/ddorf2010
Message-ID: <20101029081837.C077236E0CB@codespeak.net>
Author: cfbolz
Date: Fri Oct 29 10:18:36 2010
New Revision: 78443
Modified:
pypy/extradoc/sprintinfo/ddorf2010/planning.txt
Log:
(all): planning for today
Modified: pypy/extradoc/sprintinfo/ddorf2010/planning.txt
==============================================================================
--- pypy/extradoc/sprintinfo/ddorf2010/planning.txt (original)
+++ pypy/extradoc/sprintinfo/ddorf2010/planning.txt Fri Oct 29 10:18:36 2010
@@ -1,29 +1,32 @@
people present:
Anto
- David
Carl Friedrich
Ronny
Armin
+ H?kan
Lukas
+coming later?:
+ Sven
+
+not there today:
+ David
topics:
-- get mmap to work on Beagleboard DONE
-- bridges in the ARM backend SORT OF DONE
-- add spilling and location support IN PROGRESS (David, Armin around)
-- ARM int trace operations (David)
-- hg conversion WE HAVE MORE OF A PLAN (Anto, Ronny)
-- memory benchmarks DONE
-- refine heap analysis CONFUSING (Carl Friedrich, Lukas)
-- (Re)write _bisect module (Armin)
+- add spilling and location support IN PROGRESS
+- ARM int trace operations SOME PROGRESS
+- hg conversion WE HAVE AN EVEN DIFFERENT PLAN (Anto, Ronny)
+- refine heap analysis GETTING BETTER (Carl Friedrich, Lukas)
+- (Re)write _bisect module DONE
- buildbot improvements, locks, email on failing tests
- run pypy-c-nojit in the benchmarks
- make it possible to run benchmark on branches
- understand problems of mapdict (Carl Friedrich, Armin)
- rewrite ctypes to use _ffi
-- can classes with non-type metaclasses use typeversion? DONE
-
+- fix the remaining bugs on the jit-unroll-loops branch (Armin, H?kan)
+- have an implementation of Python longs with C long longs
+- work in Pyrolog (Sven, Carl Friedrich around)
scheduled discussions:
From fijal at codespeak.net Fri Oct 29 10:34:33 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Fri, 29 Oct 2010 10:34:33 +0200 (CEST)
Subject: [pypy-svn] r78444 - pypy/trunk/pypy/jit/metainterp
Message-ID: <20101029083433.C159236E0CB@codespeak.net>
Author: fijal
Date: Fri Oct 29 10:34:32 2010
New Revision: 78444
Modified:
pypy/trunk/pypy/jit/metainterp/compile.py
Log:
kill silly pass
Modified: pypy/trunk/pypy/jit/metainterp/compile.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/compile.py (original)
+++ pypy/trunk/pypy/jit/metainterp/compile.py Fri Oct 29 10:34:32 2010
@@ -123,7 +123,6 @@
if not we_are_translated():
show_loop(metainterp_sd)
TreeLoop.check_consistency_of(inputargs, operations)
- pass
metainterp_sd.profiler.start_backend()
debug_start("jit-backend")
try:
From afa at codespeak.net Fri Oct 29 10:42:14 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 10:42:14 +0200 (CEST)
Subject: [pypy-svn] r78445 - in pypy/branch/fast-forward/pypy/module/_io: .
test
Message-ID: <20101029084214.2259236E076@codespeak.net>
Author: afa
Date: Fri Oct 29 10:42:13 2010
New Revision: 78445
Modified:
pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
Log:
implement abstract tell() == seek(0, 1)
Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Fri Oct 29 10:42:13 2010
@@ -74,6 +74,10 @@
space.wrap("I/O operation on closed file"))
@unwrap_spec('self', ObjSpace)
+ def tell_w(self, space):
+ return space.call_method(self, "seek", space.wrap(0), space.wrap(1))
+
+ @unwrap_spec('self', ObjSpace)
def enter_w(self, space):
self._check_closed(space)
return space.wrap(self)
@@ -225,6 +229,7 @@
next = interp2app(W_IOBase.next_w),
close = interp2app(W_IOBase.close_w),
flush = interp2app(W_IOBase.flush_w),
+ tell = interp2app(W_IOBase.tell_w),
isatty = interp2app(W_IOBase.isatty_w),
readable = interp2app(W_IOBase.readable_w),
writable = interp2app(W_IOBase.writable_w),
Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (original)
+++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Fri Oct 29 10:42:13 2010
@@ -71,6 +71,13 @@
import gc; gc.collect()
assert record == [1, 2, 3]
+ def test_tell(self):
+ import io
+ class MyIO(io.IOBase):
+ def seek(self, pos, whence=0):
+ return 42
+ assert MyIO().tell() == 42
+
class AppTestOpen:
def setup_class(cls):
tmpfile = udir.join('tmpfile').ensure()
From arigo at codespeak.net Fri Oct 29 10:49:38 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 29 Oct 2010 10:49:38 +0200 (CEST)
Subject: [pypy-svn] r78446 - pypy/trunk/pypy/jit/codewriter/test
Message-ID: <20101029084938.C39F136E076@codespeak.net>
Author: arigo
Date: Fri Oct 29 10:49:37 2010
New Revision: 78446
Added:
pypy/trunk/pypy/jit/codewriter/test/test_void_list.py
Log:
Oups, forgot to check in the test.
Added: pypy/trunk/pypy/jit/codewriter/test/test_void_list.py
==============================================================================
--- (empty file)
+++ pypy/trunk/pypy/jit/codewriter/test/test_void_list.py Fri Oct 29 10:49:37 2010
@@ -0,0 +1,142 @@
+from pypy.rpython.lltypesystem import lltype
+from pypy.translator.unsimplify import varoftype
+from pypy.objspace.flow.model import Constant
+from pypy.jit.codewriter.jtransform import NotSupported
+
+from pypy.jit.codewriter.test.test_list import builtin_test
+
+# ____________________________________________________________
+# XXX support for lists of voids is limited right now
+
+FIXEDLIST = lltype.Ptr(lltype.GcArray(lltype.Void))
+VARLIST = lltype.Ptr(lltype.GcStruct('VARLIST',
+ ('length', lltype.Signed),
+ ('items', FIXEDLIST),
+ adtmeths={"ITEM": lltype.Void}))
+
+# ____________________________________________________________
+# Fixed lists
+
+def test_newlist():
+ builtin_test('newlist', [], FIXEDLIST,
+ NotSupported)
+ builtin_test('newlist', [Constant(5, lltype.Signed)], FIXEDLIST,
+ NotSupported)
+ builtin_test('newlist', [varoftype(lltype.Signed)], FIXEDLIST,
+ NotSupported)
+ builtin_test('newlist', [Constant(5, lltype.Signed),
+ Constant(0, lltype.Signed)], FIXEDLIST,
+ NotSupported)
+ builtin_test('newlist', [Constant(5, lltype.Signed),
+ Constant(1, lltype.Signed)], FIXEDLIST,
+ NotSupported)
+ builtin_test('newlist', [Constant(5, lltype.Signed),
+ varoftype(lltype.Signed)], FIXEDLIST,
+ NotSupported)
+
+def test_fixed_ll_arraycopy():
+ builtin_test('list.ll_arraycopy',
+ [varoftype(FIXEDLIST),
+ varoftype(FIXEDLIST),
+ varoftype(lltype.Signed),
+ varoftype(lltype.Signed),
+ varoftype(lltype.Signed)],
+ lltype.Void,
+ NotSupported)
+
+def test_fixed_getitem():
+ builtin_test('list.getitem/NONNEG',
+ [varoftype(FIXEDLIST), varoftype(lltype.Signed)],
+ lltype.Void, "")
+ builtin_test('list.getitem/NEG',
+ [varoftype(FIXEDLIST), varoftype(lltype.Signed)],
+ lltype.Void, "")
+ builtin_test('list.getitem/CANRAISE',
+ [varoftype(FIXEDLIST), varoftype(lltype.Signed)],
+ lltype.Void, NotSupported)
+
+def test_fixed_getitem_foldable():
+ builtin_test('list.getitem_foldable/NONNEG',
+ [varoftype(FIXEDLIST), varoftype(lltype.Signed)],
+ lltype.Void, "")
+ builtin_test('list.getitem_foldable/NEG',
+ [varoftype(FIXEDLIST), varoftype(lltype.Signed)],
+ lltype.Void, "")
+ builtin_test('list.getitem_foldable/CANRAISE',
+ [varoftype(FIXEDLIST), varoftype(lltype.Signed)],
+ lltype.Void, NotSupported)
+
+def test_fixed_setitem():
+ builtin_test('list.setitem/NONNEG', [varoftype(FIXEDLIST),
+ varoftype(lltype.Signed),
+ varoftype(lltype.Void)],
+ lltype.Void, "")
+ builtin_test('list.setitem/NEG', [varoftype(FIXEDLIST),
+ varoftype(lltype.Signed),
+ varoftype(lltype.Void)],
+ lltype.Void, "")
+ builtin_test('list.setitem/CANRAISE', [varoftype(FIXEDLIST),
+ varoftype(lltype.Signed),
+ varoftype(lltype.Void)],
+ lltype.Void, NotSupported)
+
+def test_fixed_len():
+ builtin_test('list.len', [varoftype(FIXEDLIST)], lltype.Signed,
+ NotSupported)
+
+def test_fixed_len_foldable():
+ builtin_test('list.len_foldable', [varoftype(FIXEDLIST)], lltype.Signed,
+ NotSupported)
+
+# ____________________________________________________________
+# Resizable lists
+
+def test_resizable_newlist():
+ builtin_test('newlist', [], VARLIST,
+ NotSupported)
+ builtin_test('newlist', [Constant(5, lltype.Signed)], VARLIST,
+ NotSupported)
+ builtin_test('newlist', [varoftype(lltype.Signed)], VARLIST,
+ NotSupported)
+ builtin_test('newlist', [Constant(5, lltype.Signed),
+ Constant(0, lltype.Signed)], VARLIST,
+ NotSupported)
+ builtin_test('newlist', [Constant(5, lltype.Signed),
+ Constant(1, lltype.Signed)], VARLIST,
+ NotSupported)
+ builtin_test('newlist', [Constant(5, lltype.Signed),
+ varoftype(lltype.Signed)], VARLIST,
+ NotSupported)
+
+def test_resizable_getitem():
+ builtin_test('list.getitem/NONNEG',
+ [varoftype(VARLIST), varoftype(lltype.Signed)],
+ lltype.Void, "")
+ builtin_test('list.getitem/NEG',
+ [varoftype(VARLIST), varoftype(lltype.Signed)],
+ lltype.Void, "")
+ builtin_test('list.getitem/CANRAISE',
+ [varoftype(VARLIST), varoftype(lltype.Signed)],
+ lltype.Void, NotSupported)
+
+def test_resizable_setitem():
+ builtin_test('list.setitem/NONNEG', [varoftype(VARLIST),
+ varoftype(lltype.Signed),
+ varoftype(lltype.Void)],
+ lltype.Void, "")
+ builtin_test('list.setitem/NEG', [varoftype(VARLIST),
+ varoftype(lltype.Signed),
+ varoftype(lltype.Void)],
+ lltype.Void, "")
+ builtin_test('list.setitem/CANRAISE', [varoftype(VARLIST),
+ varoftype(lltype.Signed),
+ varoftype(lltype.Void)],
+ lltype.Void, NotSupported)
+
+def test_resizable_len():
+ builtin_test('list.len', [varoftype(VARLIST)], lltype.Signed,
+ NotSupported)
+
+def test_resizable_unsupportedop():
+ builtin_test('list.foobar', [varoftype(VARLIST)], lltype.Signed,
+ NotSupported)
From arigo at codespeak.net Fri Oct 29 10:54:38 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 29 Oct 2010 10:54:38 +0200 (CEST)
Subject: [pypy-svn] r78447 - in pypy/trunk/pypy/rlib: . test
Message-ID: <20101029085438.A9B26282BE8@codespeak.net>
Author: arigo
Date: Fri Oct 29 10:54:37 2010
New Revision: 78447
Modified:
pypy/trunk/pypy/rlib/rbigint.py
pypy/trunk/pypy/rlib/test/test_rbigint.py
Log:
Move here with a good implementation some logic that is so
far in objspace/std/strutil.py and horribly slow.
Modified: pypy/trunk/pypy/rlib/rbigint.py
==============================================================================
--- pypy/trunk/pypy/rlib/rbigint.py (original)
+++ pypy/trunk/pypy/rlib/rbigint.py Fri Oct 29 10:54:37 2010
@@ -1596,11 +1596,16 @@
# a few internal helpers
-DEC_PER_DIGIT = 1
-while int('9' * DEC_PER_DIGIT) < MASK:
- DEC_PER_DIGIT += 1
-DEC_PER_DIGIT -= 1
-DEC_MAX = 10 ** DEC_PER_DIGIT
+def digits_max_for_base(base):
+ dec_per_digit = 1
+ while base ** dec_per_digit < MASK:
+ dec_per_digit += 1
+ dec_per_digit -= 1
+ return base ** dec_per_digit
+
+BASE_MAX = [0, 0] + [digits_max_for_base(_base) for _base in range(2, 37)]
+DEC_MAX = digits_max_for_base(10)
+assert DEC_MAX == BASE_MAX[10]
def _decimalstr_to_bigint(s):
# a string that has been already parsed to be decimal and valid,
@@ -1615,7 +1620,6 @@
p += 1
a = rbigint.fromint(0)
- cnt = DEC_PER_DIGIT
tens = 1
dig = 0
ord0 = ord('0')
@@ -1631,4 +1635,23 @@
a.sign = -1
return a
-
+def parse_digit_string(parser):
+ # helper for objspace.std.strutil
+ a = rbigint.fromint(0)
+ base = parser.base
+ digitmax = BASE_MAX[base]
+ tens, dig = 1, 0
+ while True:
+ digit = parser.next_digit()
+ if tens == digitmax or digit < 0:
+ a = _muladd1(a, tens, dig)
+ if digit < 0:
+ break
+ dig = digit
+ tens = base
+ else:
+ dig = dig * base + digit
+ tens *= base
+ if parser.sign == -1:
+ a.sign = -1
+ return a
Modified: pypy/trunk/pypy/rlib/test/test_rbigint.py
==============================================================================
--- pypy/trunk/pypy/rlib/test/test_rbigint.py (original)
+++ pypy/trunk/pypy/rlib/test/test_rbigint.py Fri Oct 29 10:54:37 2010
@@ -448,6 +448,30 @@
assert (rbigint.fromlong(-9**50).ulonglongmask() ==
r_ulonglong(-9**50))
+ def test_parse_digit_string(self):
+ from pypy.rlib.rbigint import parse_digit_string
+ class Parser:
+ def __init__(self, base, sign, digits):
+ self.base = base
+ self.sign = sign
+ self.next_digit = iter(digits + [-1]).next
+ x = parse_digit_string(Parser(10, 1, [6]))
+ assert x.eq(rbigint.fromint(6))
+ x = parse_digit_string(Parser(10, 1, [6, 2, 3]))
+ assert x.eq(rbigint.fromint(623))
+ x = parse_digit_string(Parser(10, -1, [6, 2, 3]))
+ assert x.eq(rbigint.fromint(-623))
+ x = parse_digit_string(Parser(16, 1, [0xA, 0x4, 0xF]))
+ assert x.eq(rbigint.fromint(0xA4F))
+ num = 0
+ for i in range(36):
+ x = parse_digit_string(Parser(36, 1, range(i)))
+ assert x.eq(rbigint.fromlong(num))
+ num = num * 36 + i
+ x = parse_digit_string(Parser(16, -1, range(15,-1,-1)*99))
+ assert x.eq(rbigint.fromlong(long('-0x' + 'FEDCBA9876543210'*99, 16)))
+
+
BASE = 2 ** SHIFT
class TestTranslatable(object):
From arigo at codespeak.net Fri Oct 29 10:58:01 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 29 Oct 2010 10:58:01 +0200 (CEST)
Subject: [pypy-svn] r78448 - pypy/trunk/pypy/objspace/std
Message-ID: <20101029085801.5A678282BE8@codespeak.net>
Author: arigo
Date: Fri Oct 29 10:58:00 2010
New Revision: 78448
Modified:
pypy/trunk/pypy/objspace/std/smallintobject.py
Log:
Kill most import; they are unneeded.
Modified: pypy/trunk/pypy/objspace/std/smallintobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/smallintobject.py (original)
+++ pypy/trunk/pypy/objspace/std/smallintobject.py Fri Oct 29 10:58:00 2010
@@ -2,18 +2,13 @@
Implementation of small ints, stored as odd-valued pointers in the
translated PyPy. To enable them, see inttype.py.
"""
-import types
-from pypy.interpreter.error import OperationError
from pypy.objspace.std import intobject
from pypy.objspace.std.model import registerimplementation, W_Object
from pypy.objspace.std.register_all import register_all
-from pypy.objspace.std.multimethod import FailedToImplementArgs
from pypy.objspace.std.noneobject import W_NoneObject
-from pypy.rlib.rarithmetic import ovfcheck, ovfcheck_lshift, LONG_BIT, r_uint
-from pypy.objspace.std.inttype import wrapint
-from pypy.objspace.std.intobject import W_IntObject, _impl_int_int_pow
+from pypy.objspace.std.intobject import W_IntObject
from pypy.rlib.objectmodel import UnboxedValue
-from pypy.rlib.rbigint import rbigint
+from pypy.tool.sourcetools import func_with_new_name
class W_SmallIntObject(W_Object, UnboxedValue):
@@ -46,9 +41,7 @@
new_name = name.replace("Int", "SmallInt")
# Copy the function, so the annotator specializes it for
# W_SmallIntObject.
- ns[new_name] = types.FunctionType(func.func_code, ns, new_name,
- func.func_defaults,
- func.func_closure)
+ ns[new_name] = func_with_new_name(func, new_name)
ns["get_integer"] = ns["pos__SmallInt"] = ns["int__SmallInt"]
ns["get_negint"] = ns["neg__SmallInt"]
From arigo at codespeak.net Fri Oct 29 10:58:49 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 29 Oct 2010 10:58:49 +0200 (CEST)
Subject: [pypy-svn] r78449 - pypy/trunk/pypy/objspace/std/test
Message-ID: <20101029085849.3940A36E0CB@codespeak.net>
Author: arigo
Date: Fri Oct 29 10:58:47 2010
New Revision: 78449
Modified:
pypy/trunk/pypy/objspace/std/test/test_longobject.py
Log:
This test contains tons of operations that are actually constant-folded
by the CPython compiler. So it doesn't really test anything :-(
Modified: pypy/trunk/pypy/objspace/std/test/test_longobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/test/test_longobject.py (original)
+++ pypy/trunk/pypy/objspace/std/test/test_longobject.py Fri Oct 29 10:58:47 2010
@@ -41,56 +41,64 @@
class AppTestLong:
def test_add(self):
- assert int(123L + 12443L) == 123 + 12443
- assert -20 + 2 + 3L + True == -14L
+ x = 123L
+ assert int(x + 12443L) == 123 + 12443
+ x = -20
+ assert x + 2 + 3L + True == -14L
def test_sub(self):
- assert int(58543L - 12332L) == 58543 - 12332
- assert 237123838281233L * 12 == 237123838281233L * 12L
+ x = 58543L
+ assert int(x - 12332L) == 58543 - 12332
+ x = 237123838281233L
+ assert x * 12 == x * 12L
def test_mul(self):
- assert 363L * 2 ** 40 == 363L << 40
+ x = 363L
+ assert x * 2 ** 40 == x << 40
def test_truediv(self):
exec "from __future__ import division; a = 31415926L / 10000000L"
assert a == 3.1415926
def test_floordiv(self):
- a = 31415926L // 10000000L
+ x = 31415926L
+ a = x // 10000000L
assert a == 3L
def test_compare(self):
+ Z = 0
+ ZL = 0L
for BIG in (1L, 1L << 62, 1L << 9999):
- assert 0 == 0L
- assert not (0 != 0L)
- assert 0L == 0
- assert not (0L != 0)
- assert not (0 == BIG)
- assert 0 != BIG
- assert not (BIG == 0)
- assert BIG != 0
- assert not (0L == BIG)
- assert 0L != BIG
- assert 0 <= 0L
- assert not (0 < 0L)
- assert 0 <= BIG
- assert 0 < BIG
- assert not (BIG <= 0)
- assert not (BIG < 0)
- assert 0L <= 0L
- assert not (0L < 0L)
- assert 0L <= BIG
- assert 0L < BIG
- assert not (BIG <= 0L)
- assert not (BIG < 0L)
- assert not (0 <= -BIG)
- assert not (0 < -BIG)
- assert -BIG <= 0
- assert -BIG < 0
- assert not (0L <= -BIG)
- assert not (0L < -BIG)
- assert -BIG <= 0L
- assert -BIG < 0L
+ assert Z == ZL
+ assert not (Z != ZL)
+ assert ZL == Z
+ assert not (ZL != Z)
+ assert not (Z == BIG)
+ assert Z != BIG
+ assert not (BIG == Z)
+ assert BIG != Z
+ assert not (ZL == BIG)
+ assert ZL != BIG
+ assert Z <= ZL
+ assert not (Z < ZL)
+ assert Z <= BIG
+ assert Z < BIG
+ assert not (BIG <= Z)
+ assert not (BIG < Z)
+ assert ZL <= ZL
+ assert not (ZL < ZL)
+ assert ZL <= BIG
+ assert ZL < BIG
+ assert not (BIG <= ZL)
+ assert not (BIG < ZL)
+ assert not (Z <= -BIG)
+ assert not (Z < -BIG)
+ assert -BIG <= Z
+ assert -BIG < Z
+ assert not (ZL <= -BIG)
+ assert not (ZL < -BIG)
+ assert -BIG <= ZL
+ assert -BIG < ZL
#
assert not (BIG < int(BIG))
assert (BIG <= int(BIG))
@@ -137,7 +145,8 @@
def test_conversion(self):
class long2(long):
pass
- x = long2(1L<<100)
+ x = 1L
+ x = long2(x<<100)
y = int(x)
assert type(y) == long
assert type(+long2(5)) is long
@@ -152,7 +161,8 @@
assert type(long2(5) // 1) is long
def test_pow(self):
- assert pow(0L, 0L, 1L) == 0L
+ x = 0L
+ assert pow(x, 0L, 1L) == 0L
def test_getnewargs(self):
assert 0L .__getnewargs__() == (0L,)
@@ -193,13 +203,14 @@
assert oct(01234567012345670L) == '01234567012345670L'
def test_bits(self):
- assert 0xAAAAAAAAL | 0x55555555L == 0xFFFFFFFFL
- assert 0xAAAAAAAAL & 0x55555555L == 0x00000000L
- assert 0xAAAAAAAAL ^ 0x55555555L == 0xFFFFFFFFL
- assert -0xAAAAAAAAL | 0x55555555L == -0xAAAAAAA9L
- assert 0xAAAAAAAAL | 0x555555555L == 0x5FFFFFFFFL
- assert 0xAAAAAAAAL & 0x555555555L == 0x000000000L
- assert 0xAAAAAAAAL ^ 0x555555555L == 0x5FFFFFFFFL
+ x = 0xAAAAAAAAL
+ assert x | 0x55555555L == 0xFFFFFFFFL
+ assert x & 0x55555555L == 0x00000000L
+ assert x ^ 0x55555555L == 0xFFFFFFFFL
+ assert -x | 0x55555555L == -0xAAAAAAA9L
+ assert x | 0x555555555L == 0x5FFFFFFFFL
+ assert x & 0x555555555L == 0x000000000L
+ assert x ^ 0x555555555L == 0x5FFFFFFFFL
def test_hash(self):
# ints have the same hash as equal longs
@@ -229,7 +240,8 @@
def test_huge_longs(self):
import operator
- huge = 1L << 40000L
+ x = 1L
+ huge = x << 40000L
raises(OverflowError, float, huge)
raises(OverflowError, operator.truediv, huge, 3)
raises(OverflowError, operator.truediv, huge, 3L)
From arigo at codespeak.net Fri Oct 29 10:59:46 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 29 Oct 2010 10:59:46 +0200 (CEST)
Subject: [pypy-svn] r78450 - pypy/trunk/pypy/objspace/std
Message-ID: <20101029085946.E255C282C1E@codespeak.net>
Author: arigo
Date: Fri Oct 29 10:59:45 2010
New Revision: 78450
Modified:
pypy/trunk/pypy/objspace/std/inttype.py
pypy/trunk/pypy/objspace/std/longtype.py
pypy/trunk/pypy/objspace/std/strutil.py
Log:
Use the more efficient parse_digit_string() from rbigint.
Modified: pypy/trunk/pypy/objspace/std/inttype.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/inttype.py (original)
+++ pypy/trunk/pypy/objspace/std/inttype.py Fri Oct 29 10:59:45 2010
@@ -1,7 +1,7 @@
from pypy.interpreter import gateway
from pypy.interpreter.error import OperationError
from pypy.objspace.std.stdtypedef import StdTypeDef
-from pypy.objspace.std.strutil import (string_to_int, string_to_w_long,
+from pypy.objspace.std.strutil import (string_to_int, string_to_bigint,
ParseStringError,
ParseStringOverflowError)
from pypy.rlib.rarithmetic import r_uint
@@ -45,10 +45,12 @@
def retry_to_w_long(space, parser, base=0):
parser.rewind()
try:
- return string_to_w_long(space, None, base=base, parser=parser)
+ bigint = string_to_bigint(None, base=base, parser=parser)
except ParseStringError, e:
raise OperationError(space.w_ValueError,
space.wrap(e.msg))
+ from pypy.objspace.std.longobject import W_LongObject
+ return W_LongObject(bigint)
def descr__new__(space, w_inttype, w_x=0, w_base=gateway.NoneNotWrapped):
from pypy.objspace.std.intobject import W_IntObject
Modified: pypy/trunk/pypy/objspace/std/longtype.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/longtype.py (original)
+++ pypy/trunk/pypy/objspace/std/longtype.py Fri Oct 29 10:59:45 2010
@@ -1,7 +1,7 @@
from pypy.interpreter.error import OperationError
from pypy.interpreter import gateway
from pypy.objspace.std.stdtypedef import StdTypeDef
-from pypy.objspace.std.strutil import string_to_w_long, ParseStringError
+from pypy.objspace.std.strutil import string_to_bigint, ParseStringError
def descr__new__(space, w_longtype, w_x=0, w_base=gateway.NoneNotWrapped):
from pypy.objspace.std.longobject import W_LongObject
@@ -9,10 +9,10 @@
if w_base is None:
# check for easy cases
if type(w_value) is W_LongObject:
- pass
+ bigint = w_value.num
elif space.is_true(space.isinstance(w_value, space.w_str)):
try:
- w_value = string_to_w_long(space, space.str_w(w_value))
+ bigint = string_to_bigint(space.str_w(w_value))
except ParseStringError, e:
raise OperationError(space.w_ValueError,
space.wrap(e.msg))
@@ -22,7 +22,7 @@
from pypy.objspace.std.ropeunicodeobject import unicode_to_decimal_w
else:
from pypy.objspace.std.unicodeobject import unicode_to_decimal_w
- w_value = string_to_w_long(space, unicode_to_decimal_w(space, w_value))
+ bigint = string_to_bigint(unicode_to_decimal_w(space, w_value))
except ParseStringError, e:
raise OperationError(space.w_ValueError,
space.wrap(e.msg))
@@ -35,10 +35,11 @@
if space.is_true(space.isinstance(w_obj, space.w_long)):
assert isinstance(w_obj, W_LongObject) # XXX this could fail!
# XXX find a way to do that even if w_obj is not a W_LongObject
- w_value = w_obj
+ bigint = w_obj.num
elif space.is_true(space.isinstance(w_obj, space.w_int)):
+ from pypy.rlib.rbigint import rbigint
intval = space.int_w(w_obj)
- w_value = W_LongObject.fromint(space, intval)
+ bigint = rbigint.fromint(intval)
else:
raise OperationError(space.w_ValueError,
space.wrap("value can't be converted to long"))
@@ -56,13 +57,13 @@
space.wrap("long() can't convert non-string "
"with explicit base"))
try:
- w_value = string_to_w_long(space, s, base)
+ bigint = string_to_bigint(s, base)
except ParseStringError, e:
raise OperationError(space.w_ValueError,
space.wrap(e.msg))
w_obj = space.allocate_instance(W_LongObject, w_longtype)
- W_LongObject.__init__(w_obj, w_value.num)
+ W_LongObject.__init__(w_obj, bigint)
return w_obj
# ____________________________________________________________
Modified: pypy/trunk/pypy/objspace/std/strutil.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/strutil.py (original)
+++ pypy/trunk/pypy/objspace/std/strutil.py Fri Oct 29 10:59:45 2010
@@ -4,6 +4,7 @@
from pypy.rlib.rarithmetic import ovfcheck, break_up_float, parts_to_float,\
INFINITY, NAN
+from pypy.rlib.rbigint import rbigint, parse_digit_string
from pypy.interpreter.error import OperationError
import math
@@ -91,9 +92,10 @@
return -1
def string_to_int(s, base=10):
- """Utility to converts a string to an integer (or possibly a long).
+ """Utility to converts a string to an integer.
If base is 0, the proper base is guessed based on the leading
characters of 's'. Raises ParseStringError in case of error.
+ Raises ParseStringOverflowError in case the result does not fit.
"""
s = literal = strip_spaces(s)
p = NumberStringParser(s, literal, base, 'int')
@@ -113,11 +115,9 @@
except OverflowError:
raise ParseStringOverflowError(p)
-def string_to_long(space, s, base=10, parser=None):
- return string_to_w_long(space, s, base, parser).longval()
-
-def string_to_w_long(space, s, base=10, parser=None):
- """As string_to_int(), but ignores an optional 'l' or 'L' suffix."""
+def string_to_bigint(s, base=10, parser=None):
+ """As string_to_int(), but ignores an optional 'l' or 'L' suffix
+ and returns an rbigint."""
if parser is None:
s = literal = strip_spaces(s)
if (s.endswith('l') or s.endswith('L')) and base < 22:
@@ -126,18 +126,7 @@
p = NumberStringParser(s, literal, base, 'long')
else:
p = parser
- w_base = space.newlong(p.base)
- w_result = space.newlong(0)
- while True:
- digit = p.next_digit()
- if digit == -1:
- if p.sign == -1:
- w_result = space.neg(w_result)
- # XXX grumble
- from pypy.objspace.std.longobject import W_LongObject
- assert isinstance(w_result, W_LongObject)
- return w_result
- w_result = space.add(space.mul(w_result,w_base), space.newlong(digit))
+ return parse_digit_string(p)
# Tim's comment:
# 57 bits are more than needed in any case.
From afa at codespeak.net Fri Oct 29 11:23:03 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 11:23:03 +0200 (CEST)
Subject: [pypy-svn] r78451 -
pypy/branch/fast-forward/lib-python/modified-2.7.0/test
Message-ID: <20101029092303.8C97236E3BC@codespeak.net>
Author: afa
Date: Fri Oct 29 11:23:01 2010
New Revision: 78451
Added:
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_socket.py
- copied, changed from r78441, pypy/branch/fast-forward/lib-python/2.7.0/test/test_socket.py
Modified:
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py
Log:
More gc_collect() to avoid a deadlock and make more tests pass
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_socket.py (from r78441, pypy/branch/fast-forward/lib-python/2.7.0/test/test_socket.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/test/test_socket.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_socket.py Fri Oct 29 11:23:01 2010
@@ -160,6 +160,7 @@
def clientTearDown(self):
self.done.set()
+ test_support.gc_collect()
thread.exit()
class ThreadedTCPSocketTest(SocketTCPTest, ThreadableTest):
@@ -246,6 +247,7 @@
self.assertEqual(p.fileno(), s.fileno())
s.close()
s = None
+ test_support.gc_collect()
try:
p.fileno()
except ReferenceError:
@@ -727,6 +729,7 @@
def testClose(self):
conn, addr = self.serv.accept()
conn.close()
+ test_support.gc_collect() # not sure why
sd = self.cli
read, write, err = select.select([sd], [], [], 1.0)
Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py
==============================================================================
--- pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py Fri Oct 29 11:23:01 2010
@@ -281,7 +281,9 @@
tempsock = socket.socket(family, socktype)
port = bind_port(tempsock)
tempsock.close()
+ # Be sure to close the socket so the port can be reused
del tempsock
+ gc_collect()
return port
def bind_port(sock, host=HOST):
From arigo at codespeak.net Fri Oct 29 11:35:42 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 29 Oct 2010 11:35:42 +0200 (CEST)
Subject: [pypy-svn] r78452 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101029093542.0AF8B282BE8@codespeak.net>
Author: arigo
Date: Fri Oct 29 11:35:40 2010
New Revision: 78452
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py
Log:
Fix this test.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py Fri Oct 29 11:35:40 2010
@@ -599,19 +599,19 @@
get_printable_location = lambda codeno : str(codeno))
def portal(codeno, j):
- i = 0
+ i = 1
while 1:
driver.jit_merge_point(codeno=codeno, i=i, j=j)
- if i == 1:
+ if (i >> 1) == 1:
if j == 0:
return
portal(2, j - 1)
- elif i == 3:
+ elif i == 5:
return
i += 1
driver.can_enter_jit(codeno=codeno, i=i, j=j)
- portal(2, 50)
+ portal(2, 5)
from pypy.jit.metainterp import compile, pyjitpl
pyjitpl._warmrunnerdesc = None
@@ -623,8 +623,8 @@
original_ctc = compile.compile_tmp_callback
try:
compile.compile_tmp_callback = my_ctc
- self.meta_interp(portal, [2, 20], inline=True)
- self.check_loops(call_assembler=1, call_may_force=0,
+ self.meta_interp(portal, [2, 5], inline=True)
+ self.check_loops(call_assembler=2, call_may_force=0,
everywhere=True)
finally:
compile.compile_tmp_callback = original_ctc
From hakanardo at codespeak.net Fri Oct 29 11:40:02 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Fri, 29 Oct 2010 11:40:02 +0200 (CEST)
Subject: [pypy-svn] r78453 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101029094002.6D5AF36E0CB@codespeak.net>
Author: hakanardo
Date: Fri Oct 29 11:40:00 2010
New Revision: 78453
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py
Log:
repeated fixed
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py Fri Oct 29 11:40:00 2010
@@ -652,14 +652,14 @@
get_printable_location = lambda codeno : str(codeno))
def portal(codeno, frame):
- i = 0
+ i = 1
while 1:
driver.jit_merge_point(codeno=codeno, i=i, frame=frame)
- if i == 1:
+ if (i >> 1) == 1:
if frame.j == 0:
return
portal(2, Frame(frame.j - 1))
- elif i == 3:
+ elif i == 5:
return
i += 1
driver.can_enter_jit(codeno=codeno, i=i, frame=frame)
@@ -667,7 +667,7 @@
def main(codeno, j):
portal(codeno, Frame(j))
- main(2, 50)
+ main(2, 5)
from pypy.jit.metainterp import compile, pyjitpl
pyjitpl._warmrunnerdesc = None
@@ -679,8 +679,8 @@
original_ctc = compile.compile_tmp_callback
try:
compile.compile_tmp_callback = my_ctc
- self.meta_interp(main, [2, 20], inline=True)
- self.check_loops(call_assembler=1, call_may_force=0,
+ self.meta_interp(main, [2, 5], inline=True)
+ self.check_loops(call_assembler=2, call_may_force=0,
everywhere=True)
finally:
compile.compile_tmp_callback = original_ctc
From afa at codespeak.net Fri Oct 29 11:51:42 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 11:51:42 +0200 (CEST)
Subject: [pypy-svn] r78454 -
pypy/branch/fast-forward/lib-python/modified-2.7.0/test
Message-ID: <20101029095142.DCC85282BE8@codespeak.net>
Author: afa
Date: Fri Oct 29 11:51:41 2010
New Revision: 78454
Added:
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_zlib.py
- copied, changed from r78441, pypy/branch/fast-forward/lib-python/2.7.0/test/test_zlib.py
Log:
Use os.urandom() instead of complex stuff.
This allows the test to run without timing out...
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_zlib.py (from r78441, pypy/branch/fast-forward/lib-python/2.7.0/test/test_zlib.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/test/test_zlib.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_zlib.py Fri Oct 29 11:51:41 2010
@@ -1,6 +1,7 @@
import unittest
from test import test_support
import binascii
+import os
import random
from test.test_support import precisionbigmemtest, _1G
@@ -93,14 +94,7 @@
class BaseCompressTestCase(object):
def check_big_compress_buffer(self, size, compress_func):
- _1M = 1024 * 1024
- fmt = "%%0%dx" % (2 * _1M)
- # Generate 10MB worth of random, and expand it by repeating it.
- # The assumption is that zlib's memory is not big enough to exploit
- # such spread out redundancy.
- data = ''.join([binascii.a2b_hex(fmt % random.getrandbits(8 * _1M))
- for i in range(10)])
- data = data * (size // len(data) + 1)
+ data = os.urandom(size)
try:
compress_func(data)
finally:
From hakanardo at codespeak.net Fri Oct 29 11:59:21 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Fri, 29 Oct 2010 11:59:21 +0200 (CEST)
Subject: [pypy-svn] r78455 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101029095921.CBEAA36E076@codespeak.net>
Author: hakanardo
Date: Fri Oct 29 11:59:20 2010
New Revision: 78455
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_jitprof.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py
Log:
fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_jitprof.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_jitprof.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_jitprof.py Fri Oct 29 11:59:20 2010
@@ -55,6 +55,8 @@
TRACING,
BACKEND,
~ BACKEND,
+ BACKEND,
+ ~ BACKEND,
~ TRACING,
RUNNING,
~ RUNNING,
@@ -62,8 +64,8 @@
~ BLACKHOLE
]
assert profiler.events == expected
- assert profiler.times == [2, 1, 1, 1]
- assert profiler.counters == [1, 1, 1, 1, 3, 3, 1, 7, 1, 0, 0, 0,
+ assert profiler.times == [3, 2, 1, 1]
+ assert profiler.counters == [1, 2, 1, 1, 3, 3, 1, 13, 2, 0, 0, 0,
0, 0, 0, 0]
def test_simple_loop_with_call(self):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py Fri Oct 29 11:59:20 2010
@@ -1016,7 +1016,7 @@
res = self.meta_interp(portal, [2, 0], inline=True,
policy=StopAtXPolicy(residual))
assert res == portal(2, 0)
- self.check_loops(call_assembler=2)
+ self.check_loops(call_assembler=3, everywhere=True)
def test_inline_without_hitting_the_loop(self):
driver = JitDriver(greens = ['codeno'], reds = ['i'],
From cfbolz at codespeak.net Fri Oct 29 12:03:21 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 29 Oct 2010 12:03:21 +0200 (CEST)
Subject: [pypy-svn] r78456 - in pypy/trunk/pypy: interpreter
objspace/std/test
Message-ID: <20101029100321.7A0E2282C1E@codespeak.net>
Author: cfbolz
Date: Fri Oct 29 12:03:20 2010
New Revision: 78456
Modified:
pypy/trunk/pypy/interpreter/typedef.py
pypy/trunk/pypy/objspace/std/test/test_mapdict.py
Log:
fix one of the nightly failures
Modified: pypy/trunk/pypy/interpreter/typedef.py
==============================================================================
--- pypy/trunk/pypy/interpreter/typedef.py (original)
+++ pypy/trunk/pypy/interpreter/typedef.py Fri Oct 29 12:03:20 2010
@@ -203,6 +203,7 @@
from pypy.objspace.std.mapdict import BaseMapdictObject, ObjectMixin
add(BaseMapdictObject)
add(ObjectMixin)
+ body["user_overridden_class"] = True
features = ()
if "user" in features: # generic feature needed by all subcls
Modified: pypy/trunk/pypy/objspace/std/test/test_mapdict.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/test/test_mapdict.py (original)
+++ pypy/trunk/pypy/objspace/std/test/test_mapdict.py Fri Oct 29 12:03:20 2010
@@ -777,3 +777,19 @@
assert res == (0, 0, 1)
res = self.check(f, 'x')
assert res == (0, 0, 1)
+
+class TestDictSubclassShortcutBug(object):
+ def setup_class(cls):
+ cls.space = gettestobjspace(
+ **{"objspace.std.withmapdict": True,
+ "objspace.std.withmethodcachecounter": True})
+
+ def test_bug(self):
+ w_dict = self.space.appexec([], """():
+ class A(dict):
+ def __getitem__(self, key):
+ return 1
+ assert eval("a", globals(), A()) == 1
+ return A()
+ """)
+ assert w_dict.user_overridden_class
From arigo at codespeak.net Fri Oct 29 13:26:45 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 29 Oct 2010 13:26:45 +0200 (CEST)
Subject: [pypy-svn] r78457 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt
Message-ID: <20101029112645.4AFF736E076@codespeak.net>
Author: arigo
Date: Fri Oct 29 13:26:43 2010
New Revision: 78457
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
Log:
(hakanardo, arigo)
Change the interface of get_forced_boxes() to enum_forced_boxes().
Implement it for VStrings too.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py Fri Oct 29 13:26:43 2010
@@ -16,7 +16,7 @@
OptIntBounds(),
OptRewrite(),
OptVirtualize(),
-# OptString(),
+ OptString(),
OptHeap(),
]
optimizer = Optimizer(metainterp_sd, loop, optimizations, virtuals)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Fri Oct 29 13:26:43 2010
@@ -47,14 +47,11 @@
def get_key_box(self):
return self.box
- # FIXME: Reuse get_args_for_fail instead?
- def get_forced_boxes(self, already_seen):
+ def enum_forced_boxes(self, boxes, already_seen):
key = self.get_key_box()
- if key in already_seen:
- return []
- box = self.force_box()
- already_seen.append(self.get_key_box())
- return [box]
+ if key not in already_seen:
+ boxes.append(self.force_box())
+ already_seen[self.get_key_box()] = None
def get_args_for_fail(self, modifier):
pass
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py Fri Oct 29 13:26:43 2010
@@ -108,6 +108,17 @@
for value in self._chars:
value.get_args_for_fail(modifier)
+ def enum_forced_boxes(self, boxes, already_seen):
+ key = self.get_key_box()
+ if key in already_seen:
+ return
+ already_seen.append(key)
+ if self.box is None:
+ for box in self._chars:
+ box.enum_forced_boxes(boxes, already_seen)
+ else:
+ boxes.append(self.box)
+
def _make_virtual(self, modifier):
return modifier.make_vstrplain()
@@ -149,6 +160,17 @@
self.left.get_args_for_fail(modifier)
self.right.get_args_for_fail(modifier)
+ def enum_forced_boxes(self, boxes, already_seen):
+ key = self.get_key_box()
+ if key in already_seen:
+ return
+ already_seen.append(key)
+ if self.box is None:
+ self.left.enum_forced_boxes(boxes, already_seen)
+ self.right.enum_forced_boxes(boxes, already_seen)
+ else:
+ boxes.append(self.box)
+
def _make_virtual(self, modifier):
return modifier.make_vstrconcat()
@@ -194,6 +216,18 @@
self.vstart.get_args_for_fail(modifier)
self.vlength.get_args_for_fail(modifier)
+ def enum_forced_boxes(self, boxes, already_seen):
+ key = self.get_key_box()
+ if key in already_seen:
+ return
+ already_seen.append(key)
+ if self.box is None:
+ self.vstr.enum_forced_boxes(boxes, already_seen)
+ self.vstart.enum_forced_boxes(boxes, already_seen)
+ self.vlength.enum_forced_boxes(boxes, already_seen)
+ else:
+ boxes.append(self.box)
+
def _make_virtual(self, modifier):
return modifier.make_vstrslice()
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Fri Oct 29 13:26:43 2010
@@ -51,9 +51,11 @@
self.snapshot_map ={None: None}
inputargs = []
- seen = []
+ seen = {}
for arg in jump_args:
- for a in self.getvalue(arg).get_forced_boxes(seen):
+ boxes = []
+ self.getvalue(arg).enum_forced_boxes(boxes, seen)
+ for a in boxes:
if not isinstance(a, Const):
inputargs.append(a)
@@ -75,8 +77,9 @@
if newop.getopnum() == rop.JUMP:
args = []
#for arg in newop.getarglist():
- for arg in [argmap[a] for a in inputargs]:
- args.extend(self.getvalue(arg).get_forced_boxes([]))
+ for arg in inputargs:
+ arg = argmap[arg]
+ self.getvalue(arg).enum_forced_boxes(args, {})
newop.initarglist(args + inputargs[len(args):])
#print 'P: ', str(newop)
@@ -105,8 +108,9 @@
jmp = self.optimizer.newoperations[-1]
if jmp.getopnum() == rop.JUMP:
newval = self.getvalue(argmap[a])
- newarg = newval.get_forced_boxes([])
- jmp.initarglist(jmp.getarglist() + newarg)
+ boxes = jmp.getarglist()[:]
+ newval.enum_forced_boxes(boxes, {})
+ jmp.initarglist(boxes)
return inputargs
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Fri Oct 29 13:26:43 2010
@@ -121,20 +121,17 @@
fieldvalue = self._fields[ofs]
fieldvalue.get_args_for_fail(modifier)
- def get_forced_boxes(self, already_seen):
+ def enum_forced_boxes(self, boxes, already_seen):
key = self.get_key_box()
if key in already_seen:
- return []
- already_seen.append(key)
+ return
+ already_seen[key] = None
if self.box is None:
lst = self._get_field_descr_list()
- fieldboxes = []
for ofs in lst:
- boxes = self._fields[ofs].get_forced_boxes(already_seen)
- fieldboxes.extend(boxes)
- return fieldboxes
+ self._fields[ofs].enum_forced_boxes(boxes, already_seen)
else:
- return [self.box]
+ boxes.append(self.box)
class VirtualValue(AbstractVirtualStructValue):
@@ -208,18 +205,16 @@
def _make_virtual(self, modifier):
return modifier.make_varray(self.arraydescr)
- def get_forced_boxes(self, already_seen):
+ def enum_forced_boxes(self, boxes, already_seen):
key = self.get_key_box()
if key in already_seen:
- return []
+ return
already_seen.append(key)
if self.box is None:
- boxes = []
for itemvalue in self._items:
- boxes.extend(itemvalue.get_forced_boxes(already_seen))
- return boxes
+ itemvalue.enum_forced_boxes(boxes, already_seen)
else:
- return [self.box]
+ boxes.append(self.box)
class __extend__(SpecNode):
From hakanardo at codespeak.net Fri Oct 29 13:28:31 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Fri, 29 Oct 2010 13:28:31 +0200 (CEST)
Subject: [pypy-svn] r78458 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101029112831.A7534282BE8@codespeak.net>
Author: hakanardo
Date: Fri Oct 29 13:28:30 2010
New Revision: 78458
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_list.py
Log:
(hakanardo, arigo) fixed test
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_list.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_list.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_list.py Fri Oct 29 13:28:30 2010
@@ -216,4 +216,4 @@
return a * b
res = self.meta_interp(f, [37])
assert res == f(37)
- self.check_loops(getfield_gc=1)
+ self.check_loops(getfield_gc=1, everywhere=True)
From arigo at codespeak.net Fri Oct 29 13:29:01 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 29 Oct 2010 13:29:01 +0200 (CEST)
Subject: [pypy-svn] r78459 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt
Message-ID: <20101029112901.D414936E076@codespeak.net>
Author: arigo
Date: Fri Oct 29 13:29:00 2010
New Revision: 78459
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
Log:
Oups. Tests are useful, if only they passed originally :-/
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py Fri Oct 29 13:29:00 2010
@@ -112,7 +112,7 @@
key = self.get_key_box()
if key in already_seen:
return
- already_seen.append(key)
+ already_seen[key] = None
if self.box is None:
for box in self._chars:
box.enum_forced_boxes(boxes, already_seen)
@@ -164,7 +164,7 @@
key = self.get_key_box()
if key in already_seen:
return
- already_seen.append(key)
+ already_seen[key] = None
if self.box is None:
self.left.enum_forced_boxes(boxes, already_seen)
self.right.enum_forced_boxes(boxes, already_seen)
@@ -220,7 +220,7 @@
key = self.get_key_box()
if key in already_seen:
return
- already_seen.append(key)
+ already_seen[key] = None
if self.box is None:
self.vstr.enum_forced_boxes(boxes, already_seen)
self.vstart.enum_forced_boxes(boxes, already_seen)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Fri Oct 29 13:29:00 2010
@@ -209,7 +209,7 @@
key = self.get_key_box()
if key in already_seen:
return
- already_seen.append(key)
+ already_seen[key] = None
if self.box is None:
for itemvalue in self._items:
itemvalue.enum_forced_boxes(boxes, already_seen)
From afa at codespeak.net Fri Oct 29 13:30:07 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 13:30:07 +0200 (CEST)
Subject: [pypy-svn] r78460 - pypy/trunk/pypy/translator/c/gcc
Message-ID: <20101029113007.00160282BE8@codespeak.net>
Author: afa
Date: Fri Oct 29 13:30:06 2010
New Revision: 78460
Modified:
pypy/trunk/pypy/translator/c/gcc/trackgcroot.py
Log:
gcmaptable now uses ".quad" in all 64bit platforms
Modified: pypy/trunk/pypy/translator/c/gcc/trackgcroot.py
==============================================================================
--- pypy/trunk/pypy/translator/c/gcc/trackgcroot.py (original)
+++ pypy/trunk/pypy/translator/c/gcc/trackgcroot.py Fri Oct 29 13:30:06 2010
@@ -1599,7 +1599,7 @@
txt = kwargs[self.format]
print >> output, "\t%s" % txt
- if self.format == 'elf64':
+ if self.format in ('elf64', 'darwin64'):
word_decl = '.quad'
else:
word_decl = '.long'
From arigo at codespeak.net Fri Oct 29 13:34:19 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 29 Oct 2010 13:34:19 +0200 (CEST)
Subject: [pypy-svn] r78461 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101029113419.4707136E0C1@codespeak.net>
Author: arigo
Date: Fri Oct 29 13:34:17 2010
New Revision: 78461
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_spec.py
Log:
Fix the test.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_spec.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_spec.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_spec.py Fri Oct 29 13:34:17 2010
@@ -7,7 +7,6 @@
optimizer = OPTIMIZER_FULL
automatic_promotion_result = {
'int_add' : 3, 'int_gt' : 1, 'guard_false' : 1, 'jump' : 1,
- 'guard_value' : 1
}
# ====> test_loop.py
From arigo at codespeak.net Fri Oct 29 13:44:31 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 29 Oct 2010 13:44:31 +0200 (CEST)
Subject: [pypy-svn] r78462 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt
Message-ID: <20101029114431.09FC9282BE8@codespeak.net>
Author: arigo
Date: Fri Oct 29 13:44:30 2010
New Revision: 78462
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
Log:
(hakanardo, arigo) Fix.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Fri Oct 29 13:44:30 2010
@@ -79,7 +79,7 @@
#for arg in newop.getarglist():
for arg in inputargs:
arg = argmap[arg]
- self.getvalue(arg).enum_forced_boxes(args, {})
+ args.append(self.getvalue(arg).force_box())
newop.initarglist(args + inputargs[len(args):])
#print 'P: ', str(newop)
@@ -108,8 +108,8 @@
jmp = self.optimizer.newoperations[-1]
if jmp.getopnum() == rop.JUMP:
newval = self.getvalue(argmap[a])
- boxes = jmp.getarglist()[:]
- newval.enum_forced_boxes(boxes, {})
+ boxes = jmp.getarglist()
+ boxes.append(newval.force_box())
jmp.initarglist(boxes)
return inputargs
From hakanardo at codespeak.net Fri Oct 29 13:47:19 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Fri, 29 Oct 2010 13:47:19 +0200 (CEST)
Subject: [pypy-svn] r78463 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101029114719.A2B1E36E3BC@codespeak.net>
Author: hakanardo
Date: Fri Oct 29 13:47:18 2010
New Revision: 78463
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py
Log:
(hakanardo, arigo) skip this for now
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py Fri Oct 29 13:47:18 2010
@@ -164,6 +164,7 @@
getfield_gc=0, setfield_gc=0)
def test_two_loops_with_virtual(self):
+ py.test.skip("We don't know how to virtualize across bridges right now")
myjitdriver = JitDriver(greens = [], reds = ['n', 'node'])
def f(n):
node = self._new()
From cfbolz at codespeak.net Fri Oct 29 13:48:11 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 29 Oct 2010 13:48:11 +0200 (CEST)
Subject: [pypy-svn] r78464 - in pypy/trunk/pypy/objspace/std: . test
Message-ID: <20101029114811.37C0F282BE8@codespeak.net>
Author: cfbolz
Date: Fri Oct 29 13:48:09 2010
New Revision: 78464
Modified:
pypy/trunk/pypy/objspace/std/mapdict.py
pypy/trunk/pypy/objspace/std/test/test_mapdict.py
Log:
fix failing weakref assertion error
Modified: pypy/trunk/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/mapdict.py (original)
+++ pypy/trunk/pypy/objspace/std/mapdict.py Fri Oct 29 13:48:09 2010
@@ -361,7 +361,8 @@
def setweakref(self, space, weakreflifeline):
from pypy.module._weakref.interp__weakref import WeakrefLifeline
- assert isinstance(weakreflifeline, WeakrefLifeline)
+ assert (isinstance(weakreflifeline, WeakrefLifeline) or
+ weakreflifeline is None)
self._get_mapdict_map().write(self, ("weakref", SPECIAL), weakreflifeline)
class ObjectMixin(object):
Modified: pypy/trunk/pypy/objspace/std/test/test_mapdict.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/test/test_mapdict.py (original)
+++ pypy/trunk/pypy/objspace/std/test/test_mapdict.py Fri Oct 29 13:48:09 2010
@@ -160,6 +160,13 @@
assert obj.getweakref() is lifeline1
assert obj.getdictvalue(space, "weakref") == 41
+ lifeline1 = WeakrefLifeline(space)
+ obj = c.instantiate()
+ assert obj.getweakref() is None
+ obj.setweakref(space, lifeline1)
+ obj.setweakref(space, None)
+
+
def test_slots():
cls = Class()
From arigo at codespeak.net Fri Oct 29 14:02:34 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 29 Oct 2010 14:02:34 +0200 (CEST)
Subject: [pypy-svn] r78466 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101029120234.7B8A236E0CB@codespeak.net>
Author: arigo
Date: Fri Oct 29 14:02:33 2010
New Revision: 78466
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py
Log:
(hakanardo, arigo)
Argh, these tests.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py Fri Oct 29 14:02:33 2010
@@ -149,9 +149,9 @@
def foo(self):
return 3
def externfn(y):
- if y % 4 == 0: return W1()
- elif y % 4 == 3: return W2()
- else: return W3()
+ lst = [W1, W1, W2, W2, W3, W3, W2, W1, W3]
+ W = lst[y % len(lst)]
+ return W()
def f(y):
while y > 0:
myjitdriver.can_enter_jit(y=y)
@@ -164,7 +164,8 @@
for j in range(69, 75):
res = self.meta_interp(f, [j], policy=policy)
assert res == 42
- self.check_loop_count(3)
+ self.check_enter_count(5)
+ self.check_loop_count(5)
def test_oosend_guard_failure(self):
myjitdriver = JitDriver(greens = [], reds = ['x', 'y', 'w'])
From afa at codespeak.net Fri Oct 29 14:08:46 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 14:08:46 +0200 (CEST)
Subject: [pypy-svn] r78467 - in pypy/branch/fast-forward/pypy/rlib: . test
Message-ID: <20101029120846.798CA282C1E@codespeak.net>
Author: afa
Date: Fri Oct 29 14:08:44 2010
New Revision: 78467
Modified:
pypy/branch/fast-forward/pypy/rlib/rzlib.py
pypy/branch/fast-forward/pypy/rlib/test/test_rzlib.py
Log:
Improve zlib error messages: in the Z_FINISH case,
call decompress() again with a size of zero.
Modified: pypy/branch/fast-forward/pypy/rlib/rzlib.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rzlib.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rzlib.py Fri Oct 29 14:08:44 2010
@@ -22,7 +22,7 @@
constantnames = '''
- Z_OK Z_STREAM_ERROR Z_BUF_ERROR Z_MEM_ERROR Z_STREAM_END
+ Z_OK Z_STREAM_ERROR Z_BUF_ERROR Z_MEM_ERROR Z_STREAM_END Z_DATA_ERROR
Z_DEFLATED Z_DEFAULT_STRATEGY Z_DEFAULT_COMPRESSION
Z_NO_FLUSH Z_FINISH Z_SYNC_FLUSH Z_FULL_FLUSH
MAX_WBITS MAX_MEM_LEVEL
@@ -200,10 +200,17 @@
"""
if stream.c_msg:
reason = rffi.charp2str(stream.c_msg)
+ elif err == Z_MEM_ERROR:
+ reason = "out of memory"
+ elif err == Z_BUF_ERROR:
+ reason = "incomplete or truncated stream"
+ elif err == Z_STREAM_ERROR:
+ reason = "inconsistent stream state"
+ elif err == Z_DATA_ERROR:
+ reason = "invalid input data"
else:
reason = ""
- if not reason and err == Z_MEM_ERROR:
- reason = "out of memory"
+
if reason:
delim = ": "
else:
@@ -313,15 +320,17 @@
should_finish = True
else:
should_finish = False
- result = _operate(stream, data, flush, max_length, _inflate,
- "while decompressing")
+ while_doing = "while decompressing data"
+ data, err, avail_in = _operate(stream, data, flush, max_length, _inflate,
+ while_doing)
if should_finish:
- # detect incomplete input in the Z_FINISHED case
- finished = result[1]
- if not finished:
- raise RZlibError("the input compressed stream of data is "
- "incomplete")
- return result
+ # detect incomplete input
+ rffi.setintfield(stream, 'c_avail_in', 0)
+ err = _inflate(stream, Z_FINISH)
+ if err < 0:
+ raise RZlibError.fromstream(stream, err, while_doing)
+ finished = (err == Z_STREAM_END)
+ return data, finished, avail_in
def _operate(stream, data, flush, max_length, cfunc, while_doing):
@@ -393,5 +402,5 @@
# then the zlib simply returns Z_OK and waits for more. If it is
# complete it returns Z_STREAM_END.
return (''.join(result),
- err == Z_STREAM_END,
+ err,
rffi.cast(lltype.Signed, stream.c_avail_in))
Modified: pypy/branch/fast-forward/pypy/rlib/test/test_rzlib.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/test/test_rzlib.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/test/test_rzlib.py Fri Oct 29 14:08:44 2010
@@ -165,9 +165,11 @@
assert finished2 is False
assert unused2 == 0
assert expanded.startswith(data)
- py.test.raises(rzlib.RZlibError,
- rzlib.decompress, stream, compressed[2000:-500],
- rzlib.Z_FINISH)
+ exc = py.test.raises(
+ rzlib.RZlibError,
+ rzlib.decompress, stream, compressed[2000:-500], rzlib.Z_FINISH)
+ msg = "Error -5 while decompressing data: incomplete or truncated stream"
+ assert str(exc.value) == msg
rzlib.inflateEnd(stream)
From cfbolz at codespeak.net Fri Oct 29 15:01:44 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Fri, 29 Oct 2010 15:01:44 +0200 (CEST)
Subject: [pypy-svn] r78468 - pypy/trunk/lib-python/modified-2.5.2/test
Message-ID: <20101029130144.D184036E0C1@codespeak.net>
Author: cfbolz
Date: Fri Oct 29 15:01:43 2010
New Revision: 78468
Modified:
pypy/trunk/lib-python/modified-2.5.2/test/test_descr.py
Log:
(arigo, cfbolz): we don't care that in pypy you can take weakrefs to objects
with slots without __weakref__
Modified: pypy/trunk/lib-python/modified-2.5.2/test/test_descr.py
==============================================================================
--- pypy/trunk/lib-python/modified-2.5.2/test/test_descr.py (original)
+++ pypy/trunk/lib-python/modified-2.5.2/test/test_descr.py Fri Oct 29 15:01:43 2010
@@ -2028,7 +2028,9 @@
except TypeError, msg:
verify(str(msg).find("weak reference") >= 0)
else:
- verify(0, "weakref.ref(no) should be illegal")
+ # in PyPy it is (sometimes) possible to take a weakref here
+ #verify(0, "weakref.ref(no) should be illegal")
+ pass
class Weak(object):
__slots__ = ['foo', '__weakref__']
yes = Weak()
From arigo at codespeak.net Fri Oct 29 15:11:14 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 29 Oct 2010 15:11:14 +0200 (CEST)
Subject: [pypy-svn] r78469 - in pypy/trunk/pypy: objspace/std/test rlib
rlib/test
Message-ID: <20101029131114.CB553282BE8@codespeak.net>
Author: arigo
Date: Fri Oct 29 15:11:13 2010
New Revision: 78469
Modified:
pypy/trunk/pypy/objspace/std/test/test_longobject.py
pypy/trunk/pypy/rlib/rbigint.py
pypy/trunk/pypy/rlib/test/test_rbigint.py
Log:
-0L == 0L.
Modified: pypy/trunk/pypy/objspace/std/test/test_longobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/test/test_longobject.py (original)
+++ pypy/trunk/pypy/objspace/std/test/test_longobject.py Fri Oct 29 15:11:13 2010
@@ -254,3 +254,7 @@
class myotherlong(long):
pass
assert long(myotherlong(21)) == 21L
+
+ def test_negative_zero(self):
+ x = eval("-0L")
+ assert x == 0L
Modified: pypy/trunk/pypy/rlib/rbigint.py
==============================================================================
--- pypy/trunk/pypy/rlib/rbigint.py (original)
+++ pypy/trunk/pypy/rlib/rbigint.py Fri Oct 29 15:11:13 2010
@@ -1631,7 +1631,7 @@
a = _muladd1(a, tens, dig)
tens = 1
dig = 0
- if sign:
+ if sign and a.sign == 1:
a.sign = -1
return a
@@ -1652,6 +1652,5 @@
else:
dig = dig * base + digit
tens *= base
- if parser.sign == -1:
- a.sign = -1
+ a.sign *= parser.sign
return a
Modified: pypy/trunk/pypy/rlib/test/test_rbigint.py
==============================================================================
--- pypy/trunk/pypy/rlib/test/test_rbigint.py (original)
+++ pypy/trunk/pypy/rlib/test/test_rbigint.py Fri Oct 29 15:11:13 2010
@@ -112,6 +112,23 @@
assert rbigint.fromrarith_int(r_uint(2*sys.maxint+1)).eq(
rbigint.fromlong(2*sys.maxint+1))
+ def test_fromdecimalstr(self):
+ x = rbigint.fromdecimalstr("12345678901234567890523897987")
+ assert x.tolong() == 12345678901234567890523897987L
+ assert x.tobool() is True
+ x = rbigint.fromdecimalstr("+12345678901234567890523897987")
+ assert x.tolong() == 12345678901234567890523897987L
+ assert x.tobool() is True
+ x = rbigint.fromdecimalstr("-12345678901234567890523897987")
+ assert x.tolong() == -12345678901234567890523897987L
+ assert x.tobool() is True
+ x = rbigint.fromdecimalstr("+0")
+ assert x.tolong() == 0
+ assert x.tobool() is False
+ x = rbigint.fromdecimalstr("-0")
+ assert x.tolong() == 0
+ assert x.tobool() is False
+
def test_add(self):
x = 123456789123456789000000L
y = 123858582373821923936744221L
@@ -470,6 +487,11 @@
num = num * 36 + i
x = parse_digit_string(Parser(16, -1, range(15,-1,-1)*99))
assert x.eq(rbigint.fromlong(long('-0x' + 'FEDCBA9876543210'*99, 16)))
+ assert x.tobool() is True
+ x = parse_digit_string(Parser(7, 1, [0, 0, 0]))
+ assert x.tobool() is False
+ x = parse_digit_string(Parser(7, -1, [0, 0, 0]))
+ assert x.tobool() is False
BASE = 2 ** SHIFT
From antocuni at codespeak.net Fri Oct 29 15:13:03 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri, 29 Oct 2010 15:13:03 +0200 (CEST)
Subject: [pypy-svn] r78470 - pypy/build/bot2-hg
Message-ID: <20101029131303.3E3C936E0CB@codespeak.net>
Author: antocuni
Date: Fri Oct 29 15:13:01 2010
New Revision: 78470
Added:
pypy/build/bot2-hg/
- copied from r78469, pypy/build/bot2/
Log:
a branch to make buildbot pulling pypy from a mercurial repo
From antocuni at codespeak.net Fri Oct 29 15:17:07 2010
From: antocuni at codespeak.net (antocuni at codespeak.net)
Date: Fri, 29 Oct 2010 15:17:07 +0200 (CEST)
Subject: [pypy-svn] r78471 - pypy/build/bot2-hg
Message-ID: <20101029131707.E1B7A282C24@codespeak.net>
Author: antocuni
Date: Fri Oct 29 15:17:06 2010
New Revision: 78471
Removed:
pypy/build/bot2-hg/
Log:
now that I think better of it, it makes more sense to develop this branch
directly on mercurial, doesn't it? :-)
From afa at codespeak.net Fri Oct 29 15:18:00 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 15:18:00 +0200 (CEST)
Subject: [pypy-svn] r78472 - in pypy/branch/fast-forward/pypy/module/imp: .
test
Message-ID: <20101029131800.2CB1336E3F0@codespeak.net>
Author: afa
Date: Fri Oct 29 15:17:58 2010
New Revision: 78472
Modified:
pypy/branch/fast-forward/pypy/module/imp/__init__.py
pypy/branch/fast-forward/pypy/module/imp/importing.py
pypy/branch/fast-forward/pypy/module/imp/test/test_app.py
pypy/branch/fast-forward/pypy/module/imp/test/test_import.py
Log:
Add imp.NullImporter. This also fixes "./test_all.py -A module/imp"
when run on top of CPython 2.7
Modified: pypy/branch/fast-forward/pypy/module/imp/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/imp/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/imp/__init__.py Fri Oct 29 15:17:58 2010
@@ -26,6 +26,7 @@
'is_builtin': 'interp_imp.is_builtin',
'is_frozen': 'interp_imp.is_frozen',
'reload': 'importing.reload',
+ 'NullImporter': 'importing.W_NullImporter',
'lock_held': 'interp_imp.lock_held',
'acquire_lock': 'interp_imp.acquire_lock',
Modified: pypy/branch/fast-forward/pypy/module/imp/importing.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/imp/importing.py (original)
+++ pypy/branch/fast-forward/pypy/module/imp/importing.py Fri Oct 29 15:17:58 2010
@@ -5,9 +5,10 @@
import sys, os, stat
from pypy.interpreter.module import Module
-from pypy.interpreter import gateway
+from pypy.interpreter.gateway import Arguments, interp2app, unwrap_spec
+from pypy.interpreter.typedef import TypeDef, generic_new_descr
from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.baseobjspace import W_Root, ObjSpace
+from pypy.interpreter.baseobjspace import W_Root, ObjSpace, Wrappable
from pypy.interpreter.eval import Code
from pypy.rlib import streamio, jit
from pypy.rlib.streamio import StreamErrors
@@ -256,8 +257,7 @@
w_path_importer_cache = space.sys.get("path_importer_cache")
w_importer = space.finditem(w_path_importer_cache, w_pathitem)
if w_importer is None:
- w_importer = space.w_None
- space.setitem(w_path_importer_cache, w_pathitem, w_importer)
+ space.setitem(w_path_importer_cache, w_pathitem, space.w_None)
for w_hook in space.unpackiterable(space.sys.get("path_hooks")):
try:
w_importer = space.call_function(w_hook, w_pathitem)
@@ -266,6 +266,8 @@
raise
else:
break
+ if w_importer is None:
+ w_importer = space.wrap(W_NullImporter(space))
if space.is_true(w_importer):
space.setitem(w_path_importer_cache, w_pathitem, w_importer)
if space.is_true(w_importer):
@@ -273,6 +275,37 @@
if space.is_true(w_loader):
return w_loader
+
+class W_NullImporter(Wrappable):
+ def __init__(self, space):
+ pass
+
+ @unwrap_spec('self', ObjSpace, str)
+ def descr_init(self, space, path):
+ if not path:
+ raise OperationError(space.w_ImportError, space.wrap(
+ "empty pathname"))
+
+ # Directory should not exist
+ try:
+ st = os.stat(path)
+ except OSError:
+ return # ok
+ if stat.S_ISDIR(st.st_mode):
+ raise OperationError(space.w_ImportError, space.wrap(
+ "existing directory"))
+
+ @unwrap_spec('self', ObjSpace, Arguments)
+ def find_module_w(self, space, __args__):
+ return space.wrap(None)
+
+W_NullImporter.typedef = TypeDef(
+ 'imp.NullImporter',
+ __new__=generic_new_descr(W_NullImporter),
+ __init__=interp2app(W_NullImporter.descr_init),
+ find_module=interp2app(W_NullImporter.find_module_w),
+ )
+
class FindInfo:
def __init__(self, modtype, filename, stream,
suffix="", filemode="", w_loader=None):
Modified: pypy/branch/fast-forward/pypy/module/imp/test/test_app.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/imp/test/test_app.py (original)
+++ pypy/branch/fast-forward/pypy/module/imp/test/test_app.py Fri Oct 29 15:17:58 2010
@@ -122,3 +122,9 @@
mod2 = self.imp.load_module('test_imp_extra_AUTO6', f, fn, descr)
f.close()
assert mod2 is mod
+
+ def test_nullimporter(self):
+ import os
+ importer = self.imp.NullImporter("path")
+ assert importer.find_module(1, 2, 3, 4) is None
+ raises(ImportError, self.imp.NullImporter, os.getcwd())
Modified: pypy/branch/fast-forward/pypy/module/imp/test/test_import.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/imp/test/test_import.py (original)
+++ pypy/branch/fast-forward/pypy/module/imp/test/test_import.py Fri Oct 29 15:17:58 2010
@@ -869,7 +869,7 @@
if path == "xxx":
return Importer()
raise ImportError()
- import sys
+ import sys, imp
try:
sys.path_hooks.append(importer_for_path)
sys.path.insert(0, "yyy")
@@ -879,7 +879,8 @@
import b
except ImportError:
pass
- assert sys.path_importer_cache['yyy'] is None
+ assert isinstance(sys.path_importer_cache['yyy'],
+ imp.NullImporter)
finally:
sys.path.pop(0)
sys.path.pop(0)
From afa at codespeak.net Fri Oct 29 15:21:25 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 15:21:25 +0200 (CEST)
Subject: [pypy-svn] r78473 - pypy/branch/fast-forward/pypy/module/imp
Message-ID: <20101029132125.DF9AE282C24@codespeak.net>
Author: afa
Date: Fri Oct 29 15:21:24 2010
New Revision: 78473
Modified:
pypy/branch/fast-forward/pypy/module/imp/importing.py
Log:
More robust syntax (I don't like "return" hidden in the code)
Modified: pypy/branch/fast-forward/pypy/module/imp/importing.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/imp/importing.py (original)
+++ pypy/branch/fast-forward/pypy/module/imp/importing.py Fri Oct 29 15:21:24 2010
@@ -290,10 +290,11 @@
try:
st = os.stat(path)
except OSError:
- return # ok
- if stat.S_ISDIR(st.st_mode):
- raise OperationError(space.w_ImportError, space.wrap(
- "existing directory"))
+ pass
+ else:
+ if stat.S_ISDIR(st.st_mode):
+ raise OperationError(space.w_ImportError, space.wrap(
+ "existing directory"))
@unwrap_spec('self', ObjSpace, Arguments)
def find_module_w(self, space, __args__):
From arigo at codespeak.net Fri Oct 29 16:23:29 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 29 Oct 2010 16:23:29 +0200 (CEST)
Subject: [pypy-svn] r78474 - pypy/trunk/pypy/jit/metainterp/optimizeopt
Message-ID: <20101029142329.0D2EB36E0C1@codespeak.net>
Author: arigo
Date: Fri Oct 29 16:23:27 2010
New Revision: 78474
Modified:
pypy/trunk/pypy/jit/metainterp/optimizeopt/rewrite.py
Log:
Simplify the logic and stick to the invariant
"getint() must *never* be called on a BoxInt, only on a ConstInt"
Modified: pypy/trunk/pypy/jit/metainterp/optimizeopt/rewrite.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/optimizeopt/rewrite.py (original)
+++ pypy/trunk/pypy/jit/metainterp/optimizeopt/rewrite.py Fri Oct 29 16:23:27 2010
@@ -244,14 +244,11 @@
self.optimizer.exception_might_have_happened = False
def optimize_CALL_LOOPINVARIANT(self, op):
- funcvalue = self.getvalue(op.getarg(0))
- if not funcvalue.is_constant():
- # XXX this code path is never executed in tests nor in production.
- # in fact, it can't even happen since residual_call in codewriter
- # expects a compile-time constant
- self.emit_operation(op)
- return
- key = make_hashable_int(op.getarg(0).getint())
+ arg = op.getarg(0)
+ # 'arg' must be a Const, because residual_call in codewriter
+ # expects a compile-time constant
+ assert isinstance(arg, Const)
+ key = make_hashable_int(arg.getint())
resvalue = self.optimizer.loop_invariant_results.get(key, None)
if resvalue is not None:
self.make_equal_to(op.result, resvalue)
From hakanardo at codespeak.net Fri Oct 29 16:43:28 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Fri, 29 Oct 2010 16:43:28 +0200 (CEST)
Subject: [pypy-svn] r78475 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: . optimizeopt test
Message-ID: <20101029144328.2EB3F282C25@codespeak.net>
Author: hakanardo
Date: Fri Oct 29 16:43:27 2010
New Revision: 78475
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/oparser.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(armin, hakanardo) separate into 3 loops
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Fri Oct 29 16:43:27 2010
@@ -9,8 +9,11 @@
distinction anymore)"""
def setup(self, virtuals):
- self.enabled = virtuals
-
+ self.enabled = virtuals
+ self.cloned_operations = []
+ for op in self.optimizer.loop.operations:
+ self.cloned_operations.append(op.clone())
+
def propagate_forward(self, op):
if not self.enabled:
self.emit_operation(op)
@@ -22,8 +25,7 @@
self.optimizer.newoperations = []
jump_args = op.getarglist()
op.initarglist([])
- inputargs = self.inline(loop.operations,
- #loop.preamble.operations + [op],
+ inputargs = self.inline(self.cloned_operations,
loop.inputargs, jump_args)
loop.inputargs = inputargs
jmp = ResOperation(rop.JUMP, loop.inputargs[:], None)
@@ -51,27 +53,24 @@
self.snapshot_map ={None: None}
inputargs = []
- seen = {}
+ seen_inputargs = {}
for arg in jump_args:
boxes = []
- self.getvalue(arg).enum_forced_boxes(boxes, seen)
+ self.getvalue(arg).enum_forced_boxes(boxes, seen_inputargs)
for a in boxes:
if not isinstance(a, Const):
inputargs.append(a)
- for op in loop_operations:
+ for newop in loop_operations:
#import pdb; pdb.set_trace()
- newop = op.clone()
newop.initarglist([self.inline_arg(a) for a in newop.getarglist()])
- if op.result:
- newop.result = op.result.clonebox()
- argmap[op.result] = newop.result
+ if newop.result:
+ old_result = newop.result
+ newop.result = newop.result.clonebox()
+ argmap[old_result] = newop.result
+
descr = newop.getdescr()
if isinstance(descr, ResumeGuardDescr):
- orgdescr = op.getdescr()
- assert isinstance(orgdescr, ResumeGuardDescr)
- orgdescr.rd_snapshot = None #FIXME: In the right place?
- descr.rd_numb = None
descr.rd_snapshot = self.inline_snapshot(descr.rd_snapshot)
if newop.getopnum() == rop.JUMP:
@@ -83,34 +82,55 @@
newop.initarglist(args + inputargs[len(args):])
#print 'P: ', str(newop)
- current = len(self.optimizer.newoperations)
+ #current = len(self.optimizer.newoperations)
self.emit_operation(newop)
+
# FIXME: force_lazy_setfield in heap.py may reorder last ops
- current = max(current-1, 0)
-
- for op in self.optimizer.newoperations[current:]:
- #print 'E: ', str(op)
- if op.is_guard():
- descr = op.getdescr()
- assert isinstance(descr, ResumeGuardDescr)
- descr.rd_snapshot = None #FIXME: In the right place?
- args = op.getarglist()
- if op.is_guard():
- args = args + op.getfailargs()
- #if op.getopnum() == rop.SETFIELD_GC:
- # import pdb; pdb.set_trace()
- for a in args:
- if not isinstance(a, Const) and a in self.optimizer.values:
- v = self.getvalue(a)
- if v.fromstart and a not in inputargs:
+ #current = max(current-1, 0)
+
+ jmp = self.optimizer.newoperations[-1]
+ assert jmp.getopnum() == rop.JUMP
+ jumpargs = jmp.getarglist()
+ for op in self.optimizer.newoperations:
+ print 'E: ', str(op)
+ #if op.is_guard():
+ # descr = op.getdescr()
+ # assert isinstance(descr, ResumeGuardDescr)
+ # descr.rd_snapshot = None #FIXME: In the right place?
+ args = op.getarglist()
+ if op.is_guard():
+ #new_failargs = []
+ #for a in op.getfailargs():
+ # new_failargs.append(self.getvalue(self.inline_arg(a)).force_box())
+ #op.setfailargs(new_failargs)
+
+ args = args + op.getfailargs()
+ print "Args: ", args
+
+ for a in args:
+ if not isinstance(a, Const) and a in self.optimizer.values:
+ v = self.getvalue(a)
+ if v.fromstart and not v.is_constant():
+ a = v.force_box()
+ if a not in inputargs:
inputargs.append(a)
- jmp = self.optimizer.newoperations[-1]
- if jmp.getopnum() == rop.JUMP:
- newval = self.getvalue(argmap[a])
- boxes = jmp.getarglist()
- boxes.append(newval.force_box())
- jmp.initarglist(boxes)
+ newval = self.getvalue(argmap[a])
+ jumpargs.append(newval.force_box())
+
+
+ #boxes = []
+ #v.enum_forced_boxes(boxes, seen_inputargs)
+ #for b in boxes:
+ # if not isinstance(b, Const):
+ # inputargs.append(b)
+ # jumpargs.append(b)
+
+ #newval = self.getvalue(argmap[a])
+ #jumpargs.append(newval.force_box())
+ jmp.initarglist(jumpargs)
+
+ print "Inputargs: ", inputargs
return inputargs
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py Fri Oct 29 16:43:27 2010
@@ -298,7 +298,7 @@
self.liveboxes_from_env = liveboxes_from_env
self.liveboxes = {}
storage.rd_numb = numb
- #storage.rd_snapshot = None FIXME!!
+ storage.rd_snapshot = None
# collect liveboxes and virtuals
n = len(liveboxes_from_env) - v
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/oparser.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/oparser.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/oparser.py Fri Oct 29 16:43:27 2010
@@ -31,6 +31,9 @@
def getopnum(self):
return self.OPNUM
+ def clone(self):
+ return ESCAPE_OP(self.OPNUM, self.getarglist()[:], self.result, self.getdescr())
+
class ExtendedTreeLoop(TreeLoop):
def getboxes(self):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Fri Oct 29 16:43:27 2010
@@ -3092,8 +3092,8 @@
jump(i2)
'''
expected = '''
- [i0]
- jump(1)
+ []
+ jump()
'''
self.optimize_loop(ops, 'Not', expected)
@@ -3109,28 +3109,30 @@
jump(i2)
'''
expected = '''
- [i0]
- jump(3)
+ []
+ jump()
'''
self.optimize_loop(ops, 'Not', expected)
def test_arraycopy_not_virtual(self):
ops = '''
- [p0]
+ []
p1 = new_array(3, descr=arraydescr)
p2 = new_array(3, descr=arraydescr)
setarrayitem_gc(p1, 2, 10, descr=arraydescr)
setarrayitem_gc(p2, 2, 13, descr=arraydescr)
call(0, p1, p2, 0, 0, 3, descr=arraycopydescr)
- jump(p2)
+ escape(p2)
+ jump()
'''
expected = '''
- [p0]
+ []
p2 = new_array(3, descr=arraydescr)
setarrayitem_gc(p2, 2, 10, descr=arraydescr)
- jump(p2)
+ escape(p2)
+ jump()
'''
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, '', expected)
def test_arraycopy_no_elem(self):
""" this was actually observed in the wild
From hakanardo at codespeak.net Fri Oct 29 16:49:36 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Fri, 29 Oct 2010 16:49:36 +0200 (CEST)
Subject: [pypy-svn] r78476 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt
Message-ID: <20101029144936.4B94F36E0C1@codespeak.net>
Author: hakanardo
Date: Fri Oct 29 16:49:34 2010
New Revision: 78476
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
Log:
(armin, hakanardo) cleanups
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Fri Oct 29 16:49:34 2010
@@ -48,10 +48,8 @@
for op in self.optimizer.pure_operations.values():
v = self.getvalue(op.result)
v.fromstart = True
-
- self.snapshot_map ={None: None}
-
+ self.snapshot_map = {None: None}
inputargs = []
seen_inputargs = {}
for arg in jump_args:
@@ -62,7 +60,6 @@
inputargs.append(a)
for newop in loop_operations:
- #import pdb; pdb.set_trace()
newop.initarglist([self.inline_arg(a) for a in newop.getarglist()])
if newop.result:
old_result = newop.result
@@ -75,38 +72,22 @@
if newop.getopnum() == rop.JUMP:
args = []
- #for arg in newop.getarglist():
for arg in inputargs:
arg = argmap[arg]
args.append(self.getvalue(arg).force_box())
newop.initarglist(args + inputargs[len(args):])
#print 'P: ', str(newop)
- #current = len(self.optimizer.newoperations)
self.emit_operation(newop)
-
- # FIXME: force_lazy_setfield in heap.py may reorder last ops
- #current = max(current-1, 0)
-
jmp = self.optimizer.newoperations[-1]
assert jmp.getopnum() == rop.JUMP
jumpargs = jmp.getarglist()
for op in self.optimizer.newoperations:
- print 'E: ', str(op)
- #if op.is_guard():
- # descr = op.getdescr()
- # assert isinstance(descr, ResumeGuardDescr)
- # descr.rd_snapshot = None #FIXME: In the right place?
+ #print 'E: ', str(op)
args = op.getarglist()
if op.is_guard():
- #new_failargs = []
- #for a in op.getfailargs():
- # new_failargs.append(self.getvalue(self.inline_arg(a)).force_box())
- #op.setfailargs(new_failargs)
-
args = args + op.getfailargs()
- print "Args: ", args
for a in args:
if not isinstance(a, Const) and a in self.optimizer.values:
@@ -118,20 +99,7 @@
newval = self.getvalue(argmap[a])
jumpargs.append(newval.force_box())
-
- #boxes = []
- #v.enum_forced_boxes(boxes, seen_inputargs)
- #for b in boxes:
- # if not isinstance(b, Const):
- # inputargs.append(b)
- # jumpargs.append(b)
-
- #newval = self.getvalue(argmap[a])
- #jumpargs.append(newval.force_box())
jmp.initarglist(jumpargs)
-
- print "Inputargs: ", inputargs
-
return inputargs
def inline_arg(self, arg):
From arigo at codespeak.net Fri Oct 29 17:02:10 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 29 Oct 2010 17:02:10 +0200 (CEST)
Subject: [pypy-svn] r78478 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101029150210.83BB5282C1E@codespeak.net>
Author: arigo
Date: Fri Oct 29 17:02:08 2010
New Revision: 78478
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
clone_if_mutable() must really clone now.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Fri Oct 29 17:02:08 2010
@@ -215,7 +215,7 @@
def __eq__(self, other):
return type(self) is type(other) # xxx obscure
def clone_if_mutable(self):
- return self
+ return Storage(self.metainterp_sd, self.original_greenkey)
def _sortboxes(boxes):
_kind2count = {history.INT: 1, history.REF: 2, history.FLOAT: 3}
From arigo at codespeak.net Fri Oct 29 17:19:45 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Fri, 29 Oct 2010 17:19:45 +0200 (CEST)
Subject: [pypy-svn] r78479 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101029151945.2854236E0CB@codespeak.net>
Author: arigo
Date: Fri Oct 29 17:19:43 2010
New Revision: 78479
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
Whack whack more.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Fri Oct 29 17:19:43 2010
@@ -215,7 +215,9 @@
def __eq__(self, other):
return type(self) is type(other) # xxx obscure
def clone_if_mutable(self):
- return Storage(self.metainterp_sd, self.original_greenkey)
+ res = Storage(self.metainterp_sd, self.original_greenkey)
+ self.copy_all_attrbutes_into(res)
+ return res
def _sortboxes(boxes):
_kind2count = {history.INT: 1, history.REF: 2, history.FLOAT: 3}
From hakanardo at codespeak.net Fri Oct 29 17:28:58 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Fri, 29 Oct 2010 17:28:58 +0200 (CEST)
Subject: [pypy-svn] r78480 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101029152858.797FA282BE8@codespeak.net>
Author: hakanardo
Date: Fri Oct 29 17:28:51 2010
New Revision: 78480
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(armin, hakanardo) fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Fri Oct 29 17:28:51 2010
@@ -242,7 +242,8 @@
assert equaloplists(optimized.operations,
expected.operations, False, remap)
- def optimize_loop(self, ops, spectext, optops, checkspecnodes=True):
+ def optimize_loop(self, ops, spectext, optops, checkspecnodes=True,
+ expected_preamble=None):
loop = self.parse(ops)
#
if checkspecnodes:
@@ -259,6 +260,7 @@
#
self.loop = loop
loop.preamble = TreeLoop('preamble')
+ loop.preamble.inputargs = loop.inputargs
metainterp_sd = FakeMetaInterpStaticData(self.cpu)
if hasattr(self, 'vrefinfo'):
metainterp_sd.virtualref_info = self.vrefinfo
@@ -270,20 +272,20 @@
print '\n'.join([str(o) for o in loop.operations])
self.assert_equal(loop, expected)
+ if expected_preamble:
+ expected = self.parse(expected_preamble)
+ self.assert_equal(loop.preamble, expected)
+
def test_simple(self):
ops = """
- [i]
- i0 = int_sub(i, 1)
- guard_value(i0, 0) [i0]
- jump(i)
- """
- expected = """
- [i]
- i0 = int_sub(i, 1)
- guard_value(i0, 0) [i0]
- jump(1)
+ []
+ f = escape()
+ f0 = float_sub(f, 1.0)
+ guard_value(f0, 0.0) [f0]
+ escape(f)
+ jump()
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, '', ops)
def test_constant_propagate(self):
ops = """
@@ -363,12 +365,16 @@
guard_class(p0, ConstClass(node_vtable)) []
jump(p0)
"""
- expected = """
+ preamble = """
[p0]
guard_class(p0, ConstClass(node_vtable)) []
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ [p0]
+ jump(p0)
+ """
+ self.optimize_loop(ops, 'Not', expected, expected_preamble=preamble)
def test_remove_guard_class_2(self):
ops = """
@@ -3718,41 +3724,48 @@
def test_bound_eq(self):
ops = """
- [i0, i1]
+ []
+ i0 = escape()
+ i1 = escape()
i2 = int_le(i0, 4)
guard_true(i2) []
i3 = int_eq(i0, i1)
guard_true(i3) []
i4 = int_lt(i1, 5)
guard_true(i4) []
- jump(i0, i1)
+ jump()
"""
expected = """
- [i0, i1]
+ []
+ i0 = escape()
+ i1 = escape()
i2 = int_le(i0, 4)
guard_true(i2) []
i3 = int_eq(i0, i1)
guard_true(i3) []
- jump(i0, i1)
+ jump()
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, '', expected)
def test_bound_eq_const(self):
ops = """
- [i0]
+ []
+ i0 = escape()
i1 = int_eq(i0, 7)
guard_true(i1) []
i2 = int_add(i0, 3)
- jump(i2)
+ escape(i2)
+ jump()
"""
expected = """
- [i0]
+ []
+ i0 = escape()
i1 = int_eq(i0, 7)
guard_true(i1) []
- jump(10)
-
+ escape(10)
+ jump()
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, '', expected)
def test_bound_eq_const_not(self):
ops = """
From fijal at codespeak.net Fri Oct 29 18:33:06 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Fri, 29 Oct 2010 18:33:06 +0200 (CEST)
Subject: [pypy-svn] r78481 - pypy/trunk/pypy/jit/tool
Message-ID: <20101029163306.98AA536E3BC@codespeak.net>
Author: fijal
Date: Fri Oct 29 18:33:04 2010
New Revision: 78481
Modified:
pypy/trunk/pypy/jit/tool/showstats.py
Log:
fix
Modified: pypy/trunk/pypy/jit/tool/showstats.py
==============================================================================
--- pypy/trunk/pypy/jit/tool/showstats.py (original)
+++ pypy/trunk/pypy/jit/tool/showstats.py Fri Oct 29 18:33:04 2010
@@ -10,9 +10,9 @@
def main(argv):
log = logparser.parse_log_file(argv[0])
- parts = logparser.extract_category(log, "jit-log-opt-")
+ parts = logparser.extract_category(log, "jit-log-noopt-")
for i, oplist in enumerate(parts):
- loop = parse(oplist, no_namespace=True)
+ loop = parse(oplist, no_namespace=True, nonstrict=True)
num_ops = 0
num_dmp = 0
num_guards = 0
@@ -23,7 +23,10 @@
num_ops += 1
if op.is_guard():
num_guards += 1
- print "Loop #%d, length: %d, opcodes: %d, guards: %d, %f" % (i, num_ops, num_dmp, num_guards, num_ops/num_dmp)
+ if num_dmp == 0:
+ print "Loop #%d, length: %d, opcodes: %d, guards: %d" % (i, num_ops, num_dmp, num_guards)
+ else:
+ print "Loop #%d, length: %d, opcodes: %d, guards: %d, %f" % (i, num_ops, num_dmp, num_guards, num_ops/num_dmp)
if __name__ == '__main__':
main(sys.argv[1:])
From afa at codespeak.net Fri Oct 29 18:36:48 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 18:36:48 +0200 (CEST)
Subject: [pypy-svn] r78482 - in
pypy/branch/fast-forward/pypy/module/zipimport: . test
Message-ID: <20101029163648.1B92A282BE8@codespeak.net>
Author: afa
Date: Fri Oct 29 18:36:46 2010
New Revision: 78482
Modified:
pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py
pypy/branch/fast-forward/pypy/module/zipimport/test/test_zipimport.py
Log:
Fix in zipimport: get_code() is supposed to return a Code object,
not the content of the .pyc file!
Modified: pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py (original)
+++ pypy/branch/fast-forward/pypy/module/zipimport/interp_zipimport.py Fri Oct 29 18:36:46 2010
@@ -183,12 +183,20 @@
return False
return True
+ def can_use_pyc(self, space, filename, magic, timestamp):
+ if magic != importing.get_pyc_magic(space):
+ return False
+ if self.check_newer_pyfile(space, filename[:-1], timestamp):
+ return False
+ if not self.check_compatible_mtime(space, filename, timestamp):
+ return False
+ return True
+
def import_pyc_file(self, space, modname, filename, buf, pkgpath):
w = space.wrap
magic = importing._get_long(buf[:4])
timestamp = importing._get_long(buf[4:8])
- if (self.check_newer_pyfile(space, filename[:-1], timestamp) or
- not self.check_compatible_mtime(space, filename, timestamp)):
+ if not self.can_use_pyc(space, filename, magic, timestamp):
return self.import_py_file(space, modname, filename[:-1], buf,
pkgpath)
buf = buf[8:] # XXX ugly copy, should use sequential read instead
@@ -276,16 +284,22 @@
def get_code(self, space, fullname):
filename = self.make_filename(fullname)
- w = space.wrap
for compiled, _, ext in ENUMERATE_EXTS:
if self.have_modulefile(space, filename + ext):
+ w_source = self.get_data(space, filename + ext)
+ source = space.str_w(w_source)
if compiled:
- return self.get_data(space, filename + ext)
+ magic = importing._get_long(source[:4])
+ timestamp = importing._get_long(source[4:8])
+ if not self.can_use_pyc(space, filename + ext,
+ magic, timestamp):
+ continue
+ code_w = importing.read_compiled_module(
+ space, filename + ext, source[8:])
else:
- w_source = self.get_data(space, filename + ext)
- w_code = space.builtin.call('compile', w_source,
- w(filename + ext), w('exec'))
- return w_code
+ code_w = importing.parse_source_module(
+ space, filename + ext, source)
+ return space.wrap(code_w)
raise operationerrfmt(self.w_ZipImportError,
"Cannot find source or code for %s in %s", filename, self.name)
get_code.unwrap_spec = ['self', ObjSpace, str]
@@ -386,7 +400,7 @@
get_data = interp2app(W_ZipImporter.get_data),
get_code = interp2app(W_ZipImporter.get_code),
get_source = interp2app(W_ZipImporter.get_source),
- _get_filename = interp2app(W_ZipImporter.get_filename),
+ get_filename = interp2app(W_ZipImporter.get_filename),
is_package = interp2app(W_ZipImporter.is_package),
load_module = interp2app(W_ZipImporter.load_module),
archive = GetSetProperty(W_ZipImporter.getarchive),
Modified: pypy/branch/fast-forward/pypy/module/zipimport/test/test_zipimport.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/zipimport/test/test_zipimport.py (original)
+++ pypy/branch/fast-forward/pypy/module/zipimport/test/test_zipimport.py Fri Oct 29 18:36:46 2010
@@ -175,7 +175,12 @@
assert mod.__file__.endswith('.zip'+os.sep+'uuu.pyc')
assert mod.get_file() == mod.__file__
assert mod.get_name() == mod.__name__
-
+ #
+ import zipimport
+ z = zipimport.zipimporter(self.zipfile)
+ code = z.get_code('uuu')
+ assert isinstance(code, type((lambda:0).func_code))
+
def test_bad_pyc(self):
import zipimport
import sys
@@ -206,6 +211,7 @@
z = zipimport.zipimporter(self.zipfile)
sys.modules['uuu'] = lambda x : x + 1
raises(ImportError, z.load_module, 'uuu')
+ raises(zipimport.ZipImportError, z.get_code, 'uuu')
def test_package(self):
import os, sys
@@ -270,7 +276,7 @@
assert z.get_source('xx') == "5"
assert z.archive == self.zipfile
mod = z.load_module('xx')
- assert z._get_filename('xx') == mod.__file__
+ assert z.get_filename('xx') == mod.__file__
def test_archive(self):
"""
@@ -300,7 +306,7 @@
z = zipimport.zipimporter(self.zipfile + "/directory")
mod = z.load_module("package")
assert z.is_package("package")
- assert z._get_filename("package") == mod.__file__
+ assert z.get_filename("package") == mod.__file__
def test_zip_directory_cache(self):
""" Check full dictionary interface
From fijal at codespeak.net Fri Oct 29 20:50:52 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Fri, 29 Oct 2010 20:50:52 +0200 (CEST)
Subject: [pypy-svn] r78484 - in pypy/trunk/pypy/jit: backend backend/llgraph
backend/x86 metainterp
Message-ID: <20101029185052.29F3636E0CB@codespeak.net>
Author: fijal
Date: Fri Oct 29 20:50:49 2010
New Revision: 78484
Modified:
pypy/trunk/pypy/jit/backend/llgraph/runner.py
pypy/trunk/pypy/jit/backend/model.py
pypy/trunk/pypy/jit/backend/x86/assembler.py
pypy/trunk/pypy/jit/backend/x86/runner.py
pypy/trunk/pypy/jit/metainterp/compile.py
Log:
Introduce log=bool flag on compile_loop and compile_bridge. The idea is that
if we want to log something about the loop (like say number of times it was
called), we want to skip loops that are not logged to normal log.
Modified: pypy/trunk/pypy/jit/backend/llgraph/runner.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/llgraph/runner.py (original)
+++ pypy/trunk/pypy/jit/backend/llgraph/runner.py Fri Oct 29 20:50:49 2010
@@ -118,13 +118,13 @@
self._descrs[key] = descr
return descr
- def compile_bridge(self, faildescr, inputargs, operations):
+ def compile_bridge(self, faildescr, inputargs, operations, log=True):
c = llimpl.compile_start()
self._compile_loop_or_bridge(c, inputargs, operations)
old, oldindex = faildescr._compiled_fail
llimpl.compile_redirect_fail(old, oldindex, c)
- def compile_loop(self, inputargs, operations, loopdescr):
+ def compile_loop(self, inputargs, operations, loopdescr, log=True):
"""In a real assembler backend, this should assemble the given
list of operations. Here we just generate a similar CompiledLoop
instance. The code here is RPython, whereas the code in llimpl
Modified: pypy/trunk/pypy/jit/backend/model.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/model.py (original)
+++ pypy/trunk/pypy/jit/backend/model.py Fri Oct 29 20:50:49 2010
@@ -33,14 +33,14 @@
pass
- def compile_loop(self, inputargs, operations, looptoken):
+ def compile_loop(self, inputargs, operations, looptoken, log=True):
"""Assemble the given loop.
Extra attributes should be put in the LoopToken to
point to the compiled loop in assembler.
"""
raise NotImplementedError
- def compile_bridge(self, faildescr, inputargs, operations):
+ def compile_bridge(self, faildescr, inputargs, operations, log=True):
"""Assemble the bridge.
The FailDescr is the descr of the original guard that failed.
"""
Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/assembler.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/assembler.py Fri Oct 29 20:50:49 2010
@@ -295,7 +295,7 @@
self.mc.RET()
self.mc.done()
- def assemble_loop(self, inputargs, operations, looptoken):
+ def assemble_loop(self, inputargs, operations, looptoken, log):
"""adds the following attributes to looptoken:
_x86_loop_code (an integer giving an address)
_x86_bootstrap_code (an integer giving an address)
@@ -310,10 +310,11 @@
self.setup()
funcname = self._find_debug_merge_point(operations)
-
+ if log:
+ self._register_counter()
+ operations = self._inject_debugging_code(operations)
regalloc = RegAlloc(self, self.cpu.translate_support_code)
- operations = self._inject_debugging_code(operations)
arglocs = regalloc.prepare_loop(inputargs, operations, looptoken)
looptoken._x86_arglocs = arglocs
@@ -340,13 +341,16 @@
self.mc.end_function()
self.write_pending_failure_recoveries()
- def assemble_bridge(self, faildescr, inputargs, operations):
+ def assemble_bridge(self, faildescr, inputargs, operations, log):
if not we_are_translated():
# Arguments should be unique
assert len(set(inputargs)) == len(inputargs)
self.setup()
funcname = self._find_debug_merge_point(operations)
+ if log:
+ self._register_counter()
+ operations = self._inject_debugging_code(operations)
arglocs = self.rebuild_faillocs_from_descr(
faildescr._x86_failure_recovery_bytecode)
@@ -401,12 +405,14 @@
else:
funcname = "" % len(self.loop_run_counters)
# invent the counter, so we don't get too confused
+ return funcname
+
+ def _register_counter(self):
if self._debug:
struct = lltype.malloc(DEBUG_COUNTER, flavor='raw',
track_allocation=False) # known to leak
struct.i = 0
self.loop_run_counters.append((len(self.loop_run_counters), struct))
- return funcname
def patch_jump_for_descr(self, faildescr, adr_new_target):
adr_jump_offset = faildescr._x86_adr_jump_offset
Modified: pypy/trunk/pypy/jit/backend/x86/runner.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/x86/runner.py (original)
+++ pypy/trunk/pypy/jit/backend/x86/runner.py Fri Oct 29 20:50:49 2010
@@ -49,11 +49,13 @@
self.assembler.finish_once()
self.profile_agent.shutdown()
- def compile_loop(self, inputargs, operations, looptoken):
- self.assembler.assemble_loop(inputargs, operations, looptoken)
-
- def compile_bridge(self, faildescr, inputargs, operations):
- self.assembler.assemble_bridge(faildescr, inputargs, operations)
+ def compile_loop(self, inputargs, operations, looptoken, log=True):
+ self.assembler.assemble_loop(inputargs, operations, looptoken,
+ log=log)
+
+ def compile_bridge(self, faildescr, inputargs, operations, log=True):
+ self.assembler.assemble_bridge(faildescr, inputargs, operations,
+ log=log)
def set_future_value_int(self, index, intvalue):
self.assembler.fail_boxes_int.setitem(index, intvalue)
Modified: pypy/trunk/pypy/jit/metainterp/compile.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/compile.py (original)
+++ pypy/trunk/pypy/jit/metainterp/compile.py Fri Oct 29 20:50:49 2010
@@ -599,5 +599,5 @@
ResOperation(rop.FINISH, finishargs, None, descr=jd.portal_finishtoken)
]
operations[1].setfailargs([])
- cpu.compile_loop(inputargs, operations, loop_token)
+ cpu.compile_loop(inputargs, operations, loop_token, log=False)
return loop_token
From agaynor at codespeak.net Fri Oct 29 20:59:16 2010
From: agaynor at codespeak.net (agaynor at codespeak.net)
Date: Fri, 29 Oct 2010 20:59:16 +0200 (CEST)
Subject: [pypy-svn] r78485 - pypy/extradoc/talk/pycon2011
Message-ID: <20101029185916.8CA3B282B9D@codespeak.net>
Author: agaynor
Date: Fri Oct 29 20:59:15 2010
New Revision: 78485
Modified:
pypy/extradoc/talk/pycon2011/pypy-optimizations.txt
Log:
MOre work on pycon-talk.
Modified: pypy/extradoc/talk/pycon2011/pypy-optimizations.txt
==============================================================================
--- pypy/extradoc/talk/pycon2011/pypy-optimizations.txt (original)
+++ pypy/extradoc/talk/pycon2011/pypy-optimizations.txt Fri Oct 29 20:59:15 2010
@@ -1,60 +1,44 @@
Title: Optimizations in PyPy
Description:
-PyPy is a virtual machine for Python, featuring an advanced just in time compiler, which can deliver exceptional performance. This talk is going to be a deep dive into what exactly makes Python such a hard language to optimize, how PyPy is organized, and what optimizations our JIT can do for your code.
+PyPy is a virtual machine for Python, featuring an advanced just in time
+compiler, which can deliver exceptional performance. This talk is going to be
+a deep dive into what exactly makes Python such a hard language to optimize,
+how PyPy is organized, and what optimizations our JIT can do for your code.
Session type: Talk
Classification: Discuss in depth
Abstract:
-The talk would contain a list of features that are hard to optimize
-in Python, why and how we're fixing it in PyPy. This also might be
-used as a rough guide what features in Python are optimizable right
-now and what are not.
+The talk would contain a list of features that are hard to optimize in Python,
+why and how we're fixing it in PyPy. This also might be used as a rough guide
+what features in Python are optimizable right now and what are not.
The list of mentioned features is not exhaustive, however we will try to
focus at least on the following:
-* Dynamic language - In Python code we have no known types, like a
- static language. Even operations like "a + b" can do anything,
- unless we know more about the code.
-
-* Frame introspection - Frame objects need to be allocated for every
- function call, and all local variables are stored on the frame, and
- must be accessible from further down the call stack.
-
-* Object model - All user defined Python objects have a dictionary which
- stores their attributes, as does every type. When Python does an
- attribute lookup this requires 2 dictionary lookups.
+* Dynamic language - In Python code we have no known types, like a static
+ language. Even operations like "a + b" can do anything, unless we know more
+ about the code.
+
+* Frame introspection - Frame objects need to be allocated for every function
+ call, and all local variables are stored on the frame, and must be accessible
+ from further down the call stack.
+
+* Object model - All user defined Python objects have a dictionary which stores
+ their attributes, as does every type. When Python does an attribute lookup
+ this requires 2 dictionary lookups.
In PyPy we use an approach similar to one used by v8 with hidden classes
(except more pypy specific) called shared dictionaries.
-* FFI calls - calling C from Python is costly and hard to optimize.
- In PyPy we decided to go via ctypes, this part will explain how are
- we going to handle ctypes calls to be fast.
-
-This talk is 3 parts, what's hard to optimize about Python, how PyPy is organized, and what optimizations our JIT can do.
-
-I. Why Python is hard (8 minutes)
- A) Dynamic language - No known types, even operations such as "+" can
- B) Frame introspection.
- C) Object model.
- D) CPython
- 1) Interpretation
- 2) Boxing
-II. How PyPy is organized (5 minutes)
- A) Written in Python
- B) RPython
- C) The JIT
- 1) Source transformation on the code
- 2) Orthogonal to the interpreter
-III. Optimizations the JIT does (13 minutes)
- A) Unboxing
- B) Attribute lookup
- 1) "Hidden classes"
- C) Globals/builtin "freezing"
- D) array module
- E) The future
- 1) ctypes
-IV. Questions (4 minutes)
+* FFI calls - calling C from Python is costly and hard to optimize. In PyPy we
+ decided to go via ctypes, this part will explain how are we going to handle
+ ctypes calls to be fast.
+
+* `array` module - users of the CPython array module probably know it can save
+ them quiet a bit of memory, however it's also slower than using a list, due
+ to the overhead of boxing and unboxing on every operations. Here we will tie
+ everything together and describe how the ``array`` module is much faster with
+ PyPy's JIT, combining our optimizations to: unbox values, remove the
+ dynamicism within traces, and deliver great performance.
From afa at codespeak.net Fri Oct 29 22:15:08 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 22:15:08 +0200 (CEST)
Subject: [pypy-svn] r78486 - pypy/branch/fast-forward/pypy/module/zlib
Message-ID: <20101029201508.60985282B9D@codespeak.net>
Author: afa
Date: Fri Oct 29 22:15:05 2010
New Revision: 78486
Modified:
pypy/branch/fast-forward/pypy/module/zlib/interp_zlib.py
Log:
Even it the decompressobj.flush() always returns "", mimic the checks done by CPython.
Modified: pypy/branch/fast-forward/pypy/module/zlib/interp_zlib.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/zlib/interp_zlib.py (original)
+++ pypy/branch/fast-forward/pypy/module/zlib/interp_zlib.py Fri Oct 29 22:15:05 2010
@@ -321,6 +321,9 @@
because each call to decompress() immediately returns as much
data as possible.
"""
+ if length <= 0:
+ raise OperationError(self.space.w_ValueError, self.space.wrap(
+ "length must be greater than zero"))
# We could call rzlib.decompress(self.stream, '', rzlib.Z_FINISH)
# which would complain if the input stream so far is not complete;
# however CPython's zlib module does not behave like that.
From afa at codespeak.net Fri Oct 29 22:30:01 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 22:30:01 +0200 (CEST)
Subject: [pypy-svn] r78487 - pypy/branch/fast-forward/pypy/module/_hashlib
Message-ID: <20101029203001.C30D2282B9E@codespeak.net>
Author: afa
Date: Fri Oct 29 22:30:00 2010
New Revision: 78487
Modified:
pypy/branch/fast-forward/pypy/module/_hashlib/__init__.py
Log:
Expose the same _hashlib items as CPython
Modified: pypy/branch/fast-forward/pypy/module/_hashlib/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_hashlib/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/_hashlib/__init__.py Fri Oct 29 22:30:00 2010
@@ -5,14 +5,13 @@
class Module(MixedModule):
interpleveldefs = {
'new' : 'interp_hashlib.new',
- 'HASH': 'interp_hashlib.W_Hash',
}
appleveldefs = {
}
for name in algorithms:
- interpleveldefs[name] = 'interp_hashlib.new_%s' % (name,)
+ interpleveldefs['openssl_' + name] = 'interp_hashlib.new_' + name
def startup(self, space):
from pypy.rlib.ropenssl import init_digests
From afa at codespeak.net Fri Oct 29 22:40:52 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 22:40:52 +0200 (CEST)
Subject: [pypy-svn] r78488 -
pypy/branch/fast-forward/pypy/module/_hashlib/test
Message-ID: <20101029204052.D84D3282B9E@codespeak.net>
Author: afa
Date: Fri Oct 29 22:40:51 2010
New Revision: 78488
Modified:
pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py
Log:
Fix test. hum
Modified: pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py (original)
+++ pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py Fri Oct 29 22:40:51 2010
@@ -7,7 +7,7 @@
def test_simple(self):
import _hashlib
- assert isinstance(_hashlib.new('md5'), _hashlib.HASH)
+ assert _hashlib.new('md5').__class__.__name__ == 'HASH'
assert len(_hashlib.new('md5').hexdigest()) == 32
def test_attributes(self):
@@ -59,5 +59,5 @@
def test_unicode(self):
import _hashlib
- assert isinstance(_hashlib.new('sha1', u'xxx'), _hashlib.HASH)
+ assert _hashlib.new('sha1', u'xxx').__class__.__name__ == 'HASH'
From afa at codespeak.net Fri Oct 29 23:05:43 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 23:05:43 +0200 (CEST)
Subject: [pypy-svn] r78489 - in pypy/branch/fast-forward/pypy/interpreter: .
test
Message-ID: <20101029210543.78979282BAD@codespeak.net>
Author: afa
Date: Fri Oct 29 23:05:41 2010
New Revision: 78489
Modified:
pypy/branch/fast-forward/pypy/interpreter/generator.py
pypy/branch/fast-forward/pypy/interpreter/test/test_generator.py
pypy/branch/fast-forward/pypy/interpreter/typedef.py
Log:
add a __repr__ to generators
Modified: pypy/branch/fast-forward/pypy/interpreter/generator.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/generator.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/generator.py Fri Oct 29 23:05:41 2010
@@ -13,6 +13,12 @@
self.frame = frame
self.running = False
+ def descr__repr__(self, space):
+ code_name = self.frame.pycode.co_name
+ addrstring = self.getaddrstring(space)
+ return space.wrap("" %
+ (code_name, addrstring))
+
def descr__reduce__(self, space):
from pypy.interpreter.mixedmodule import MixedModule
w_mod = space.getbuiltinmodule('_pickle_support')
Modified: pypy/branch/fast-forward/pypy/interpreter/test/test_generator.py
==============================================================================
--- pypy/branch/fast-forward/pypy/interpreter/test/test_generator.py (original)
+++ pypy/branch/fast-forward/pypy/interpreter/test/test_generator.py Fri Oct 29 23:05:41 2010
@@ -241,3 +241,8 @@
res = f()
""" in d
assert d['res'] == (10, 7)
+
+ def test_repr(self):
+ def myFunc():
+ yield 1
+ assert repr(myFunc()).startswith("
Author: afa
Date: Fri Oct 29 23:10:24 2010
New Revision: 78490
Added:
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_genexps.py
- copied, changed from r78483, pypy/branch/fast-forward/lib-python/2.7.0/test/test_genexps.py
Log:
Skip one CPython-specific test and update docstrings
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_genexps.py (from r78483, pypy/branch/fast-forward/lib-python/2.7.0/test/test_genexps.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/test/test_genexps.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_genexps.py Fri Oct 29 23:10:24 2010
@@ -128,9 +128,9 @@
Verify re-use of tuples (a side benefit of using genexps over listcomps)
- >>> tupleids = map(id, ((i,i) for i in xrange(10)))
- >>> int(max(tupleids) - min(tupleids))
- 0
+## >>> tupleids = map(id, ((i,i) for i in xrange(10)))
+## >>> int(max(tupleids) - min(tupleids))
+## 0
Verify that syntax error's are raised for genexps used as lvalues
@@ -204,7 +204,7 @@
g.next()
File "", line 1, in
g = (10 // i for i in (5, 0, 2))
- ZeroDivisionError: integer division or modulo by zero
+ ZeroDivisionError: integer division by zero
>>> g.next()
Traceback (most recent call last):
File "", line 1, in -toplevel-
@@ -224,7 +224,7 @@
True
>>> print g.next.__doc__
- x.next() -> the next value, or raise StopIteration
+ next() -> the next value, or raise StopIteration
>>> import types
>>> isinstance(g, types.GeneratorType)
True
From afa at codespeak.net Fri Oct 29 23:28:58 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 23:28:58 +0200 (CEST)
Subject: [pypy-svn] r78491 - in pypy/branch/fast-forward/pypy/rlib: . test
Message-ID: <20101029212858.BDC44282B9E@codespeak.net>
Author: afa
Date: Fri Oct 29 23:28:57 2010
New Revision: 78491
Modified:
pypy/branch/fast-forward/pypy/rlib/rmmap.py
pypy/branch/fast-forward/pypy/rlib/test/test_rmmap.py
Log:
Implement fix for CPython bug #2111:
mmap segfaults when trying to write a block opened with PROT_READ
Modified: pypy/branch/fast-forward/pypy/rlib/rmmap.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rmmap.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rmmap.py Fri Oct 29 23:28:57 2010
@@ -591,6 +591,9 @@
else:
raise RValueError("mmap invalid access parameter.")
+ if prot == PROT_READ:
+ access = ACCESS_READ
+
# check file size
try:
st = os.fstat(fd)
Modified: pypy/branch/fast-forward/pypy/rlib/test/test_rmmap.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/test/test_rmmap.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/test/test_rmmap.py Fri Oct 29 23:28:57 2010
@@ -229,6 +229,16 @@
interpret(func, [f.fileno()])
f.close()
+ def test_write_readonly(self):
+ if os.name == "nt":
+ skip("Needs PROT_READ")
+ f = open(self.tmpname + "l", "w+")
+ f.write("foobar")
+ f.flush()
+ m = mmap.mmap(f.fileno(), 6, prot=mmap.PROT_READ)
+ raises(RTypeError, m.write, "foo")
+ f.close()
+
def test_size(self):
f = open(self.tmpname + "l", "w+")
From afa at codespeak.net Fri Oct 29 23:41:28 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 23:41:28 +0200 (CEST)
Subject: [pypy-svn] r78492 -
pypy/branch/fast-forward/lib-python/modified-2.7.0/test
Message-ID: <20101029214128.1B21E282B9D@codespeak.net>
Author: afa
Date: Fri Oct 29 23:41:26 2010
New Revision: 78492
Added:
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_mmap.py
- copied, changed from r78483, pypy/branch/fast-forward/lib-python/2.7.0/test/test_mmap.py
Log:
Flush data before reopening the same file
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_mmap.py (from r78483, pypy/branch/fast-forward/lib-python/2.7.0/test/test_mmap.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/test/test_mmap.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_mmap.py Fri Oct 29 23:41:26 2010
@@ -118,7 +118,8 @@
def test_access_parameter(self):
# Test for "access" keyword parameter
mapsize = 10
- open(TESTFN, "wb").write("a"*mapsize)
+ with open(TESTFN, "wb") as f:
+ f.write("a"*mapsize)
f = open(TESTFN, "rb")
m = mmap.mmap(f.fileno(), mapsize, access=mmap.ACCESS_READ)
self.assertEqual(m[:], 'a'*mapsize, "Readonly memory map data incorrect.")
@@ -494,7 +495,8 @@
if not hasattr(mmap, 'PROT_READ'):
return
mapsize = 10
- open(TESTFN, "wb").write("a"*mapsize)
+ with open(TESTFN, "wb") as f:
+ f.write("a"*mapsize)
f = open(TESTFN, "rb")
m = mmap.mmap(f.fileno(), mapsize, prot=mmap.PROT_READ)
self.assertRaises(TypeError, m.write, "foo")
@@ -506,7 +508,8 @@
def test_io_methods(self):
data = "0123456789"
- open(TESTFN, "wb").write("x"*len(data))
+ with open(TESTFN, "wb") as f:
+ f.write("x"*len(data))
f = open(TESTFN, "r+b")
m = mmap.mmap(f.fileno(), len(data))
f.close()
From afa at codespeak.net Fri Oct 29 23:42:27 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 23:42:27 +0200 (CEST)
Subject: [pypy-svn] r78493 - pypy/branch/fast-forward/pypy/rlib
Message-ID: <20101029214227.7FA72282BEA@codespeak.net>
Author: afa
Date: Fri Oct 29 23:42:25 2010
New Revision: 78493
Modified:
pypy/branch/fast-forward/pypy/rlib/rmmap.py
Log:
Array bound check in mmap.write_byte()
Modified: pypy/branch/fast-forward/pypy/rlib/rmmap.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rmmap.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rmmap.py Fri Oct 29 23:42:25 2010
@@ -421,8 +421,11 @@
if len(byte) != 1:
raise RTypeError("write_byte() argument must be char")
-
+
self.check_writeable()
+ if self.pos >= self.size:
+ raise RValueError("write byte out of range")
+
self.data[self.pos] = byte[0]
self.pos += 1
From afa at codespeak.net Fri Oct 29 23:53:20 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 23:53:20 +0200 (CEST)
Subject: [pypy-svn] r78494 - in pypy/branch/fast-forward/pypy/module/mmap: .
test
Message-ID: <20101029215320.B248B36E0C1@codespeak.net>
Author: afa
Date: Fri Oct 29 23:53:19 2010
New Revision: 78494
Modified:
pypy/branch/fast-forward/pypy/module/mmap/__init__.py
pypy/branch/fast-forward/pypy/module/mmap/interp_mmap.py
pypy/branch/fast-forward/pypy/module/mmap/test/test_mmap.py
Log:
mmap.mmap is now a class (it was a function)
Modified: pypy/branch/fast-forward/pypy/module/mmap/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/mmap/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/mmap/__init__.py Fri Oct 29 23:53:19 2010
@@ -4,7 +4,7 @@
class Module(MixedModule):
interpleveldefs = {
'PAGESIZE': 'space.wrap(interp_mmap.PAGESIZE)',
- 'mmap': 'interp_mmap.mmap'
+ 'mmap': 'interp_mmap.W_MMap'
}
appleveldefs = {
Modified: pypy/branch/fast-forward/pypy/module/mmap/interp_mmap.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/mmap/interp_mmap.py (original)
+++ pypy/branch/fast-forward/pypy/module/mmap/interp_mmap.py Fri Oct 29 23:53:19 2010
@@ -188,7 +188,43 @@
return space.wrap(StringLikeBuffer(space, space.wrap(self)))
descr_buffer.unwrap_spec = ['self']
+if rmmap._POSIX:
+
+ def mmap(space, w_subtype, fileno, length, flags=rmmap.MAP_SHARED,
+ prot=rmmap.PROT_WRITE | rmmap.PROT_READ,
+ access=rmmap._ACCESS_DEFAULT):
+ self = space.allocate_instance(W_MMap, w_subtype)
+ try:
+ W_MMap.__init__(self, space,
+ rmmap.mmap(fileno, length, flags, prot, access))
+ except OSError, e:
+ raise wrap_oserror(space, e, exception_name='w_EnvironmentError')
+ except RValueError, e:
+ raise OperationError(space.w_ValueError, space.wrap(e.message))
+ except RTypeError, e:
+ raise OperationError(space.w_TypeError, space.wrap(e.message))
+ return space.wrap(self)
+ mmap.unwrap_spec = [ObjSpace, W_Root, int, 'index', int, int, int]
+
+elif rmmap._MS_WINDOWS:
+
+ def mmap(space, w_subtype, fileno, length, tagname="",
+ access=rmmap._ACCESS_DEFAULT):
+ self = space.allocate_instance(W_MMap, w_subtype)
+ try:
+ W_MMap.__init__(self, space,
+ rmmap.mmap(fileno, length, tagname, access))
+ except OSError, e:
+ raise wrap_oserror(space, e, exception_name='w_EnvironmentError')
+ except RValueError, e:
+ raise OperationError(space.w_ValueError, space.wrap(e.message))
+ except RTypeError, e:
+ raise OperationError(space.w_TypeError, space.wrap(e.message))
+ return space.wrap(self)
+ mmap.unwrap_spec = [ObjSpace, W_Root, int, 'index', str, int]
+
W_MMap.typedef = TypeDef("mmap",
+ __new__ = interp2app(mmap),
close = interp2app(W_MMap.close),
read_byte = interp2app(W_MMap.read_byte),
readline = interp2app(W_MMap.readline),
@@ -210,43 +246,5 @@
__buffer__ = interp2app(W_MMap.descr_buffer),
)
-def _check_map_size(space, size):
- if size < 0:
- raise OperationError(space.w_TypeError,
- space.wrap("memory mapped size must be positive"))
- if rffi.cast(size_t, size) != size:
- raise OperationError(space.w_OverflowError,
- space.wrap("memory mapped size is too large (limited by C int)"))
-
-if rmmap._POSIX:
-
- def mmap(space, fileno, length, flags=rmmap.MAP_SHARED,
- prot=rmmap.PROT_WRITE | rmmap.PROT_READ, access=rmmap._ACCESS_DEFAULT):
-
- try:
- return space.wrap(W_MMap(space, rmmap.mmap(fileno, length,
- flags, prot, access)))
- except OSError, e:
- raise wrap_oserror(space, e, exception_name='w_EnvironmentError')
- except RValueError, e:
- raise OperationError(space.w_ValueError, space.wrap(e.message))
- except RTypeError, e:
- raise OperationError(space.w_TypeError, space.wrap(e.message))
- mmap.unwrap_spec = [ObjSpace, int, 'index', int, int, int]
-
-elif rmmap._MS_WINDOWS:
-
- def mmap(space, fileno, length, tagname="", access=rmmap._ACCESS_DEFAULT):
- try:
- return space.wrap(W_MMap(space, rmmap.mmap(fileno, length,
- tagname, access)))
- except OSError, e:
- raise wrap_oserror(space, e, exception_name='w_EnvironmentError')
- except RValueError, e:
- raise OperationError(space.w_ValueError, space.wrap(e.message))
- except RTypeError, e:
- raise OperationError(space.w_TypeError, space.wrap(e.message))
- mmap.unwrap_spec = [ObjSpace, int, 'index', str, int]
-
constants = rmmap.constants
PAGESIZE = rmmap.PAGESIZE
Modified: pypy/branch/fast-forward/pypy/module/mmap/test/test_mmap.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/mmap/test/test_mmap.py (original)
+++ pypy/branch/fast-forward/pypy/module/mmap/test/test_mmap.py Fri Oct 29 23:53:19 2010
@@ -34,7 +34,8 @@
from mmap import mmap
import os
import sys
-
+
+ assert isinstance(mmap, type)
raises(TypeError, mmap, "foo")
raises(TypeError, mmap, 0, "foo")
From afa at codespeak.net Fri Oct 29 23:54:59 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Fri, 29 Oct 2010 23:54:59 +0200 (CEST)
Subject: [pypy-svn] r78495 - pypy/branch/fast-forward/pypy/module/mmap/test
Message-ID: <20101029215459.16D62282C24@codespeak.net>
Author: afa
Date: Fri Oct 29 23:54:57 2010
New Revision: 78495
Modified:
pypy/branch/fast-forward/pypy/module/mmap/test/test_mmap.py
Log:
Add a real test for a mmap subclass
Modified: pypy/branch/fast-forward/pypy/module/mmap/test/test_mmap.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/mmap/test/test_mmap.py (original)
+++ pypy/branch/fast-forward/pypy/module/mmap/test/test_mmap.py Fri Oct 29 23:54:57 2010
@@ -35,7 +35,6 @@
import os
import sys
- assert isinstance(mmap, type)
raises(TypeError, mmap, "foo")
raises(TypeError, mmap, 0, "foo")
@@ -53,6 +52,13 @@
raises(TypeError, mmap, 0, 1, access="foo")
raises(ValueError, mmap, 0, 1, access=-1)
+ def test_subclass(self):
+ import mmap
+ class anon_mmap(mmap.mmap):
+ def __new__(klass, *args, **kwargs):
+ return mmap.mmap.__new__(klass, -1, *args, **kwargs)
+ anon_mmap(mmap.PAGESIZE)
+
def test_file_size(self):
import os
if os.name == "nt":
From afa at codespeak.net Sat Oct 30 00:13:56 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Sat, 30 Oct 2010 00:13:56 +0200 (CEST)
Subject: [pypy-svn] r78496 - in pypy/branch/fast-forward/pypy: module/mmap
module/mmap/test rlib
Message-ID: <20101029221356.0DE8D282BD6@codespeak.net>
Author: afa
Date: Sat Oct 30 00:13:55 2010
New Revision: 78496
Modified:
pypy/branch/fast-forward/pypy/module/mmap/__init__.py
pypy/branch/fast-forward/pypy/module/mmap/interp_mmap.py
pypy/branch/fast-forward/pypy/module/mmap/test/test_mmap.py
pypy/branch/fast-forward/pypy/rlib/rmmap.py
Log:
Add mmap.ALLOCATIONGRANULARITY
Modified: pypy/branch/fast-forward/pypy/module/mmap/__init__.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/mmap/__init__.py (original)
+++ pypy/branch/fast-forward/pypy/module/mmap/__init__.py Sat Oct 30 00:13:55 2010
@@ -4,6 +4,7 @@
class Module(MixedModule):
interpleveldefs = {
'PAGESIZE': 'space.wrap(interp_mmap.PAGESIZE)',
+ 'ALLOCATIONGRANULARITY': 'space.wrap(interp_mmap.ALLOCATIONGRANULARITY)',
'mmap': 'interp_mmap.W_MMap'
}
Modified: pypy/branch/fast-forward/pypy/module/mmap/interp_mmap.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/mmap/interp_mmap.py (original)
+++ pypy/branch/fast-forward/pypy/module/mmap/interp_mmap.py Sat Oct 30 00:13:55 2010
@@ -248,3 +248,4 @@
constants = rmmap.constants
PAGESIZE = rmmap.PAGESIZE
+ALLOCATIONGRANULARITY = rmmap.ALLOCATIONGRANULARITY
Modified: pypy/branch/fast-forward/pypy/module/mmap/test/test_mmap.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/mmap/test/test_mmap.py (original)
+++ pypy/branch/fast-forward/pypy/module/mmap/test/test_mmap.py Sat Oct 30 00:13:55 2010
@@ -11,8 +11,11 @@
def test_page_size(self):
import mmap
assert mmap.PAGESIZE > 0
+ assert mmap.ALLOCATIONGRANULARITY > 0
assert isinstance(mmap.PAGESIZE, int)
-
+ assert isinstance(mmap.ALLOCATIONGRANULARITY, int)
+ assert mmap.ALLOCATIONGRANULARITY % mmap.PAGESIZE == 0
+
def test_attributes(self):
import mmap
import os
Modified: pypy/branch/fast-forward/pypy/rlib/rmmap.py
==============================================================================
--- pypy/branch/fast-forward/pypy/rlib/rmmap.py (original)
+++ pypy/branch/fast-forward/pypy/rlib/rmmap.py Sat Oct 30 00:13:55 2010
@@ -119,6 +119,7 @@
# this one is always safe
_, _get_page_size = external('getpagesize', [], rffi.INT)
+ _get_allocation_granularity = _get_page_size
def _get_error_no():
return rposix.get_errno()
@@ -195,7 +196,7 @@
VirtualFree = winexternal('VirtualFree',
[rffi.VOIDP, rffi.SIZE_T, DWORD], BOOL)
-
+
def _get_page_size():
try:
si = rffi.make(SYSTEM_INFO)
@@ -203,7 +204,15 @@
return int(si.c_dwPageSize)
finally:
lltype.free(si, flavor="raw")
-
+
+ def _get_allocation_granularity():
+ try:
+ si = rffi.make(SYSTEM_INFO)
+ GetSystemInfo(si)
+ return int(si.c_dwAllocationGranularity)
+ finally:
+ lltype.free(si, flavor="raw")
+
def _get_file_size(handle):
# XXX use native Windows types like WORD
high_ref = lltype.malloc(LPDWORD.TO, 1, flavor='raw')
@@ -232,6 +241,7 @@
INVALID_HANDLE = INVALID_HANDLE_VALUE
PAGESIZE = _get_page_size()
+ALLOCATIONGRANULARITY = _get_allocation_granularity()
NULL = lltype.nullptr(PTR.TO)
NODATA = lltype.nullptr(PTR.TO)
From afa at codespeak.net Sat Oct 30 01:06:43 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Sat, 30 Oct 2010 01:06:43 +0200 (CEST)
Subject: [pypy-svn] r78499 - in
pypy/branch/fast-forward/lib-python/modified-2.7.0: . test
Message-ID: <20101029230643.78BDE282B9D@codespeak.net>
Author: afa
Date: Sat Oct 30 01:06:41 2010
New Revision: 78499
Added:
pypy/branch/fast-forward/lib-python/modified-2.7.0/socket.py
- copied, changed from r78496, pypy/branch/fast-forward/lib-python/2.7.0/socket.py
Modified:
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_socket.py
pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py
Log:
Try to introduce py3k's idea of explicit reference counting of sockets
This is not the same because in py3k:
- py3k uses inheritance of the raw _socket.socket, not delegation
- the C module implements a real dup(), even on Windows
- socket.makefile does not take ownership of the fd
But it seems to work, and I could even remove most gc.collect() from the tests.
Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/socket.py (from r78496, pypy/branch/fast-forward/lib-python/2.7.0/socket.py)
==============================================================================
--- pypy/branch/fast-forward/lib-python/2.7.0/socket.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/socket.py Sat Oct 30 01:06:41 2010
@@ -180,12 +180,15 @@
__doc__ = _realsocket.__doc__
- __slots__ = ["_sock", "__weakref__"] + list(_delegate_methods)
+ __slots__ = ["_sock", "__weakref__", "_io_refs", "_closed"
+ ] + list(_delegate_methods)
def __init__(self, family=AF_INET, type=SOCK_STREAM, proto=0, _sock=None):
if _sock is None:
_sock = _realsocket(family, type, proto)
self._sock = _sock
+ self._io_refs = 0
+ self._closed = False
for method in _delegate_methods:
setattr(self, method, getattr(_sock, method))
@@ -216,6 +219,22 @@
and bufsize arguments are as for the built-in open() function."""
return _fileobject(self._sock, mode, bufsize)
+ def _decref_socketios(self):
+ if self._io_refs > 0:
+ self._io_refs -= 1
+ if self._closed:
+ self.close()
+
+ def _real_close(self):
+ # This function should not reference any globals. See issue #808164.
+ self._sock.close()
+
+ def close(self):
+ # This function should not reference any globals. See issue #808164.
+ self._closed = True
+ if self._io_refs <= 0:
+ self._real_close()
+
family = property(lambda self: self._sock.family, doc="the socket family")
type = property(lambda self: self._sock.type, doc="the socket type")
proto = property(lambda self: self._sock.proto, doc="the socket protocol")
Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_socket.py
==============================================================================
--- pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_socket.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_socket.py Sat Oct 30 01:06:41 2010
@@ -160,7 +160,6 @@
def clientTearDown(self):
self.done.set()
- test_support.gc_collect()
thread.exit()
class ThreadedTCPSocketTest(SocketTCPTest, ThreadableTest):
@@ -729,7 +728,6 @@
def testClose(self):
conn, addr = self.serv.accept()
conn.close()
- test_support.gc_collect() # not sure why
sd = self.cli
read, write, err = select.select([sd], [], [], 1.0)
Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py
==============================================================================
--- pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py (original)
+++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py Sat Oct 30 01:06:41 2010
@@ -281,9 +281,7 @@
tempsock = socket.socket(family, socktype)
port = bind_port(tempsock)
tempsock.close()
- # Be sure to close the socket so the port can be reused
del tempsock
- gc_collect()
return port
def bind_port(sock, host=HOST):
From hakanardo at codespeak.net Sat Oct 30 11:07:29 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 11:07:29 +0200 (CEST)
Subject: [pypy-svn] r78503 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030090729.8D581282BAD@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 11:07:28 2010
New Revision: 78503
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(armin, hakanardo) fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 11:07:28 2010
@@ -242,21 +242,9 @@
assert equaloplists(optimized.operations,
expected.operations, False, remap)
- def optimize_loop(self, ops, spectext, optops, checkspecnodes=True,
- expected_preamble=None):
+ def optimize_loop(self, ops, spectext, optops, expected_preamble=None):
loop = self.parse(ops)
- #
- if checkspecnodes:
- # verify that 'spectext' is indeed what optimizefindnode would
- # compute for this loop
- cpu = self.cpu
- perfect_specialization_finder = PerfectSpecializationFinder(cpu)
- perfect_specialization_finder.find_nodes_loop(loop)
- self.check_specnodes(loop.token.specnodes, spectext)
- else:
- # for cases where we want to see how optimizeopt behaves with
- # combinations different from the one computed by optimizefindnode
- loop.token.specnodes = self.unpack_specnodes(spectext)
+ loop.token.specnodes = self.unpack_specnodes(spectext)
#
self.loop = loop
loop.preamble = TreeLoop('preamble')
@@ -267,11 +255,16 @@
optimize_loop_1(metainterp_sd, loop)
#
expected = self.parse(optops)
+
print
- print "Ops: "
+ print loop.preamble.inputargs
+ print '\n'.join([str(o) for o in loop.preamble.operations])
+ print
+ print loop.inputargs
print '\n'.join([str(o) for o in loop.operations])
+ print
+
self.assert_equal(loop, expected)
-
if expected_preamble:
expected = self.parse(expected_preamble)
self.assert_equal(loop.preamble, expected)
@@ -414,13 +407,17 @@
guard_false(i2) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_lt(i0, 0)
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, 'Not', expected, expected_preamble=preamble)
def test_constant_boolrewrite_gt(self):
ops = """
@@ -431,13 +428,17 @@
guard_false(i2) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_gt(i0, 0)
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, 'Not', expected, expected_preamble=preamble)
def test_constant_boolrewrite_reflex(self):
ops = """
@@ -448,13 +449,17 @@
guard_true(i2) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_gt(i0, 0)
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, 'Not', expected, expected_preamble=preamble)
def test_constant_boolrewrite_reflex_invers(self):
ops = """
@@ -465,13 +470,17 @@
guard_false(i2) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_gt(i0, 0)
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, 'Not', expected, expected_preamble=preamble)
def test_remove_consecutive_guard_value_constfold(self):
ops = """
@@ -497,13 +506,13 @@
ops = """
[p1]
guard_value(p1, ConstPtr(myptr)) []
- jump(ConstPtr(myptr))
+ jump(p1)
"""
expected = """
[]
jump()
"""
- self.optimize_loop(ops, 'Constant(myptr)', expected)
+ self.optimize_loop(ops, 'Not', expected)
def test_ooisnull_oononnull_1(self):
ops = """
@@ -512,12 +521,16 @@
guard_nonnull(p0) []
jump(p0)
"""
- expected = """
+ preamble = """
[p0]
guard_class(p0, ConstClass(node_vtable)) []
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ [p0]
+ jump(p0)
+ """
+ self.optimize_loop(ops, 'Not', expected, preamble)
def test_int_is_true_1(self):
ops = """
@@ -528,13 +541,17 @@
guard_true(i2) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_is_true(i0)
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, 'Not', expected, preamble)
def test_int_is_true_is_zero(self):
py.test.skip("XXX implement me")
@@ -561,12 +578,16 @@
guard_nonnull(p0) []
jump(p0)
"""
- expected = """
+ preamble = """
[p0]
guard_nonnull(p0) []
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ [p0]
+ jump(p0)
+ """
+ self.optimize_loop(ops, 'Not', expected, preamble)
def test_ooisnull_on_null_ptr_1(self):
ops = """
@@ -594,12 +615,16 @@
guard_nonnull(p1) []
jump(p0)
"""
- expected = """
+ preamble = """
[p0]
guard_nonnull(p0) []
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ [p0]
+ jump(p0)
+ """
+ self.optimize_loop(ops, 'Not', expected, preamble)
def test_oois_1(self):
ops = """
@@ -615,12 +640,16 @@
guard_false(i1) []
jump(p0)
"""
- expected = """
+ preamble = """
[p0]
guard_class(p0, ConstClass(node_vtable)) []
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ [p0]
+ jump(p0)
+ """
+ self.optimize_loop(ops, 'Not', expected, preamble)
def test_nonnull_1(self):
ops = """
@@ -673,13 +702,17 @@
guard_value(i1, 1) [i]
jump(i)
"""
- expected = """
+ preamble = """
[i]
i1 = int_lt(i, 3)
guard_true(i1) [i]
jump(i)
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ [i]
+ jump(i)
+ """
+ self.optimize_loop(ops, 'Not', expected, preamble)
def test_guard_value_to_guard_false(self):
ops = """
@@ -688,13 +721,17 @@
guard_value(i1, 0) [i]
jump(i)
"""
- expected = """
+ preamble = """
[i]
i1 = int_is_true(i)
guard_false(i1) [i]
jump(i)
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ [i]
+ jump(i)
+ """
+ self.optimize_loop(ops, 'Not', expected, preamble)
def test_guard_value_on_nonbool(self):
ops = """
@@ -703,13 +740,17 @@
guard_value(i1, 0) [i]
jump(i)
"""
- expected = """
+ preamble = """
[i]
i1 = int_add(i, 3)
guard_value(i1, 0) [i]
- jump(-3)
+ jump()
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ []
+ jump()
+ """
+ self.optimize_loop(ops, 'Not', expected, preamble)
def test_int_is_true_of_bool(self):
ops = """
@@ -720,13 +761,17 @@
guard_value(i4, 0) [i0, i1]
jump(i0, i1)
"""
- expected = """
+ preamble = """
[i0, i1]
i2 = int_gt(i0, i1)
guard_false(i2) [i0, i1]
jump(i0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ expected = """
+ [i0, i1]
+ jump(i0, i1)
+ """
+ self.optimize_loop(ops, 'Not, Not', expected, preamble)
@@ -740,8 +785,22 @@
setfield_gc(p1, i1, descr=valuedescr)
jump(i1, p1, p2)
"""
+ preamble = """
+ [i1, p2, p3]
+ i3 = getfield_gc(p3, descr=valuedescr)
+ escape(i3)
+ jump(i1, p2)
+ """
+ expected = """
+ [i1, p2]
+ i3 = getfield_gc(p2, descr=valuedescr)
+ escape(i3)
+ p3 = new_with_vtable(ConstClass(node_vtable))
+ setfield_gc(p3, i1, descr=valuedescr)
+ jump(i1, p3)
+ """
# We cannot track virtuals that survive for more than two iterations.
- self.optimize_loop(ops, 'Not, Not, Not', ops)
+ self.optimize_loop(ops, 'Not, Not, Not', expected, preamble)
def test_p123_nested(self):
ops = """
@@ -757,7 +816,24 @@
"""
# The same as test_p123_simple, but with a virtual containing another
# virtual.
- self.optimize_loop(ops, 'Not, Not, Not', ops)
+ preamble = """
+ [i1, p2, p3]
+ i3 = getfield_gc(p3, descr=valuedescr)
+ escape(i3)
+ jump(i1, p2)
+ """
+ expected = """
+ [i1, p2]
+ i3 = getfield_gc(p2, descr=valuedescr)
+ escape(i3)
+ p4 = new_with_vtable(ConstClass(node_vtable))
+ setfield_gc(p4, i1, descr=valuedescr)
+ p1sub = new_with_vtable(ConstClass(node_vtable2))
+ setfield_gc(p1sub, i1, descr=valuedescr)
+ setfield_gc(p4, p1sub, descr=nextdescr)
+ jump(i1, p4)
+ """
+ self.optimize_loop(ops, 'Not, Not, Not', expected, preamble)
def test_p123_anti_nested(self):
ops = """
@@ -773,7 +849,23 @@
"""
# The same as test_p123_simple, but in the end the "old" p2 contains
# a "young" virtual p2sub. Make sure it is all forced.
- self.optimize_loop(ops, 'Not, Not, Not', ops)
+ preamble = """
+ [i1, p2, p3]
+ p3sub = getfield_gc(p3, descr=nextdescr)
+ i3 = getfield_gc(p3sub, descr=valuedescr)
+ escape(i3)
+ jump(i1, p2)
+ """
+ expected = """
+ [i1, p3]
+ p2sub = new_with_vtable(ConstClass(node_vtable2))
+ setfield_gc(p2sub, i1, descr=valuedescr)
+ setfield_gc(p3, p2sub, descr=nextdescr)
+ escape(i1)
+ p4 = new_with_vtable(ConstClass(node_vtable))
+ jump(i1, p4)
+ """
+ self.optimize_loop(ops, 'Not, Not, Not', expected, preamble)
# ----------
From cfbolz at codespeak.net Sat Oct 30 11:52:26 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 11:52:26 +0200 (CEST)
Subject: [pypy-svn] r78504 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030095226.0067D282BD4@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 11:52:25 2010
New Revision: 78504
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(hakanardo, cfbolz) a test to try to understand how unrolling works with delayed
setfields
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 11:52:25 2010
@@ -1932,6 +1932,34 @@
"""
self.optimize_loop(ops, 'Not, Not, Not, Not, Not', expected)
+ def test_duplicate_setfield_virtual(self):
+ # the difference with the previous test is that the field value is
+ # a virtual, which we try hard to keep virtual
+ ops = """
+ [p1, i2, i3, p4]
+ p2 = new_with_vtable(ConstClass(node_vtable))
+ setfield_gc(p2, p4, descr=nextdescr)
+ setfield_gc(p1, p2, descr=nextdescr)
+ guard_true(i3) []
+ i4 = int_neg(i2)
+ jump(p1, i2, i4, p4)
+ """
+ preamble = """
+ [p1, i2, i3, p4]
+ guard_true(i3) [p1, p4]
+ i4 = int_neg(i2)
+ jump(p1, i2, i4, p4)
+ """
+ expected = """
+ [p1, i2, i4, p4]
+ guard_true(i4) [p1, p4]
+ p2 = new_with_vtable(ConstClass(node_vtable))
+ setfield_gc(p2, p4, descr=nextdescr)
+ setfield_gc(p1, p2, descr=nextdescr)
+ jump(p1, i2, 1, p4)
+ """
+ self.optimize_loop(ops, 'Not, Not, Not, Not', expected, preamble)
+
def test_bug_1(self):
ops = """
[i0, p1]
From cfbolz at codespeak.net Sat Oct 30 11:52:38 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 11:52:38 +0200 (CEST)
Subject: [pypy-svn] r78505 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt
Message-ID: <20101030095238.2796B282BDB@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 11:52:36 2010
New Revision: 78505
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
Log:
(hakanardo, cfbolz): an XXX and a typo
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Sat Oct 30 11:52:36 2010
@@ -4,7 +4,7 @@
from pypy.jit.metainterp.resume import Snapshot
class OptUnroll(Optimization):
- """Unroll the loop into two itterations. The first one will
+ """Unroll the loop into two iterations. The first one will
become the preamble or entry bridge (don't think there is a
distinction anymore)"""
@@ -43,7 +43,7 @@
for v in self.optimizer.values.values():
v.last_guard_index = -1 # FIXME: Are there any more indexes stored?
if not v.is_constant() and v.box:
- v.fromstart = True
+ v.fromstart = True # XXX do we really need to use an attribute?
for op in self.optimizer.pure_operations.values():
v = self.getvalue(op.result)
From cfbolz at codespeak.net Sat Oct 30 12:01:27 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 12:01:27 +0200 (CEST)
Subject: [pypy-svn] r78506 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030100127.3754D282BDD@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 12:01:25 2010
New Revision: 78506
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
kill copy-pasted comment
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 12:01:25 2010
@@ -1933,8 +1933,6 @@
self.optimize_loop(ops, 'Not, Not, Not, Not, Not', expected)
def test_duplicate_setfield_virtual(self):
- # the difference with the previous test is that the field value is
- # a virtual, which we try hard to keep virtual
ops = """
[p1, i2, i3, p4]
p2 = new_with_vtable(ConstClass(node_vtable))
From arigo at codespeak.net Sat Oct 30 12:18:12 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 30 Oct 2010 12:18:12 +0200 (CEST)
Subject: [pypy-svn] r78507 - in pypy/trunk/pypy: annotation objspace/std
objspace/std/test rlib rpython/lltypesystem rpython/test
Message-ID: <20101030101812.7B7C1282BE8@codespeak.net>
Author: arigo
Date: Sat Oct 30 12:18:10 2010
New Revision: 78507
Modified:
pypy/trunk/pypy/annotation/unaryop.py
pypy/trunk/pypy/objspace/std/dictmultiobject.py
pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py
pypy/trunk/pypy/rlib/objectmodel.py
pypy/trunk/pypy/rpython/lltypesystem/rdict.py
pypy/trunk/pypy/rpython/test/test_rdict.py
Log:
Add dict.popitem() as an RPython operation, which gives it a better implementation
(see POPITEMINDEX in rdict.py)
Modified: pypy/trunk/pypy/annotation/unaryop.py
==============================================================================
--- pypy/trunk/pypy/annotation/unaryop.py (original)
+++ pypy/trunk/pypy/annotation/unaryop.py Sat Oct 30 12:18:10 2010
@@ -434,6 +434,9 @@
def method_clear(dct):
pass
+ def method_popitem(dct):
+ return dct.getanyitem('items')
+
def _can_only_throw(dic, *ignore):
if dic1.dictdef.dictkey.custom_eq_hash:
return None # r_dict: can throw anything
Modified: pypy/trunk/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/trunk/pypy/objspace/std/dictmultiobject.py Sat Oct 30 12:18:10 2010
@@ -157,6 +157,16 @@
key = OPTIMIZED_BUILTINS[i]
return self.impl_getitem_str(key)
+ def impl_popitem(self):
+ # default implementation
+ space = self.space
+ iterator = self.impl_iter()
+ w_key, w_value = iterator.next()
+ if w_key is None:
+ raise KeyError
+ self.impl_delitem(w_key)
+ return w_key, w_value
+
# _________________________________________________________________
# fallback implementation methods
@@ -196,6 +206,9 @@
key = OPTIMIZED_BUILTINS[i]
return self.impl_fallback_getitem_str(key)
+ def impl_fallback_popitem(self):
+ return self.r_dict_content.popitem()
+
implementation_methods = [
("getitem", 1),
@@ -210,6 +223,7 @@
("keys", 0),
("clear", 0),
("get_builtin_indexed", 1),
+ ("popitem", 0),
]
@@ -779,16 +793,11 @@
return w_item
def dict_popitem__DictMulti(space, w_dict):
- # XXX should somehow use the same trick as CPython: saving the index
- # of the last popped item in the hash table, so that the next call to
- # popitem() can be more efficient, instead of always starting from the
- # beginning of the hash table.
- iterator = w_dict.iter()
- w_key, w_value = iterator.next()
- if w_key is None:
+ try:
+ w_key, w_value = w_dict.popitem()
+ except KeyError:
raise OperationError(space.w_KeyError,
space.wrap("popitem(): dictionary is empty"))
- w_dict.delitem(w_key)
return space.newtuple([w_key, w_value])
app = gateway.applevel('''
Modified: pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py (original)
+++ pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py Sat Oct 30 12:18:10 2010
@@ -238,7 +238,16 @@
assert len(d) == 0
assert (it!=it1) and (it1==(1,2) or it1==(3,4))
raises(KeyError, d.popitem)
-
+
+ def test_popitem_2(self):
+ class A(object):
+ pass
+ d = A().__dict__
+ d['x'] = 5
+ it1 = d.popitem()
+ assert it1 == ('x', 5)
+ raises(KeyError, d.popitem)
+
def test_setdefault(self):
d = {1:2, 3:4}
dd = d.copy()
Modified: pypy/trunk/pypy/rlib/objectmodel.py
==============================================================================
--- pypy/trunk/pypy/rlib/objectmodel.py (original)
+++ pypy/trunk/pypy/rlib/objectmodel.py Sat Oct 30 12:18:10 2010
@@ -475,6 +475,10 @@
def setdefault(self, key, default):
return self._dict.setdefault(_r_dictkey(self, key), default)
+ def popitem(self):
+ dk, value = self._dict.popitem()
+ return dk.key, value
+
def copy(self):
result = r_dict(self.key_eq, self.key_hash)
result.update(self)
Modified: pypy/trunk/pypy/rpython/lltypesystem/rdict.py
==============================================================================
--- pypy/trunk/pypy/rpython/lltypesystem/rdict.py (original)
+++ pypy/trunk/pypy/rpython/lltypesystem/rdict.py Sat Oct 30 12:18:10 2010
@@ -306,6 +306,13 @@
hop.exception_cannot_occur()
return hop.gendirectcall(ll_clear, v_dict)
+ def rtype_method_popitem(self, hop):
+ v_dict, = hop.inputargs(self)
+ r_tuple = hop.r_result
+ cTUPLE = hop.inputconst(lltype.Void, r_tuple.lowleveltype)
+ hop.exception_is_here()
+ return hop.gendirectcall(ll_popitem, cTUPLE, v_dict)
+
class __extend__(pairtype(DictRepr, rmodel.Repr)):
def rtype_getitem((r_dict, r_key), hop):
@@ -465,6 +472,10 @@
i = ll_dict_lookup(d, key, d.keyhash(key))
if not d.entries.valid(i):
raise KeyError
+ _ll_dict_del(d, i)
+ll_dict_delitem.oopspec = 'dict.delitem(d, key)'
+
+def _ll_dict_del(d, i):
d.entries.mark_deleted(i)
d.num_items -= 1
# clear the key and the value if they are GC pointers
@@ -481,7 +492,6 @@
num_entries = len(d.entries)
if num_entries > DICT_INITSIZE and d.num_items < num_entries / 4:
ll_dict_resize(d)
-ll_dict_delitem.oopspec = 'dict.delitem(d, key)'
def ll_dict_resize(d):
old_entries = d.entries
@@ -810,3 +820,26 @@
i = ll_dict_lookup(d, key, d.keyhash(key))
return d.entries.valid(i)
ll_contains.oopspec = 'dict.contains(d, key)'
+
+POPITEMINDEX = lltype.Struct('PopItemIndex', ('nextindex', lltype.Signed))
+global_popitem_index = lltype.malloc(POPITEMINDEX, zero=True, immortal=True)
+
+def ll_popitem(ELEM, dic):
+ entries = dic.entries
+ dmask = len(entries) - 1
+ base = global_popitem_index.nextindex
+ counter = 0
+ while counter <= dmask:
+ i = (base + counter) & dmask
+ counter += 1
+ if entries.valid(i):
+ break
+ else:
+ raise KeyError
+ global_popitem_index.nextindex += counter
+ entry = entries[i]
+ r = lltype.malloc(ELEM.TO)
+ r.item0 = recast(ELEM.TO.item0, entry.key)
+ r.item1 = recast(ELEM.TO.item1, entry.value)
+ _ll_dict_del(dic, i)
+ return r
Modified: pypy/trunk/pypy/rpython/test/test_rdict.py
==============================================================================
--- pypy/trunk/pypy/rpython/test/test_rdict.py (original)
+++ pypy/trunk/pypy/rpython/test/test_rdict.py Sat Oct 30 12:18:10 2010
@@ -682,6 +682,26 @@
# if it does not crash, we are fine. It crashes if you forget the hash field.
self.interpret(func, [])
+ def test_dict_popitem(self):
+ def func():
+ d = {}
+ d[5] = 2
+ d[6] = 3
+ k1, v1 = d.popitem()
+ assert len(d) == 1
+ k2, v2 = d.popitem()
+ try:
+ d.popitem()
+ except KeyError:
+ pass
+ else:
+ assert 0, "should have raised KeyError"
+ assert len(d) == 0
+ return k1*1000 + v1*100 + k2*10 + v2
+
+ res = self.interpret(func, [])
+ assert res in [5263, 6352]
+
# ____________________________________________________________
def test_opt_nullkeymarker(self):
From arigo at codespeak.net Sat Oct 30 12:22:11 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 30 Oct 2010 12:22:11 +0200 (CEST)
Subject: [pypy-svn] r78508 - pypy/trunk/pypy/objspace/std/test
Message-ID: <20101030102211.0EF07282BEA@codespeak.net>
Author: arigo
Date: Sat Oct 30 12:22:05 2010
New Revision: 78508
Modified:
pypy/trunk/pypy/objspace/std/test/test_strutil.py
Log:
Fix the tests.
Modified: pypy/trunk/pypy/objspace/std/test/test_strutil.py
==============================================================================
--- pypy/trunk/pypy/objspace/std/test/test_strutil.py (original)
+++ pypy/trunk/pypy/objspace/std/test/test_strutil.py Sat Oct 30 12:22:05 2010
@@ -25,7 +25,7 @@
]
for s, expected in cases:
assert string_to_int(s) == expected
- assert string_to_w_long(space, s).longval() == expected
+ assert string_to_bigint(s).tolong() == expected
def test_string_to_int_base(self):
space = self.space
@@ -119,17 +119,16 @@
raises(ParseStringError, string_to_int, '+'+s, base)
raises(ParseStringError, string_to_int, '-'+s, base)
- def test_string_to_w_long(self):
- space = self.space
- assert string_to_w_long(space, '123L').longval() == 123
- assert string_to_w_long(space, '123L ').longval() == 123
- raises(ParseStringError, string_to_w_long, space, 'L')
- raises(ParseStringError, string_to_w_long, space, 'L ')
- assert string_to_w_long(space, '123L', 4).longval() == 27
- assert string_to_w_long(space, '123L', 30).longval() == 27000 + 1800 + 90 + 21
- assert string_to_w_long(space, '123L', 22).longval() == 10648 + 968 + 66 + 21
- assert string_to_w_long(space, '123L', 21).longval() == 441 + 42 + 3
- assert string_to_w_long(space, '1891234174197319').longval() == 1891234174197319
+ def test_string_to_bigint(self):
+ assert string_to_bigint('123L').tolong() == 123
+ assert string_to_bigint('123L ').tolong() == 123
+ raises(ParseStringError, string_to_bigint, 'L')
+ raises(ParseStringError, string_to_bigint, 'L ')
+ assert string_to_bigint('123L', 4).tolong() == 27
+ assert string_to_bigint('123L', 30).tolong() == 27000 + 1800 + 90 + 21
+ assert string_to_bigint('123L', 22).tolong() == 10648 + 968 + 66 + 21
+ assert string_to_bigint('123L', 21).tolong() == 441 + 42 + 3
+ assert string_to_bigint('1891234174197319').tolong() == 1891234174197319
def test_string_to_float(self):
def string_to_float(x):
From hakanardo at codespeak.net Sat Oct 30 12:30:37 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 12:30:37 +0200 (CEST)
Subject: [pypy-svn] r78509 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test
Message-ID: <20101030103037.69EF8282BEB@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 12:30:35 2010
New Revision: 78509
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(hakanardo, cfbolz) force lazy setfileds at end of preamble
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py Sat Oct 30 12:30:35 2010
@@ -148,6 +148,10 @@
self.clean_caches()
+ def force_at_end_of_preamble(self):
+ self.force_all_lazy_setfields()
+ self.next_optimization.force_at_end_of_preamble()
+
def force_lazy_setfield(self, descr, before_guard=False):
try:
op = self.lazy_setfields[descr]
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Sat Oct 30 12:30:35 2010
@@ -150,6 +150,8 @@
oohelper.CVAL_NULLREF = ConstantValue(oohelper.CONST_NULL)
class Optimization(object):
+ next_optimization = None
+
def propagate_forward(self, op):
raise NotImplementedError
@@ -191,6 +193,10 @@
def setup(self, virtuals):
pass
+ def force_at_end_of_preamble(self):
+ if self.next_optimization:
+ self.next_optimization.force_at_end_of_preamble()
+
class Optimizer(Optimization):
def __init__(self, metainterp_sd, loop, optimizations=None, virtuals=True):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Sat Oct 30 12:30:35 2010
@@ -18,8 +18,9 @@
if not self.enabled:
self.emit_operation(op)
return
-
+
if op.getopnum() == rop.JUMP:
+ self.force_at_end_of_preamble()
loop = self.optimizer.loop
loop.preamble.operations = self.optimizer.newoperations
self.optimizer.newoperations = []
@@ -93,10 +94,13 @@
if not isinstance(a, Const) and a in self.optimizer.values:
v = self.getvalue(a)
if v.fromstart and not v.is_constant():
- a = v.force_box()
- if a not in inputargs:
- inputargs.append(a)
- newval = self.getvalue(argmap[a])
+ b = v.force_box()
+ if b not in inputargs:
+ #boxes = []
+ #v.enum_forced_boxes(boxes, seen_inputargs)
+ #for b in boxes:
+ inputargs.append(b)
+ newval = self.getvalue(argmap[b])
jumpargs.append(newval.force_box())
jmp.initarglist(jumpargs)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 12:30:35 2010
@@ -854,16 +854,20 @@
p3sub = getfield_gc(p3, descr=nextdescr)
i3 = getfield_gc(p3sub, descr=valuedescr)
escape(i3)
- jump(i1, p2)
- """
- expected = """
- [i1, p3]
p2sub = new_with_vtable(ConstClass(node_vtable2))
setfield_gc(p2sub, i1, descr=valuedescr)
- setfield_gc(p3, p2sub, descr=nextdescr)
- escape(i1)
- p4 = new_with_vtable(ConstClass(node_vtable))
- jump(i1, p4)
+ setfield_gc(p2, p2sub, descr=nextdescr)
+ jump(i1, p2, p2sub)
+ """
+ expected = """
+ [i1, p2, p2sub]
+ i3 = getfield_gc(p2sub, descr=valuedescr)
+ escape(i3)
+ p1 = new_with_vtable(ConstClass(node_vtable))
+ p3sub = new_with_vtable(ConstClass(node_vtable2))
+ setfield_gc(p3sub, i1, descr=valuedescr)
+ setfield_gc(p1, p3sub, descr=nextdescr)
+ jump(i1, p1, p3sub)
"""
self.optimize_loop(ops, 'Not, Not, Not', expected, preamble)
From arigo at codespeak.net Sat Oct 30 12:33:20 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 30 Oct 2010 12:33:20 +0200 (CEST)
Subject: [pypy-svn] r78510 - pypy/branch/jit-unroll-loops/pypy/jit/metainterp
Message-ID: <20101030103320.A5020282BEC@codespeak.net>
Author: arigo
Date: Sat Oct 30 12:33:19 2010
New Revision: 78510
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py
Log:
Fix the message
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py Sat Oct 30 12:33:19 2010
@@ -1680,7 +1680,7 @@
# raises in case it works -- which is the common case
self.compile(original_boxes, live_arg_boxes, start)
# creation of the loop was cancelled!
- self.staticdata.log('cancelled, going on...')
+ self.staticdata.log('cancelled, tracing more...')
# Otherwise, no loop found so far, so continue tracing.
start = len(self.history.operations)
From cfbolz at codespeak.net Sat Oct 30 13:01:07 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 13:01:07 +0200 (CEST)
Subject: [pypy-svn] r78511 - pypy/trunk/pypy/jit/metainterp/test
Message-ID: <20101030110107.447D2282BD4@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 13:01:05 2010
New Revision: 78511
Modified:
pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
make this test test a bit more
Modified: pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 13:01:05 2010
@@ -807,10 +807,10 @@
guard_value(i2, 1) []
i3 = call_loopinvariant(1, i1, descr=nonwritedescr)
guard_no_exception() []
- guard_value(i2, 1) []
+ guard_value(i3, 1) []
i4 = call_loopinvariant(1, i1, descr=nonwritedescr)
guard_no_exception() []
- guard_value(i2, 1) []
+ guard_value(i4, 1) []
jump(i1)
"""
expected = """
From cfbolz at codespeak.net Sat Oct 30 13:15:25 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 13:15:25 +0200 (CEST)
Subject: [pypy-svn] r78512 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt
Message-ID: <20101030111525.A7683282BD4@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 13:15:24 2010
New Revision: 78512
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
Log:
- only use unroll optimization if we are not in a bridge
- rename the confusing flag "virtuals" to "not_a_bridge"
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py Sat Oct 30 13:15:24 2010
@@ -6,20 +6,21 @@
from pypy.jit.metainterp.optimizeopt.string import OptString
from pypy.jit.metainterp.optimizeopt.unroll import OptUnroll
-def optimize_loop_1(metainterp_sd, loop, virtuals=True):
+def optimize_loop_1(metainterp_sd, loop, not_a_bridge=True):
"""Optimize loop.operations to make it match the input of loop.specnodes
and to remove internal overheadish operations. Note that loop.specnodes
must be applicable to the loop; you will probably get an AssertionError
if not.
"""
- optimizations = [OptUnroll(),
- OptIntBounds(),
+ optimizations = [OptIntBounds(),
OptRewrite(),
OptVirtualize(),
OptString(),
OptHeap(),
]
- optimizer = Optimizer(metainterp_sd, loop, optimizations, virtuals)
+ if not_a_bridge:
+ optimizations.insert(0, OptUnroll())
+ optimizer = Optimizer(metainterp_sd, loop, optimizations, not_a_bridge)
optimizer.propagate_all_forward()
def optimize_bridge_1(metainterp_sd, bridge):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Sat Oct 30 13:15:24 2010
@@ -190,7 +190,7 @@
op = ResOperation(opnum, args, result)
self.optimizer.pure_operations[self.optimizer.make_args_key(op)] = op
- def setup(self, virtuals):
+ def setup(self, not_a_bridge):
pass
def force_at_end_of_preamble(self):
@@ -199,7 +199,7 @@
class Optimizer(Optimization):
- def __init__(self, metainterp_sd, loop, optimizations=None, virtuals=True):
+ def __init__(self, metainterp_sd, loop, optimizations=None, not_a_bridge=True):
self.metainterp_sd = metainterp_sd
self.cpu = metainterp_sd.cpu
self.loop = loop
@@ -220,7 +220,7 @@
optimizations[-1].next_optimization = self
for o in optimizations:
o.optimizer = self
- o.setup(virtuals)
+ o.setup(not_a_bridge)
else:
self.first_optimization = self
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Sat Oct 30 13:15:24 2010
@@ -278,8 +278,8 @@
class OptVirtualize(Optimization):
"Virtualize objects until they escape."
- def setup(self, virtuals):
- if not virtuals:
+ def setup(self, not_a_bridge):
+ if not not_a_bridge:
return
inputargs = self.optimizer.loop.inputargs
From cfbolz at codespeak.net Sat Oct 30 13:18:32 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 13:18:32 +0200 (CEST)
Subject: [pypy-svn] r78513 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt
Message-ID: <20101030111832.711A236E0C1@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 13:18:30 2010
New Revision: 78513
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
Log:
this is no longer necessary now
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Sat Oct 30 13:18:30 2010
@@ -8,16 +8,13 @@
become the preamble or entry bridge (don't think there is a
distinction anymore)"""
- def setup(self, virtuals):
- self.enabled = virtuals
+ def setup(self, not_a_bridge):
+ assert not_a_bridge
self.cloned_operations = []
for op in self.optimizer.loop.operations:
self.cloned_operations.append(op.clone())
def propagate_forward(self, op):
- if not self.enabled:
- self.emit_operation(op)
- return
if op.getopnum() == rop.JUMP:
self.force_at_end_of_preamble()
From arigo at codespeak.net Sat Oct 30 13:20:40 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 30 Oct 2010 13:20:40 +0200 (CEST)
Subject: [pypy-svn] r78514 - pypy/trunk/lib-python/modified-2.5.2/test
Message-ID: <20101030112040.A395D282B90@codespeak.net>
Author: arigo
Date: Sat Oct 30 13:20:39 2010
New Revision: 78514
Added:
pypy/trunk/lib-python/modified-2.5.2/test/test_dict.py
- copied, changed from r78508, pypy/trunk/lib-python/2.5.2/test/test_dict.py
Log:
Implementation detail.
Copied: pypy/trunk/lib-python/modified-2.5.2/test/test_dict.py (from r78508, pypy/trunk/lib-python/2.5.2/test/test_dict.py)
==============================================================================
--- pypy/trunk/lib-python/2.5.2/test/test_dict.py (original)
+++ pypy/trunk/lib-python/modified-2.5.2/test/test_dict.py Sat Oct 30 13:20:39 2010
@@ -300,7 +300,8 @@
self.assertEqual(va, int(ka))
kb, vb = tb = b.popitem()
self.assertEqual(vb, int(kb))
- self.assert_(not(copymode < 0 and ta != tb))
+ if test_support.check_impl_detail():
+ self.assert_(not(copymode < 0 and ta != tb))
self.assert_(not a)
self.assert_(not b)
From fijal at codespeak.net Sat Oct 30 13:31:16 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Sat, 30 Oct 2010 13:31:16 +0200 (CEST)
Subject: [pypy-svn] r78515 - pypy/trunk/pypy/objspace
Message-ID: <20101030113116.E89EB282BAD@codespeak.net>
Author: fijal
Date: Sat Oct 30 13:31:15 2010
New Revision: 78515
Modified:
pypy/trunk/pypy/objspace/descroperation.py
Log:
Make everything newstyle in descroperation
Modified: pypy/trunk/pypy/objspace/descroperation.py
==============================================================================
--- pypy/trunk/pypy/objspace/descroperation.py (original)
+++ pypy/trunk/pypy/objspace/descroperation.py Sat Oct 30 13:31:15 2010
@@ -51,7 +51,7 @@
return space.is_w(w_typ1, w_typ2)
-class Object:
+class Object(object):
def descr__getattribute__(space, w_obj, w_name):
name = space.str_w(w_name)
w_descr = space.lookup(w_obj, name)
@@ -96,7 +96,7 @@
def descr__init__(space, w_obj, __args__):
pass
-class DescrOperation:
+class DescrOperation(object):
_mixin_ = True
def is_data_descr(space, w_obj):
@@ -541,7 +541,7 @@
# what is the maximum value slices can get on CPython?
# we need to stick to that value, because fake.py etc.
-class Temp:
+class Temp(object):
def __getslice__(self, i, j):
return j
slice_max = Temp()[:]
From fijal at codespeak.net Sat Oct 30 13:32:51 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Sat, 30 Oct 2010 13:32:51 +0200 (CEST)
Subject: [pypy-svn] r78516 - pypy/trunk/pypy/interpreter
Message-ID: <20101030113251.62367282BD4@codespeak.net>
Author: fijal
Date: Sat Oct 30 13:32:49 2010
New Revision: 78516
Modified:
pypy/trunk/pypy/interpreter/gateway.py
Log:
MAke those newstyle as well
Modified: pypy/trunk/pypy/interpreter/gateway.py
==============================================================================
--- pypy/trunk/pypy/interpreter/gateway.py (original)
+++ pypy/trunk/pypy/interpreter/gateway.py Sat Oct 30 13:32:49 2010
@@ -28,7 +28,7 @@
# internal non-translatable parts:
import py
-class SignatureBuilder:
+class SignatureBuilder(object):
"NOT_RPYTHON"
def __init__(self, func=None, argnames=None, varargname=None,
kwargname=None, name = None):
@@ -51,7 +51,7 @@
#________________________________________________________________
-class UnwrapSpecRecipe:
+class UnwrapSpecRecipe(object):
"NOT_RPYTHON"
bases_order = [Wrappable, W_Root, ObjSpace, Arguments, object]
From fijal at codespeak.net Sat Oct 30 13:35:43 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Sat, 30 Oct 2010 13:35:43 +0200 (CEST)
Subject: [pypy-svn] r78517 - pypy/trunk/pypy/annotation
Message-ID: <20101030113543.5F70F36E0CB@codespeak.net>
Author: fijal
Date: Sat Oct 30 13:35:41 2010
New Revision: 78517
Modified:
pypy/trunk/pypy/annotation/bookkeeper.py
pypy/trunk/pypy/annotation/classdef.py
pypy/trunk/pypy/annotation/description.py
pypy/trunk/pypy/annotation/dictdef.py
pypy/trunk/pypy/annotation/listdef.py
pypy/trunk/pypy/annotation/specialize.py
Log:
sprinkle (object) around annotation as well
Modified: pypy/trunk/pypy/annotation/bookkeeper.py
==============================================================================
--- pypy/trunk/pypy/annotation/bookkeeper.py (original)
+++ pypy/trunk/pypy/annotation/bookkeeper.py Sat Oct 30 13:35:41 2010
@@ -24,7 +24,7 @@
from pypy.rpython import extregistry
from pypy.tool.identity_dict import identity_dict
-class Stats:
+class Stats(object):
def __init__(self, bookkeeper):
self.bookkeeper = bookkeeper
@@ -137,7 +137,7 @@
def consider_dict_delitem(self, dic):
return dic
-class Bookkeeper:
+class Bookkeeper(object):
"""The log of choices that have been made while analysing the operations.
It ensures that the same 'choice objects' will be returned if we ask
again during reflowing. Like ExecutionContext, there is an implicit
@@ -736,7 +736,7 @@
return True
# for parsing call arguments
-class RPythonCallsSpace:
+class RPythonCallsSpace(object):
"""Pseudo Object Space providing almost no real operation.
For the Arguments class: if it really needs other operations, it means
that the call pattern is too complex for R-Python.
Modified: pypy/trunk/pypy/annotation/classdef.py
==============================================================================
--- pypy/trunk/pypy/annotation/classdef.py (original)
+++ pypy/trunk/pypy/annotation/classdef.py Sat Oct 30 13:35:41 2010
@@ -58,7 +58,7 @@
# same name in all subclasses of A, if any. (Parent class attributes can
# be visible in reads from instances of subclasses.)
-class Attribute:
+class Attribute(object):
# readonly-ness
# SomeThing-ness
# NB. an attribute is readonly if it is a constant class attribute.
@@ -402,7 +402,7 @@
# ____________________________________________________________
-class InstanceSource:
+class InstanceSource(object):
instance_level = True
def __init__(self, bookkeeper, obj):
Modified: pypy/trunk/pypy/annotation/description.py
==============================================================================
--- pypy/trunk/pypy/annotation/description.py (original)
+++ pypy/trunk/pypy/annotation/description.py Sat Oct 30 13:35:41 2010
@@ -6,7 +6,7 @@
from pypy.tool.sourcetools import valid_identifier
from pypy.tool.pairtype import extendabletype
-class CallFamily:
+class CallFamily(object):
"""A family of Desc objects that could be called from common call sites.
The call families are conceptually a partition of all (callable) Desc
objects, where the equivalence relation is the transitive closure of
@@ -51,7 +51,7 @@
self.total_calltable_size += 1
-class FrozenAttrFamily:
+class FrozenAttrFamily(object):
"""A family of FrozenDesc objects that have any common 'getattr' sites.
The attr families are conceptually a partition of FrozenDesc objects,
where the equivalence relation is the transitive closure of:
@@ -80,7 +80,7 @@
self.attrs[attrname] = s_value
-class ClassAttrFamily:
+class ClassAttrFamily(object):
"""A family of ClassDesc objects that have common 'getattr' sites for a
given attribute name. The attr families are conceptually a partition
of ClassDesc objects, where the equivalence relation is the transitive
Modified: pypy/trunk/pypy/annotation/dictdef.py
==============================================================================
--- pypy/trunk/pypy/annotation/dictdef.py (original)
+++ pypy/trunk/pypy/annotation/dictdef.py Sat Oct 30 13:35:41 2010
@@ -77,7 +77,7 @@
dictdef.dictvalue = self
-class DictDef:
+class DictDef(object):
"""A dict definition remembers how general the keys and values in that
particular dict have to be. Every dict creation makes a new DictDef,
and the union of two dicts merges the DictKeys and DictValues that each
Modified: pypy/trunk/pypy/annotation/listdef.py
==============================================================================
--- pypy/trunk/pypy/annotation/listdef.py (original)
+++ pypy/trunk/pypy/annotation/listdef.py Sat Oct 30 13:35:41 2010
@@ -6,7 +6,7 @@
class TooLateForChange(Exception):
pass
-class ListItem:
+class ListItem(object):
mutated = False # True for lists mutated after creation
resized = False # True for lists resized after creation
range_step = None # the step -- only for lists only created by a range()
@@ -117,7 +117,7 @@
return updated
-class ListDef:
+class ListDef(object):
"""A list definition remembers how general the items in that particular
list have to be. Every list creation makes a new ListDef, and the union
of two lists merges the ListItems that each ListDef stores."""
Modified: pypy/trunk/pypy/annotation/specialize.py
==============================================================================
--- pypy/trunk/pypy/annotation/specialize.py (original)
+++ pypy/trunk/pypy/annotation/specialize.py Sat Oct 30 13:35:41 2010
@@ -100,7 +100,7 @@
# ____________________________________________________________________________
# specializations
-class MemoTable:
+class MemoTable(object):
def __init__(self, funcdesc, args, value):
self.funcdesc = funcdesc
self.table = {args: value}
From fijal at codespeak.net Sat Oct 30 13:36:00 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Sat, 30 Oct 2010 13:36:00 +0200 (CEST)
Subject: [pypy-svn] r78518 - pypy/branch/faster-compilation
Message-ID: <20101030113600.708BC282BDB@codespeak.net>
Author: fijal
Date: Sat Oct 30 13:35:59 2010
New Revision: 78518
Removed:
pypy/branch/faster-compilation/
Log:
Everything worth noting has been re-applied to trunk
From cfbolz at codespeak.net Sat Oct 30 13:43:27 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 13:43:27 +0200 (CEST)
Subject: [pypy-svn] r78519 -
pypy/branch/mapdict-without-jit/pypy/objspace/std
Message-ID: <20101030114327.63AE6282BEB@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 13:43:26 2010
New Revision: 78519
Modified:
pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py
Log:
cannot use jit.purefunction here, because the argument is a tuple. hack
differently.
Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py (original)
+++ pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py Sat Oct 30 13:43:26 2010
@@ -41,9 +41,9 @@
def delete(self, obj, selector):
return None
- @jit.purefunction
def index(self, selector):
- if self.space.config.objspace.std.withmethodcache:
+ if (self.space.config.objspace.std.withmethodcache and
+ not jit.we_are_jitted()):
return self._index_cache(selector)
else:
return self._index(selector)
From cfbolz at codespeak.net Sat Oct 30 13:45:09 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 13:45:09 +0200 (CEST)
Subject: [pypy-svn] r78520 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt
Message-ID: <20101030114509.36BFE282BEF@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 13:45:07 2010
New Revision: 78520
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
Log:
import * is evil
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Sat Oct 30 13:45:07 2010
@@ -1,18 +1,20 @@
+from pypy.jit.metainterp.history import Const, ConstInt
from pypy.jit.metainterp.specnode import SpecNode, NotSpecNode, ConstantSpecNode
from pypy.jit.metainterp.specnode import AbstractVirtualStructSpecNode
from pypy.jit.metainterp.specnode import VirtualInstanceSpecNode
from pypy.jit.metainterp.specnode import VirtualArraySpecNode
from pypy.jit.metainterp.specnode import VirtualStructSpecNode
from pypy.jit.metainterp.resoperation import rop, ResOperation
-from pypy.jit.metainterp.optimizeutil import _findall
+from pypy.jit.metainterp.optimizeutil import _findall, sort_descrs
+from pypy.jit.metainterp.optimizeutil import descrlist_dict
from pypy.rlib.objectmodel import we_are_translated
-from pypy.jit.metainterp.optimizeopt.optimizer import *
+from pypy.jit.metainterp.optimizeopt import optimizer
-class AbstractVirtualValue(OptValue):
+class AbstractVirtualValue(optimizer.OptValue):
_attrs_ = ('optimizer', 'keybox', 'source_op', '_cached_vinfo')
box = None
- level = LEVEL_NONNULL
+ level = optimizer.LEVEL_NONNULL
_cached_vinfo = None
def __init__(self, optimizer, keybox, source_op=None):
@@ -67,7 +69,7 @@
return self._fields.get(ofs, default)
def setfield(self, ofs, fieldvalue):
- assert isinstance(fieldvalue, OptValue)
+ assert isinstance(fieldvalue, optimizer.OptValue)
self._fields[ofs] = fieldvalue
def _really_force(self):
@@ -135,7 +137,7 @@
class VirtualValue(AbstractVirtualStructValue):
- level = LEVEL_KNOWNCLASS
+ level = optimizer.LEVEL_KNOWNCLASS
def __init__(self, optimizer, known_class, keybox, source_op=None):
AbstractVirtualStructValue.__init__(self, optimizer, keybox, source_op)
@@ -172,7 +174,7 @@
return res
def setitem(self, index, itemvalue):
- assert isinstance(itemvalue, OptValue)
+ assert isinstance(itemvalue, optimizer.OptValue)
self._items[index] = itemvalue
def _really_force(self):
@@ -275,7 +277,7 @@
subspecnode = self.items[index]
subspecnode.teardown_virtual_node(optimizer, subvalue, newexitargs)
-class OptVirtualize(Optimization):
+class OptVirtualize(optimizer.Optimization):
"Virtualize objects until they escape."
def setup(self, not_a_bridge):
From arigo at codespeak.net Sat Oct 30 13:48:46 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 30 Oct 2010 13:48:46 +0200 (CEST)
Subject: [pypy-svn] r78521 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030114846.279F7282BF0@codespeak.net>
Author: arigo
Date: Sat Oct 30 13:48:44 2010
New Revision: 78521
Added:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
- copied, changed from r77505, pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
A copy of test_optimizeopt from trunk, which makes sense
to keep running without the OptUnroll optimization.
Copied: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py (from r77505, pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py)
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py Sat Oct 30 13:48:44 2010
@@ -6,7 +6,6 @@
from pypy.jit.metainterp.optimizefindnode import PerfectSpecializationFinder
import pypy.jit.metainterp.optimizeopt.optimizer as optimizeopt
import pypy.jit.metainterp.optimizeopt.virtualize as virtualize
-from pypy.jit.metainterp.optimizeopt import optimize_loop_1
from pypy.jit.metainterp.optimizeutil import InvalidLoop
from pypy.jit.metainterp.history import AbstractDescr, ConstInt, BoxInt
from pypy.jit.metainterp.jitprof import EmptyProfiler
@@ -219,7 +218,7 @@
_kind2count = {history.INT: 1, history.REF: 2, history.FLOAT: 3}
return sorted(boxes, key=lambda box: _kind2count[box.type])
-class BaseTestOptimizeOpt(BaseTest):
+class BaseTestOptimizeBasic(BaseTest):
def invent_fail_descr(self, fail_args):
if fail_args is None:
@@ -240,24 +239,29 @@
def optimize_loop(self, ops, spectext, optops, checkspecnodes=True):
loop = self.parse(ops)
- #
- if checkspecnodes:
- # verify that 'spectext' is indeed what optimizefindnode would
- # compute for this loop
- cpu = self.cpu
- perfect_specialization_finder = PerfectSpecializationFinder(cpu)
- perfect_specialization_finder.find_nodes_loop(loop)
- self.check_specnodes(loop.token.specnodes, spectext)
- else:
- # for cases where we want to see how optimizeopt behaves with
- # combinations different from the one computed by optimizefindnode
- loop.token.specnodes = self.unpack_specnodes(spectext)
+ loop.token.specnodes = self.unpack_specnodes(spectext)
#
self.loop = loop
metainterp_sd = FakeMetaInterpStaticData(self.cpu)
if hasattr(self, 'vrefinfo'):
metainterp_sd.virtualref_info = self.vrefinfo
- optimize_loop_1(metainterp_sd, loop)
+ #
+ # XXX list the exact optimizations that are needed for each test
+ from pypy.jit.metainterp.optimizeopt import (OptIntBounds,
+ OptRewrite,
+ OptVirtualize,
+ OptString,
+ OptHeap,
+ Optimizer)
+ optimizations = [OptIntBounds(),
+ OptRewrite(),
+ OptVirtualize(),
+ OptString(),
+ OptHeap(),
+ ]
+ optimizer = Optimizer(metainterp_sd, loop, optimizations,
+ virtuals=True)
+ optimizer.propagate_all_forward()
#
expected = self.parse(optops)
print '\n'.join([str(o) for o in loop.operations])
@@ -490,6 +494,7 @@
[]
jump()
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, 'Constant(myptr)', expected)
def test_ooisnull_oononnull_1(self):
@@ -752,10 +757,10 @@
p3sub = getfield_gc(p3, descr=nextdescr)
i3 = getfield_gc(p3sub, descr=valuedescr)
escape(i3)
- p1 = new_with_vtable(ConstClass(node_vtable))
p2sub = new_with_vtable(ConstClass(node_vtable2))
setfield_gc(p2sub, i1, descr=valuedescr)
setfield_gc(p2, p2sub, descr=nextdescr)
+ p1 = new_with_vtable(ConstClass(node_vtable))
jump(i1, p1, p2)
"""
# The same as test_p123_simple, but in the end the "old" p2 contains
@@ -827,8 +832,9 @@
i1 = int_add(i2, i)
jump(i, i1)
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, 'Not, Virtual(node_vtable, valuedescr=Not)',
- expected, checkspecnodes=False)
+ expected)
def test_virtual_float(self):
ops = """
@@ -843,6 +849,7 @@
f1 = float_add(f2, f)
jump(f, f1)
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, 'Not, Virtual(node_vtable, floatdescr=Not)',
expected, checkspecnodes=False)
@@ -860,6 +867,7 @@
i1 = int_add(i2, i)
jump(i, i1)
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, 'Not, Virtual(node_vtable, valuedescr=Not)',
expected)
@@ -894,6 +902,7 @@
# all constant-folded :-)
jump(p2)
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, '''Virtual(node_vtable),
Virtual(node_vtable),
Not''',
@@ -931,6 +940,7 @@
"""
# the 'expected' is sub-optimal, but it should be done by another later
# optimization step. See test_find_nodes_default_field() for why.
+ py.test.skip("XXX")
self.optimize_loop(ops, 'Virtual(node_vtable, valuedescr=Not)',
expected)
@@ -967,6 +977,7 @@
i3 = int_add(i0, i1)
jump(i3, i2)
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, 'Not, Virtual(node_vtable, valuedescr=Not)',
expected)
@@ -990,6 +1001,7 @@
i3 = int_add(i0, i1)
jump(i3, i2, i1)
"""
+ py.test.skip("XXX")
self.optimize_loop(ops,
'''Not, Virtual(node_vtable,
valuedescr=Not,
@@ -1241,6 +1253,7 @@
guard_value(i3, 15) []
jump(i0, 20, i0)
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, 'Not, VArray(arraydescr, Not, Not)', expected)
def test_p123_array(self):
@@ -1291,6 +1304,7 @@
escape(i2)
jump(i1, i1)
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, 'Not, VStruct(ssize, adescr=Not)', expected)
def test_p123_vstruct(self):
@@ -1472,6 +1486,7 @@
escape(i1)
jump()
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, 'Constant(myptr)', expected)
def test_duplicate_getfield_sideeffects_1(self):
@@ -1693,6 +1708,7 @@
setfield_gc(ConstPtr(myptr), i2, descr=valuedescr)
jump(i1, i2)
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, 'Constant(myptr), Not, Not', expected)
def test_duplicate_getarrayitem_1(self):
@@ -1847,6 +1863,7 @@
p3 = escape()
jump(i0, p3)
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, 'Not, Virtual(node_vtable, nextdescr=Not)',
expected)
@@ -1870,6 +1887,7 @@
p3 = escape()
jump(i0, p3)
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, 'Not, VArray(arraydescr2, Not)',
expected)
@@ -1903,6 +1921,7 @@
p2a = new_with_vtable(ConstClass(node_vtable))
jump(p2a, p3a)
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, 'Virtual(node_vtable2, nextdescr=Not, otherdescr=Not)', expected)
def test_bug_3bis(self):
@@ -1935,6 +1954,7 @@
escape(p3a)
jump(p2a, p3a)
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, 'Virtual(node_vtable2, nextdescr=Not, otherdescr=Not)', expected)
def test_invalid_loop_1(self):
@@ -1945,6 +1965,7 @@
p2 = new_with_vtable(ConstClass(node_vtable))
jump(p2)
"""
+ py.test.skip("XXX")
py.test.raises(InvalidLoop, self.optimize_loop,
ops, 'Virtual(node_vtable)', None)
@@ -1972,6 +1993,7 @@
setfield_gc(p3, p4, descr=nextdescr)
jump(p3)
"""
+ py.test.skip("XXX")
py.test.raises(InvalidLoop, self.optimize_loop, ops,
'Virtual(node_vtable, nextdescr=Virtual(node_vtable))',
None)
@@ -2501,6 +2523,7 @@
guard_true(i0, descr=fdescr) [i1b]
jump(i1, i1, i1)
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, '''Virtual(node_vtable, valuedescr=Not),
Not, Not''', expected)
self.check_expanded_fail_descr('''p0
@@ -2575,6 +2598,7 @@
guard_true(i1, descr=fdescr) [ia, iv, i2]
jump(1, 1, i2, NULL, i2)
"""
+ py.test.skip("XXX")
self.optimize_loop(ops, '''
Not,
VArray(arraydescr2,
@@ -2643,7 +2667,7 @@
''', rop.GUARD_TRUE)
-class TestLLtype(BaseTestOptimizeOpt, LLtypeMixin):
+class TestLLtype(BaseTestOptimizeBasic, LLtypeMixin):
def test_residual_call_does_not_invalidate_caches(self):
ops = """
@@ -4495,7 +4519,7 @@
# XXX str2unicode
-##class TestOOtype(BaseTestOptimizeOpt, OOtypeMixin):
+##class TestOOtype(BaseTestOptimizeBasic, OOtypeMixin):
## def test_instanceof(self):
## ops = """
From arigo at codespeak.net Sat Oct 30 13:54:45 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 30 Oct 2010 13:54:45 +0200 (CEST)
Subject: [pypy-svn] r78522 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt
Message-ID: <20101030115445.9C7A8282BAD@codespeak.net>
Author: arigo
Date: Sat Oct 30 13:54:44 2010
New Revision: 78522
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py
Log:
For now, kill {get,set}field_raw support. If we want it we need to be more careful.
For example, a signal handler could have changed the field -- so we would need some
'volatile' declarations like C.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py Sat Oct 30 13:54:44 2010
@@ -107,7 +107,6 @@
return
opnum = op.getopnum()
if (opnum == rop.SETFIELD_GC or
- opnum == rop.SETFIELD_RAW or
opnum == rop.SETARRAYITEM_GC or
opnum == rop.DEBUG_MERGE_POINT):
return
@@ -241,10 +240,6 @@
# remember the result of future reads of the field
self.cache_field_value(op.getdescr(), value, fieldvalue, write=True)
- optimize_GETFIELD_RAW = optimize_GETFIELD_GC
- optimize_SETFIELD_RAW = optimize_SETFIELD_GC
- # FIXME: Do we need separate caches for raw and gc?
-
def optimize_GETARRAYITEM_GC(self, op):
value = self.getvalue(op.getarg(0))
indexvalue = self.getvalue(op.getarg(1))
From david at codespeak.net Sat Oct 30 13:54:56 2010
From: david at codespeak.net (david at codespeak.net)
Date: Sat, 30 Oct 2010 13:54:56 +0200 (CEST)
Subject: [pypy-svn] r78523 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101030115456.6D6CD282BF0@codespeak.net>
Author: david
Date: Sat Oct 30 13:54:54 2010
New Revision: 78523
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py
Log:
Implement int_and, int_or, int_xor operations
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Sat Oct 30 13:54:54 2010
@@ -169,7 +169,7 @@
# cpu interface
def assemble_loop(self, inputargs, operations, looptoken):
longevity = compute_vars_longevity(inputargs, operations)
- regalloc = ARMRegisterManager(longevity, assembler=self.mc, frame_manager=ARMFrameManager())
+ regalloc = ARMRegisterManager(longevity, assembler=self, frame_manager=ARMFrameManager())
self.align()
loop_start=self.mc.curraddr()
self.gen_func_prolog()
@@ -190,7 +190,7 @@
def assemble_bridge(self, faildescr, inputargs, operations):
enc = rffi.cast(rffi.CCHARP, faildescr._failure_recovery_code)
longevity = compute_vars_longevity(inputargs, operations)
- regalloc = ARMRegisterManager(longevity, assembler=self.mc, frame_manager=ARMFrameManager())
+ regalloc = ARMRegisterManager(longevity, assembler=self, frame_manager=ARMFrameManager())
regalloc.update_bindings(enc, inputargs)
bridge_head = self.mc.curraddr()
@@ -218,6 +218,13 @@
b.gen_load_int(reg.value, bridge_addr, fcond)
b.MOV_rr(r.pc.value, reg.value, cond=fcond)
+ # regalloc support
+ def regalloc_mov(self, prev_loc, loc):
+ if isinstance(prev_loc, ConstInt):
+ # XXX check size of imm for current instr
+ self.mc.gen_load_int(loc.value, prev_loc.getint())
+ else:
+ self.mc.MOV_rr(loc.value, prev_loc.value)
def make_operation_list():
def notimplemented(self, op, regalloc, fcond):
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Sat Oct 30 13:54:54 2010
@@ -104,13 +104,6 @@
self.MOV_ri(ip, t, cond=cond)
self.ORR_rr(r, r, ip, offset, cond=cond)
- # regalloc support
- def regalloc_mov(self, prev_loc, loc):
- if isinstance(prev_loc, ConstInt):
- # XXX check size of imm for current instr
- self.gen_load_int(loc.value, prev_loc.getint())
- else:
- self.MOV_rr(loc.value, prev_loc.value)
class ARMv7InMemoryBuilder(AbstractARMv7Builder):
def __init__(self, start, end):
@@ -121,7 +114,7 @@
class ARMv7Builder(AbstractARMv7Builder):
def __init__(self):
- map_size = 1024
+ map_size = 4096
data = alloc(map_size)
self._pos = 0
self._init(data, map_size)
@@ -140,13 +133,13 @@
def _add_more_mem(self):
new_mem = alloc(self._size)
new_mem_addr = rffi.cast(lltype.Signed, new_mem)
- self.PUSH([reg.r0.value])
+ self.PUSH([reg.r0.value, reg.ip.value])
self.gen_load_int(reg.r0.value, new_mem_addr)
self.MOV_rr(reg.pc.value, reg.r0.value)
self._dump_trace('data%d.asm' % self.n_data)
self.n_data+=1
self._data = new_mem
self._pos = 0
- self.LDM(reg.sp.value, [reg.r0.value], w=1) # XXX Replace with POP instr. someday
+ self.LDM(reg.sp.value, [reg.r0.value, reg.ip.value], w=1) # XXX Replace with POP instr. someday
define_instructions(AbstractARMv7Builder)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py Sat Oct 30 13:54:54 2010
@@ -34,7 +34,7 @@
}
data_proc_imm = {
- 'ADD_ri': {'op': 0, 'rncond':'', 'result':True, 'base':True},
+ 'AND_ri': {'op': 0, 'rncond':'', 'result':True, 'base':True},
'EOR_ri': {'op': 0x2, 'rncond':'', 'result':True, 'base':True},
'SUB_ri': {'op': 0x4, 'rncond':'!0xF', 'result':True, 'base':True},
#'ADR_ri': {'op': 0x4, 'rncond':'0xF', 'result':True, 'base':True},
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Sat Oct 30 13:54:54 2010
@@ -12,6 +12,35 @@
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.lltypesystem import lltype, rffi, llmemory
+def gen_emit_op_ri(opname):
+ def f(self, op, regalloc, fcond):
+ ri_op = getattr(self.mc, '%s_ri' % opname)
+ rr_op = getattr(self.mc, '%s_rr' % opname)
+
+ arg0 = op.getarg(0)
+ arg1 = op.getarg(1)
+ res = regalloc.try_allocate_reg(op.result)
+ if self._check_imm_arg(arg0, 0xFF) and not isinstance(arg1, ConstInt):
+ print 'arg0 is imm'
+ reg = regalloc.try_allocate_reg(arg1)
+ ri_op(res.value, reg.value, imm=arg0.getint(), cond=fcond)
+ elif self._check_imm_arg(arg1, 0xFF) and not isinstance(arg0, ConstInt):
+ print 'arg1 is imm'
+ box = Box()
+ reg = regalloc.try_allocate_reg(arg0)
+ ri_op(res.value, reg.value, imm=arg1.getint(), cond=fcond)
+ else:
+ print 'generating rr'
+ reg = self._put_in_reg(arg0, regalloc)
+ reg2 = self._put_in_reg(arg1, regalloc)
+ rr_op(res.value, reg.value, reg2.value)
+ regalloc.possibly_free_var(reg2)
+
+ regalloc.possibly_free_var(res)
+ regalloc.possibly_free_var(reg)
+ return fcond
+ return f
+
class IntOpAsslember(object):
_mixin_ = True
@@ -89,11 +118,11 @@
return fcond
def emit_op_int_mod(self, op, regalloc, fcond):
+ res = regalloc.force_allocate_reg(op.result)
arg1 = regalloc.make_sure_var_in_reg(op.getarg(0), selected_reg=r.r0)
arg2 = regalloc.make_sure_var_in_reg(op.getarg(1), selected_reg=r.r1)
assert arg1 == r.r0
assert arg2 == r.r1
- res = regalloc.try_allocate_reg(op.result)
self.mc.MOD(fcond)
self.mc.MOV_rr(res.value, r.r0.value, cond=fcond)
regalloc.possibly_free_vars_for_op(op)
@@ -108,6 +137,14 @@
reg = regalloc.try_allocate_reg(box)
return reg
+ emit_op_int_and = gen_emit_op_ri('AND')
+ emit_op_int_or = gen_emit_op_ri('ORR')
+ emit_op_int_xor = gen_emit_op_ri('EOR')
+
+ def _check_imm_arg(self, arg, size):
+ return isinstance(arg, ConstInt) and arg.getint() <= size and arg.getint() >= 0
+
+
class GuardOpAssembler(object):
_mixin_ = True
From hakanardo at codespeak.net Sat Oct 30 13:59:27 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 13:59:27 +0200 (CEST)
Subject: [pypy-svn] r78524 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test
Message-ID: <20101030115927.4D3F4282BAD@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 13:59:25 2010
New Revision: 78524
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(hakanardo, cfbolz, arigo) dont propagate guard_no_exception from one iteration to the next of a loop
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Sat Oct 30 13:59:25 2010
@@ -23,6 +23,8 @@
self.optimizer.newoperations = []
jump_args = op.getarglist()
op.initarglist([])
+ # Exceptions not caught in one itteration should not propagate to the next
+ self.optimizer.exception_might_have_happened = False
inputargs = self.inline(self.cloned_operations,
loop.inputargs, jump_args)
loop.inputargs = inputargs
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 13:59:25 2010
@@ -893,7 +893,7 @@
i3 = call(i2, descr=nonwritedescr)
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, 'Not', expected, expected)
# ----------
@@ -905,20 +905,24 @@
guard_value(i2, 1) []
i3 = call_loopinvariant(1, i1, descr=nonwritedescr)
guard_no_exception() []
- guard_value(i2, 1) []
+ guard_value(i3, 1) []
i4 = call_loopinvariant(1, i1, descr=nonwritedescr)
guard_no_exception() []
- guard_value(i2, 1) []
+ guard_value(i4, 1) []
jump(i1)
"""
- expected = """
+ preamble = """
[i1]
i2 = call(1, i1, descr=nonwritedescr)
guard_no_exception() []
guard_value(i2, 1) []
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ [i1]
+ jump(i1)
+ """
+ self.optimize_loop(ops, 'Not', expected, preamble)
# ----------
@@ -928,49 +932,44 @@
[i, p0]
i0 = getfield_gc(p0, descr=valuedescr)
i1 = int_add(i0, i)
- setfield_gc(p0, i1, descr=valuedescr)
- jump(i, p0)
+ p1 = new_with_vtable(ConstClass(node_vtable))
+ setfield_gc(p1, i1, descr=valuedescr)
+ jump(i, p1)
+ """
+ preamble = """
+ [i, p0]
+ i0 = getfield_gc(p0, descr=valuedescr)
+ i1 = int_add(i0, i)
+ jump(i, i1)
"""
expected = """
[i, i2]
i1 = int_add(i2, i)
jump(i, i1)
"""
- self.optimize_loop(ops, 'Not, Virtual(node_vtable, valuedescr=Not)',
- expected, checkspecnodes=False)
+ self.optimize_loop(ops, 'Not, Not', expected, preamble)
def test_virtual_float(self):
ops = """
[f, p0]
f0 = getfield_gc(p0, descr=floatdescr)
f1 = float_add(f0, f)
- setfield_gc(p0, f1, descr=floatdescr)
- jump(f, p0)
+ p1 = new_with_vtable(ConstClass(node_vtable))
+ setfield_gc(p1, f1, descr=floatdescr)
+ jump(f, p1)
"""
- expected = """
- [f, f2]
+ preamble = """
+ [f, p0]
+ f2 = getfield_gc(p0, descr=floatdescr)
f1 = float_add(f2, f)
jump(f, f1)
"""
- self.optimize_loop(ops, 'Not, Virtual(node_vtable, floatdescr=Not)',
- expected, checkspecnodes=False)
-
- def test_virtual_2(self):
- ops = """
- [i, p0]
- i0 = getfield_gc(p0, descr=valuedescr)
- i1 = int_add(i0, i)
- p1 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, i1, descr=valuedescr)
- jump(i, p1)
- """
expected = """
- [i, i2]
- i1 = int_add(i2, i)
- jump(i, i1)
+ [f, f2]
+ f1 = float_add(f2, f)
+ jump(f, f1)
"""
- self.optimize_loop(ops, 'Not, Virtual(node_vtable, valuedescr=Not)',
- expected)
+ self.optimize_loop(ops, 'Not, Not', expected, preamble)
def test_virtual_oois(self):
ops = """
@@ -999,14 +998,10 @@
jump(p0, p1, p2)
"""
expected = """
- [p2]
+ [p0, p1, p2]
# all constant-folded :-)
- jump(p2)
+ jump(p0, p1, p2)
"""
- self.optimize_loop(ops, '''Virtual(node_vtable),
- Virtual(node_vtable),
- Not''',
- expected, checkspecnodes=False)
#
# to be complete, we also check the no-opt case where most comparisons
# are not removed. The exact set of comparisons removed depends on
@@ -1022,7 +1017,7 @@
guard_true(i11) []
jump(p0, p1, p2)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected2)
+ self.optimize_loop(ops, 'Not, Not, Not', expected, expected2)
def test_virtual_default_field(self):
ops = """
@@ -1033,15 +1028,17 @@
# the field 'value' has its default value of 0
jump(p1)
"""
+ preamble = """
+ [p0]
+ i0 = getfield_gc(p0, descr=valuedescr)
+ guard_value(i0, 0) []
+ jump(p1)
+ """
expected = """
- [i]
- guard_value(i, 0) []
- jump(0)
+ []
+ jump()
"""
- # the 'expected' is sub-optimal, but it should be done by another later
- # optimization step. See test_find_nodes_default_field() for why.
- self.optimize_loop(ops, 'Virtual(node_vtable, valuedescr=Not)',
- expected)
+ self.optimize_loop(ops, 'Not', expected, preamble)
def test_virtual_3(self):
ops = """
From arigo at codespeak.net Sat Oct 30 14:00:10 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 30 Oct 2010 14:00:10 +0200 (CEST)
Subject: [pypy-svn] r78525 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030120010.9FF5E282BF0@codespeak.net>
Author: arigo
Date: Sat Oct 30 14:00:09 2010
New Revision: 78525
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
Reduce this file's length by importing stuff from test_optimizebasic.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 14:00:09 2010
@@ -13,14 +13,7 @@
from pypy.jit.metainterp import executor, compile, resume, history
from pypy.jit.metainterp.resoperation import rop, opname, ResOperation
from pypy.jit.metainterp.test.oparser import pure_parse
-
-##class FakeFrame(object):
-## parent_resumedata_snapshot = None
-## parent_resumedata_frame_info_list = None
-
-## def __init__(self, code="", pc=0):
-## self.jitcode = code
-## self.pc = pc
+from pypy.jit.metainterp.test.test_optimizebasic import equaloplists
class Fake(object):
failargs_limit = 1000
@@ -34,175 +27,6 @@
self.options = Fake()
self.globaldata = Fake()
-def test_store_final_boxes_in_guard():
- from pypy.jit.metainterp.compile import ResumeGuardDescr
- from pypy.jit.metainterp.resume import tag, TAGBOX
- b0 = BoxInt()
- b1 = BoxInt()
- opt = optimizeopt.Optimizer(FakeMetaInterpStaticData(LLtypeMixin.cpu),
- None)
- fdescr = ResumeGuardDescr(None, None)
- op = ResOperation(rop.GUARD_TRUE, ['dummy'], None, descr=fdescr)
- # setup rd data
- fi0 = resume.FrameInfo(None, "code0", 11)
- fdescr.rd_frame_info_list = resume.FrameInfo(fi0, "code1", 33)
- snapshot0 = resume.Snapshot(None, [b0])
- fdescr.rd_snapshot = resume.Snapshot(snapshot0, [b1])
- #
- opt.store_final_boxes_in_guard(op)
- if op.getfailargs() == [b0, b1]:
- assert fdescr.rd_numb.nums == [tag(1, TAGBOX)]
- assert fdescr.rd_numb.prev.nums == [tag(0, TAGBOX)]
- else:
- assert op.getfailargs() == [b1, b0]
- assert fdescr.rd_numb.nums == [tag(0, TAGBOX)]
- assert fdescr.rd_numb.prev.nums == [tag(1, TAGBOX)]
- assert fdescr.rd_virtuals is None
- assert fdescr.rd_consts == []
-
-def test_sharing_field_lists_of_virtual():
- class FakeOptimizer(object):
- class cpu(object):
- pass
- opt = FakeOptimizer()
- virt1 = virtualize.AbstractVirtualStructValue(opt, None)
- lst1 = virt1._get_field_descr_list()
- assert lst1 == []
- lst2 = virt1._get_field_descr_list()
- assert lst1 is lst2
- virt1.setfield(LLtypeMixin.valuedescr, optimizeopt.OptValue(None))
- lst3 = virt1._get_field_descr_list()
- assert lst3 == [LLtypeMixin.valuedescr]
- lst4 = virt1._get_field_descr_list()
- assert lst3 is lst4
-
- virt2 = virtualize.AbstractVirtualStructValue(opt, None)
- lst5 = virt2._get_field_descr_list()
- assert lst5 is lst1
- virt2.setfield(LLtypeMixin.valuedescr, optimizeopt.OptValue(None))
- lst6 = virt1._get_field_descr_list()
- assert lst6 is lst3
-
-def test_reuse_vinfo():
- class FakeVInfo(object):
- def set_content(self, fieldnums):
- self.fieldnums = fieldnums
- def equals(self, fieldnums):
- return self.fieldnums == fieldnums
- class FakeVirtualValue(virtualize.AbstractVirtualValue):
- def _make_virtual(self, *args):
- return FakeVInfo()
- v1 = FakeVirtualValue(None, None, None)
- vinfo1 = v1.make_virtual_info(None, [1, 2, 4])
- vinfo2 = v1.make_virtual_info(None, [1, 2, 4])
- assert vinfo1 is vinfo2
- vinfo3 = v1.make_virtual_info(None, [1, 2, 6])
- assert vinfo3 is not vinfo2
- vinfo4 = v1.make_virtual_info(None, [1, 2, 6])
- assert vinfo3 is vinfo4
-
-def test_descrlist_dict():
- from pypy.jit.metainterp import optimizeutil
- h1 = optimizeutil.descrlist_hash([])
- h2 = optimizeutil.descrlist_hash([LLtypeMixin.valuedescr])
- h3 = optimizeutil.descrlist_hash(
- [LLtypeMixin.valuedescr, LLtypeMixin.nextdescr])
- assert h1 != h2
- assert h2 != h3
- assert optimizeutil.descrlist_eq([], [])
- assert not optimizeutil.descrlist_eq([], [LLtypeMixin.valuedescr])
- assert optimizeutil.descrlist_eq([LLtypeMixin.valuedescr],
- [LLtypeMixin.valuedescr])
- assert not optimizeutil.descrlist_eq([LLtypeMixin.valuedescr],
- [LLtypeMixin.nextdescr])
- assert optimizeutil.descrlist_eq([LLtypeMixin.valuedescr, LLtypeMixin.nextdescr],
- [LLtypeMixin.valuedescr, LLtypeMixin.nextdescr])
- assert not optimizeutil.descrlist_eq([LLtypeMixin.nextdescr, LLtypeMixin.valuedescr],
- [LLtypeMixin.valuedescr, LLtypeMixin.nextdescr])
-
- # descrlist_eq should compare by identity of the descrs, not by the result
- # of sort_key
- class FakeDescr(object):
- def sort_key(self):
- return 1
-
- assert not optimizeutil.descrlist_eq([FakeDescr()], [FakeDescr()])
-
-
-# ____________________________________________________________
-
-def equaloplists(oplist1, oplist2, strict_fail_args=True, remap={}):
- print '-'*20, 'Comparing lists', '-'*20
- for op1, op2 in zip(oplist1, oplist2):
- txt1 = str(op1)
- txt2 = str(op2)
- while txt1 or txt2:
- print '%-39s| %s' % (txt1[:39], txt2[:39])
- txt1 = txt1[39:]
- txt2 = txt2[39:]
- assert op1.getopnum() == op2.getopnum()
- assert op1.numargs() == op2.numargs()
- for i in range(op1.numargs()):
- x = op1.getarg(i)
- y = op2.getarg(i)
- assert x == remap.get(y, y)
- if op2.result in remap:
- assert op1.result == remap[op2.result]
- else:
- remap[op2.result] = op1.result
- if op1.getopnum() != rop.JUMP: # xxx obscure
- assert op1.getdescr() == op2.getdescr()
- if op1.getfailargs() or op2.getfailargs():
- assert len(op1.getfailargs()) == len(op2.getfailargs())
- if strict_fail_args:
- for x, y in zip(op1.getfailargs(), op2.getfailargs()):
- assert x == remap.get(y, y)
- else:
- fail_args1 = set(op1.getfailargs())
- fail_args2 = set([remap.get(y, y) for y in op2.getfailargs()])
- assert fail_args1 == fail_args2
- assert len(oplist1) == len(oplist2)
- print '-'*57
- return True
-
-def test_equaloplists():
- ops = """
- [i0]
- i1 = int_add(i0, 1)
- i2 = int_add(i1, 1)
- guard_true(i1) [i2]
- jump(i1)
- """
- namespace = {}
- loop1 = pure_parse(ops, namespace=namespace)
- loop2 = pure_parse(ops, namespace=namespace)
- loop3 = pure_parse(ops.replace("i2 = int_add", "i2 = int_sub"),
- namespace=namespace)
- assert equaloplists(loop1.operations, loop2.operations)
- py.test.raises(AssertionError,
- "equaloplists(loop1.operations, loop3.operations)")
-
-def test_equaloplists_fail_args():
- ops = """
- [i0]
- i1 = int_add(i0, 1)
- i2 = int_add(i1, 1)
- guard_true(i1) [i2, i1]
- jump(i1)
- """
- namespace = {}
- loop1 = pure_parse(ops, namespace=namespace)
- loop2 = pure_parse(ops.replace("[i2, i1]", "[i1, i2]"),
- namespace=namespace)
- py.test.raises(AssertionError,
- "equaloplists(loop1.operations, loop2.operations)")
- assert equaloplists(loop1.operations, loop2.operations,
- strict_fail_args=False)
- loop3 = pure_parse(ops.replace("[i2, i1]", "[i2, i0]"),
- namespace=namespace)
- py.test.raises(AssertionError,
- "equaloplists(loop1.operations, loop3.operations)")
-
# ____________________________________________________________
class Storage(compile.ResumeGuardDescr):
From david at codespeak.net Sat Oct 30 16:02:18 2010
From: david at codespeak.net (david at codespeak.net)
Date: Sat, 30 Oct 2010 16:02:18 +0200 (CEST)
Subject: [pypy-svn] r78526 - in
pypy/branch/arm-backend/pypy/jit/backend/arm: . test
Message-ID: <20101030140218.96EB2282BDD@codespeak.net>
Author: david
Date: Sat Oct 30 16:02:16 2010
New Revision: 78526
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
Log:
Implement shift operations int_lshift, int_rshift, uint_rshift
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py Sat Oct 30 16:02:16 2010
@@ -94,6 +94,42 @@
| reg_operation(rd, rn, rm, imm, s, shifttype))
return f
+def define_data_proc_register_shifted(name, table):
+ n = ((0x1 << 4) | (table['op1'] & 0x1F) << 20 | (table['op2'] & 0x3) << 5)
+ if 'result' in table and not table['result']:
+ result = False
+ else:
+ result = True
+ if name[-2:] == 'sr':
+ if result:
+ def f(self, rd, rn, rm, rs, cond=cond.AL, s=0, shifttype=0):
+ self.write32(n
+ | cond << 28
+ | (s & 0x1) << 20
+ | (rn & 0xF) << 16
+ | (rd & 0xF) << 12
+ | (rs & 0xF) << 8
+ | (shifttype & 0x3) << 5
+ | (rm & 0xF))
+ else:
+ def f(self, rn, rm, rs, cond=cond.AL, s=0, shifttype=0):
+ self.write32(n
+ | cond << 28
+ | (s & 0x1) << 20
+ | (rn & 0xF) << 16
+ | (rs & 0xF) << 8
+ | (shifttype & 0x3) << 5
+ | (rm & 0xF))
+ else:
+ def f(self, rd, rn, rm, cond=cond.AL, s=0):
+ self.write32(n
+ | cond << 28
+ | (s & 0x1) << 20
+ | (rd & 0xF) << 12
+ | (rm & 0xF) << 8
+ | (rn & 0xF))
+ return f
+
def define_supervisor_and_coproc(name, table):
n = (0x3 << 26 | (table['op1'] & 0x3F) << 20 | (table['op'] & 0x1) << 4)
def f(self, coproc, opc1, rt, crn, crm, opc2=0, cond=cond.AL):
@@ -162,7 +198,8 @@
(instructions.data_proc, define_data_proc),
(instructions.data_proc_imm, define_data_proc_imm),
(instructions.supervisor_and_coproc, define_supervisor_and_coproc),
- (instructions.multiply, define_multiply_instructions)]
+ (instructions.multiply, define_multiply_instructions),
+ (instructions.data_proc_reg_shift_reg, define_data_proc_register_shifted)]
for inss, gen in i_g_map:
for key, val in inss.iteritems():
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py Sat Oct 30 16:02:16 2010
@@ -31,6 +31,30 @@
'ASR_ri': {'op1':0x1A, 'op2':0, 'op3':0x2, 'op2cond':'', 'result':False, 'base':True},
#'RRX_ri': {'op1':0x1A, 'op2':0, 'op3':0x3, 'op2cond':'0', 'result':False, 'base':True},
'ROR_ri': {'op1':0x1A, 'op2':0x0, 'op3':0x3, 'op2cond':'!0', 'result':True, 'base':False},
+ #BIC
+ #MVN
+}
+
+data_proc_reg_shift_reg = {
+ 'AND_rr_sr': {'op1':0x0, 'op2':0},
+ 'EOR_rr_sr': {'op1':0x2, 'op2':0},
+ 'SUB_rr_sr': {'op1':0x4, 'op2':0},
+ 'RSB_rr_sr': {'op1':0x6, 'op2':0},
+ 'ADD_rr_sr': {'op1':0x8, 'op2':0},
+ 'ADC_rr_sr': {'op1':0xA, 'op2':0},
+ 'SBC_rr_sr': {'op1':0xC, 'op2':0},
+ 'RSC_rr_sr': {'op1':0xE, 'op2':0},
+ 'TST_rr_sr': {'op1':0x11, 'op2':0, 'result': False},
+ 'TEQ_rr_sr': {'op1':0x13, 'op2':0, 'result': False},
+ 'CMP_rr_sr': {'op1':0x15, 'op2':0, 'result': False},
+ 'CMN_rr_sr': {'op1':0x17, 'op2':0, 'result': False},
+ 'ORR_rr_sr': {'op1':0x18, 'op2':0},
+ 'LSL_rr': {'op1':0x1A, 'op2':0, },
+ 'LSR_rr': {'op1':0x1A, 'op2':0x1},
+ 'ASR_rr': {'op1':0x1A, 'op2':0x2},
+ #'RRX_rr': {'op1':0x1A, 'op2':0,},
+ 'ROR_rr': {'op1':0x1A, 'op2':0x3},
+ # BIC, MVN
}
data_proc_imm = {
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Sat Oct 30 16:02:16 2010
@@ -12,7 +12,7 @@
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.lltypesystem import lltype, rffi, llmemory
-def gen_emit_op_ri(opname):
+def gen_emit_op_ri(opname, imm_size=0xFF, commutative=True):
def f(self, op, regalloc, fcond):
ri_op = getattr(self.mc, '%s_ri' % opname)
rr_op = getattr(self.mc, '%s_rr' % opname)
@@ -20,17 +20,15 @@
arg0 = op.getarg(0)
arg1 = op.getarg(1)
res = regalloc.try_allocate_reg(op.result)
- if self._check_imm_arg(arg0, 0xFF) and not isinstance(arg1, ConstInt):
- print 'arg0 is imm'
+ if (commutative
+ and self._check_imm_arg(arg0, imm_size)
+ and not isinstance(arg1, ConstInt)):
reg = regalloc.try_allocate_reg(arg1)
ri_op(res.value, reg.value, imm=arg0.getint(), cond=fcond)
- elif self._check_imm_arg(arg1, 0xFF) and not isinstance(arg0, ConstInt):
- print 'arg1 is imm'
- box = Box()
+ elif self._check_imm_arg(arg1, imm_size) and not isinstance(arg0, ConstInt):
reg = regalloc.try_allocate_reg(arg0)
ri_op(res.value, reg.value, imm=arg1.getint(), cond=fcond)
else:
- print 'generating rr'
reg = self._put_in_reg(arg0, regalloc)
reg2 = self._put_in_reg(arg1, regalloc)
rr_op(res.value, reg.value, reg2.value)
@@ -140,9 +138,14 @@
emit_op_int_and = gen_emit_op_ri('AND')
emit_op_int_or = gen_emit_op_ri('ORR')
emit_op_int_xor = gen_emit_op_ri('EOR')
+ emit_op_int_lshift = gen_emit_op_ri('LSL', imm_size=0x1F, commutative=False)
+ emit_op_int_rshift = gen_emit_op_ri('ASR', imm_size=0x1F, commutative=False)
+ emit_op_uint_rshift = gen_emit_op_ri('LSR', imm_size=0x1F, commutative=False)
+
def _check_imm_arg(self, arg, size):
- return isinstance(arg, ConstInt) and arg.getint() <= size and arg.getint() >= 0
+ #XXX check ranges for different operations
+ return isinstance(arg, ConstInt) and arg.getint() <= size and arg.getint() > 0
class GuardOpAssembler(object):
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Sat Oct 30 16:02:16 2010
@@ -126,23 +126,6 @@
def setup_method(self, ffuu_method):
self.cb = CodeBuilder()
-def build_tests():
- test_name = 'test_generated_%s'
- for key, value in instructions.load_store.iteritems():
- if value['imm']:
- f = gen_test_imm_func
- else:
- f = gen_test_reg_func
- build_test(f, key, value, test_name)
-
- for key, value, in instructions.data_proc.iteritems():
- build_test(gen_test_data_reg_func, key, value, test_name)
-
- for key, value, in instructions.data_proc_imm.iteritems():
- build_test(gen_test_data_proc_imm_func, key, value, test_name)
-
- for key, value, in instructions.multiply.iteritems():
- build_test(gen_test_mul_func, key, value, test_name)
# XXX refactor this functions
def build_test(builder, key, value, test_name):
@@ -200,6 +183,31 @@
self.assert_equal('%s r3, r7, r12' % name)
return f
+def gen_test_data_reg_shift_reg_func(name, table):
+ if name[-2:] == 'rr':
+ def f(self):
+ func = getattr(self.cb, name)
+ func(r.r3.value, r.r7.value, r.r12.value)
+ self.assert_equal('%s r3, r7, r12' % name[:name.index('_')])
+
+ else:
+ if 'result' in table and not table['result']:
+ result = False
+ else:
+ result = True
+ if result:
+ def f(self):
+ func = getattr(self.cb, name)
+ func(r.r3.value, r.r7.value, r.r8.value, r.r11.value, shifttype=0x2)
+ self.assert_equal('%s r3, r7, r8, ASR r11' % name[:name.index('_')])
+ else:
+ def f(self):
+ func = getattr(self.cb, name)
+ func(r.r3.value, r.r7.value, r.r11.value, shifttype=0x2)
+ self.assert_equal('%s r3, r7, ASR r11' % name[:name.index('_')])
+
+ return f
+
def gen_test_data_reg_func(name, table):
if name[-2:] == 'ri':
def f(self):
@@ -220,4 +228,25 @@
return f
+def build_tests():
+ test_name = 'test_generated_%s'
+ for key, value in instructions.load_store.iteritems():
+ if value['imm']:
+ f = gen_test_imm_func
+ else:
+ f = gen_test_reg_func
+ build_test(f, key, value, test_name)
+
+ for key, value, in instructions.data_proc.iteritems():
+ build_test(gen_test_data_reg_func, key, value, test_name)
+
+ for key, value, in instructions.data_proc_reg_shift_reg.iteritems():
+ build_test(gen_test_data_reg_shift_reg_func, key, value, test_name)
+
+ for key, value, in instructions.data_proc_imm.iteritems():
+ build_test(gen_test_data_proc_imm_func, key, value, test_name)
+
+ for key, value, in instructions.multiply.iteritems():
+ build_test(gen_test_mul_func, key, value, test_name)
+
build_tests()
From hakanardo at codespeak.net Sat Oct 30 16:03:55 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 16:03:55 +0200 (CEST)
Subject: [pypy-svn] r78527 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test
Message-ID: <20101030140355.22BC3282BEF@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 16:03:53 2010
New Revision: 78527
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(hakanardo, cfbolz, arigo) default values of virtuals
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Sat Oct 30 16:03:53 2010
@@ -371,7 +371,8 @@
# optimizefindnode should ensure that fieldvalue is found
assert isinstance(value, AbstractVirtualValue)
fieldvalue = value.getfield(op.getdescr(), None)
- assert fieldvalue is not None
+ if fieldvalue is None:
+ fieldvalue = self.optimizer.new_const(op.getdescr())
self.make_equal_to(op.result, fieldvalue)
else:
value.ensure_nonnull()
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 16:03:53 2010
@@ -856,7 +856,7 @@
[p0]
i0 = getfield_gc(p0, descr=valuedescr)
guard_value(i0, 0) []
- jump(p1)
+ jump()
"""
expected = """
[]
From hakanardo at codespeak.net Sat Oct 30 16:20:28 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 16:20:28 +0200 (CEST)
Subject: [pypy-svn] r78528 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test
Message-ID: <20101030142028.AD2AC36C394@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 16:20:26 2010
New Revision: 78528
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Sat Oct 30 16:20:26 2010
@@ -13,6 +13,7 @@
self.cloned_operations = []
for op in self.optimizer.loop.operations:
self.cloned_operations.append(op.clone())
+
def propagate_forward(self, op):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 16:20:26 2010
@@ -891,14 +891,21 @@
setfield_gc(p1, i2, descr=valuedescr)
jump(i3, p1)
"""
+ preamble = """
+ [i0, p0]
+ guard_class(p0, ConstClass(node_vtable)) []
+ i1 = getfield_gc(p0, descr=valuedescr)
+ i2 = int_sub(i1, 1)
+ i3 = int_add(i0, i1)
+ jump(i3, i2)
+ """
expected = """
[i0, i1]
i2 = int_sub(i1, 1)
i3 = int_add(i0, i1)
jump(i3, i2)
"""
- self.optimize_loop(ops, 'Not, Virtual(node_vtable, valuedescr=Not)',
- expected)
+ self.optimize_loop(ops, 'Not, Not', expected, preamble)
def test_virtual_5(self):
ops = """
@@ -914,18 +921,21 @@
setfield_gc(p1, p2, descr=nextdescr)
jump(i3, p1)
"""
+ preamble = """
+ [i0, p0]
+ guard_class(p0, ConstClass(node_vtable)) []
+ i1 = getfield_gc(p0, descr=valuedescr)
+ i2 = int_sub(i1, 1)
+ i3 = int_add(i0, i1)
+ jump(i3, i2, i1)
+ """
expected = """
[i0, i1, i1bis]
i2 = int_sub(i1, 1)
i3 = int_add(i0, i1)
jump(i3, i2, i1)
"""
- self.optimize_loop(ops,
- '''Not, Virtual(node_vtable,
- valuedescr=Not,
- nextdescr=Virtual(node_vtable2,
- valuedescr=Not))''',
- expected)
+ self.optimize_loop(ops, 'Not, Not', expected, preamble)
def test_virtual_constant_isnull(self):
ops = """
From arigo at codespeak.net Sat Oct 30 16:28:33 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 30 Oct 2010 16:28:33 +0200 (CEST)
Subject: [pypy-svn] r78529 - pypy/trunk/pypy/translator/c/gcc
Message-ID: <20101030142833.CBEDF282BEC@codespeak.net>
Author: arigo
Date: Sat Oct 30 16:28:32 2010
New Revision: 78529
Modified:
pypy/trunk/pypy/translator/c/gcc/trackgcroot.py
Log:
Add psllX.
Modified: pypy/trunk/pypy/translator/c/gcc/trackgcroot.py
==============================================================================
--- pypy/trunk/pypy/translator/c/gcc/trackgcroot.py (original)
+++ pypy/trunk/pypy/translator/c/gcc/trackgcroot.py Sat Oct 30 16:28:32 2010
@@ -455,7 +455,7 @@
'inc', 'dec', 'not', 'neg', 'or', 'and', 'sbb', 'adc',
'shl', 'shr', 'sal', 'sar', 'rol', 'ror', 'mul', 'imul', 'div', 'idiv',
'bswap', 'bt', 'rdtsc',
- 'punpck', 'pshufd',
+ 'punpck', 'pshufd', 'psll',
# zero-extending moves should not produce GC pointers
'movz',
])
From hakanardo at codespeak.net Sat Oct 30 16:41:30 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 16:41:30 +0200 (CEST)
Subject: [pypy-svn] r78530 - in pypy/branch/jit-unroll-loops: . dotviewer
lib-python lib-python/modified-2.5.2/test lib_pypy/_ctypes
pypy pypy/annotation pypy/annotation/test pypy/config
pypy/config/test pypy/doc pypy/doc/config pypy/interpreter
pypy/interpreter/astcompiler/test pypy/interpreter/pyparser
pypy/interpreter/test pypy/jit/backend
pypy/jit/backend/llgraph pypy/jit/backend/llsupport
pypy/jit/backend/llsupport/test pypy/jit/backend/test
pypy/jit/backend/x86 pypy/jit/backend/x86/test
pypy/jit/codewriter pypy/jit/codewriter/test
pypy/jit/metainterp pypy/jit/metainterp/optimizeopt
pypy/jit/metainterp/test pypy/jit/tl pypy/jit/tl/spli
pypy/jit/tl/tla pypy/jit/tool pypy/module/__builtin__
pypy/module/__builtin__/test pypy/module/_bisect
pypy/module/_bisect/test pypy/module/_ffi
pypy/module/_ffi/test pypy/module/_rawffi
pypy/module/_rawffi/test pypy/module/_socket/test
pypy/module/_sre pypy/module/_weakref pypy/module/_winreg
pypy/module/array pypy/module/array/benchmark
pypy/module/array/test pypy/module/bz2 pypy/module/cpyext
pypy/module/cpyext/test pypy/module/gc pypy/module/gc/test
pypy/module/imp pypy/module/imp/test pypy/module/parser/test
pypy/module/pypyjit pypy/module/pypyjit/test
pypy/module/signal pypy/module/sys
pypy/module/test_lib_pypy/ctypes_tests pypy/module/thread
pypy/objspace pypy/objspace/flow pypy/objspace/flow/test
pypy/objspace/std pypy/objspace/std/test pypy/objspace/test
pypy/rlib pypy/rlib/rsre pypy/rlib/rsre/test pypy/rlib/test
pypy/rpython pypy/rpython/lltypesystem
pypy/rpython/lltypesystem/test pypy/rpython/memory
pypy/rpython/memory/gc pypy/rpython/memory/gc/test
pypy/rpython/memory/gctransform pypy/rpython/memory/test
pypy/rpython/module pypy/rpython/test pypy/tool
pypy/tool/release pypy/tool/release/test pypy/tool/test
pypy/translator pypy/translator/c pypy/translator/c/gcc
pypy/translator/c/gcc/test pypy/translator/c/gcc/test/darwin64
pypy/translator/c/gcc/test/elf
pypy/translator/c/gcc/test/elf64 pypy/translator/c/src
pypy/translator/c/test pypy/translator/goal pypy/translator/jvm/test
pypy/translator/oosupport/test_template pypy/translator/platform
Message-ID: <20101030144130.B614C282BF0@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 16:41:17 2010
New Revision: 78530
Added:
pypy/branch/jit-unroll-loops/lib-python/modified-2.5.2/test/test_dict.py
- copied unchanged from r78528, pypy/trunk/lib-python/modified-2.5.2/test/test_dict.py
pypy/branch/jit-unroll-loops/pypy/doc/config/objspace.std.withmapdict.txt
- copied unchanged from r78528, pypy/trunk/pypy/doc/config/objspace.std.withmapdict.txt
pypy/branch/jit-unroll-loops/pypy/doc/config/objspace.std.withstrbuf.txt
- copied unchanged from r78528, pypy/trunk/pypy/doc/config/objspace.std.withstrbuf.txt
pypy/branch/jit-unroll-loops/pypy/doc/config/objspace.usemodules._bisect.txt
- copied unchanged from r78528, pypy/trunk/pypy/doc/config/objspace.usemodules._bisect.txt
pypy/branch/jit-unroll-loops/pypy/doc/config/objspace.usemodules._ffi.txt
- copied unchanged from r78528, pypy/trunk/pypy/doc/config/objspace.usemodules._ffi.txt
pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/autopath.py
- copied unchanged from r78528, pypy/trunk/pypy/interpreter/pyparser/autopath.py
pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/genpytokenize.py
- copied unchanged from r78528, pypy/trunk/pypy/interpreter/pyparser/genpytokenize.py
pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/pylexer.py
- copied unchanged from r78528, pypy/trunk/pypy/interpreter/pyparser/pylexer.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/conftest.py
- copied unchanged from r78528, pypy/trunk/pypy/jit/backend/conftest.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/ffisupport.py
- copied unchanged from r78528, pypy/trunk/pypy/jit/backend/llsupport/ffisupport.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_ffisupport.py
- copied unchanged from r78528, pypy/trunk/pypy/jit/backend/llsupport/test/test_ffisupport.py
pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_void_list.py
- copied unchanged from r78528, pypy/trunk/pypy/jit/codewriter/test/test_void_list.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/greenfield.py
- copied unchanged from r78528, pypy/trunk/pypy/jit/metainterp/greenfield.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/fficall.py
- copied unchanged from r78528, pypy/trunk/pypy/jit/metainterp/optimizeopt/fficall.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_fficall.py
- copied unchanged from r78528, pypy/trunk/pypy/jit/metainterp/test/test_fficall.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_greenfield.py
- copied unchanged from r78528, pypy/trunk/pypy/jit/metainterp/test/test_greenfield.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizefficall.py
- copied unchanged from r78528, pypy/trunk/pypy/jit/metainterp/test/test_optimizefficall.py
pypy/branch/jit-unroll-loops/pypy/jit/tl/jittest.py
- copied unchanged from r78528, pypy/trunk/pypy/jit/tl/jittest.py
pypy/branch/jit-unroll-loops/pypy/jit/tool/oparser.py
- copied unchanged from r78528, pypy/trunk/pypy/jit/tool/oparser.py
pypy/branch/jit-unroll-loops/pypy/jit/tool/pypytrace.vim
- copied unchanged from r78528, pypy/trunk/pypy/jit/tool/pypytrace.vim
pypy/branch/jit-unroll-loops/pypy/module/_bisect/ (props changed)
- copied from r78528, pypy/trunk/pypy/module/_bisect/
pypy/branch/jit-unroll-loops/pypy/module/_ffi/ (props changed)
- copied from r78528, pypy/trunk/pypy/module/_ffi/
pypy/branch/jit-unroll-loops/pypy/objspace/std/mapdict.py
- copied unchanged from r78528, pypy/trunk/pypy/objspace/std/mapdict.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/strbufobject.py
- copied unchanged from r78528, pypy/trunk/pypy/objspace/std/strbufobject.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_mapdict.py
- copied unchanged from r78528, pypy/trunk/pypy/objspace/std/test/test_mapdict.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_methodcache.py
- copied unchanged from r78528, pypy/trunk/pypy/objspace/std/test/test_methodcache.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_strbufobject.py
- copied unchanged from r78528, pypy/trunk/pypy/objspace/std/test/test_strbufobject.py
pypy/branch/jit-unroll-loops/pypy/rlib/clibffi.py
- copied unchanged from r78528, pypy/trunk/pypy/rlib/clibffi.py
pypy/branch/jit-unroll-loops/pypy/rlib/libffi.py
- copied unchanged from r78528, pypy/trunk/pypy/rlib/libffi.py
pypy/branch/jit-unroll-loops/pypy/rlib/rerased.py
- copied unchanged from r78528, pypy/trunk/pypy/rlib/rerased.py
pypy/branch/jit-unroll-loops/pypy/rlib/rsre/rsre_jit.py
- copied unchanged from r78528, pypy/trunk/pypy/rlib/rsre/rsre_jit.py
pypy/branch/jit-unroll-loops/pypy/rlib/rsre/test/conftest.py
- copied unchanged from r78528, pypy/trunk/pypy/rlib/rsre/test/conftest.py
pypy/branch/jit-unroll-loops/pypy/rlib/rsre/test/test_zjit.py
- copied unchanged from r78528, pypy/trunk/pypy/rlib/rsre/test/test_zjit.py
pypy/branch/jit-unroll-loops/pypy/rlib/test/test_clibffi.py
- copied unchanged from r78528, pypy/trunk/pypy/rlib/test/test_clibffi.py
pypy/branch/jit-unroll-loops/pypy/rlib/test/test_libffi.py
- copied unchanged from r78528, pypy/trunk/pypy/rlib/test/test_libffi.py
pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rerased.py
- copied unchanged from r78528, pypy/trunk/pypy/rlib/test/test_rerased.py
pypy/branch/jit-unroll-loops/pypy/tool/leakfinder.py
- copied unchanged from r78528, pypy/trunk/pypy/tool/leakfinder.py
pypy/branch/jit-unroll-loops/pypy/tool/test/test_leakfinder.py
- copied unchanged from r78528, pypy/trunk/pypy/tool/test/test_leakfinder.py
pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/test/darwin64/
- copied from r78528, pypy/trunk/pypy/translator/c/gcc/test/darwin64/
pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/test/elf/track10.s
- copied unchanged from r78528, pypy/trunk/pypy/translator/c/gcc/test/elf/track10.s
pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/test/elf/track11.s
- copied unchanged from r78528, pypy/trunk/pypy/translator/c/gcc/test/elf/track11.s
pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/test/elf64/track_loadconst.s
- copied unchanged from r78528, pypy/trunk/pypy/translator/c/gcc/test/elf64/track_loadconst.s
pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/test/elf64/track_rpyassertfailed.s
- copied unchanged from r78528, pypy/trunk/pypy/translator/c/gcc/test/elf64/track_rpyassertfailed.s
pypy/branch/jit-unroll-loops/pypy/translator/c/src/debug_alloc.h
- copied unchanged from r78528, pypy/trunk/pypy/translator/c/src/debug_alloc.h
Removed:
pypy/branch/jit-unroll-loops/pypy/doc/config/objspace.std.withinlineddict.txt
pypy/branch/jit-unroll-loops/pypy/doc/config/objspace.std.withshadowtracking.txt
pypy/branch/jit-unroll-loops/pypy/doc/config/objspace.std.withsharingdict.txt
pypy/branch/jit-unroll-loops/pypy/jit/backend/test/conftest.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/oparser.py
pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/test_find.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/inlinedict.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/sharingdict.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_inlinedict.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_shadowtracking.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_sharingdict.py
pypy/branch/jit-unroll-loops/pypy/rpython/rspecialcase.py
pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rspecialcase.py
Modified:
pypy/branch/jit-unroll-loops/ (props changed)
pypy/branch/jit-unroll-loops/dotviewer/drawgraph.py
pypy/branch/jit-unroll-loops/lib-python/conftest.py
pypy/branch/jit-unroll-loops/lib-python/modified-2.5.2/test/test_descr.py
pypy/branch/jit-unroll-loops/lib_pypy/_ctypes/function.py
pypy/branch/jit-unroll-loops/lib_pypy/_ctypes/structure.py
pypy/branch/jit-unroll-loops/lib_pypy/_ctypes/union.py
pypy/branch/jit-unroll-loops/pypy/ (props changed)
pypy/branch/jit-unroll-loops/pypy/annotation/bookkeeper.py
pypy/branch/jit-unroll-loops/pypy/annotation/builtin.py
pypy/branch/jit-unroll-loops/pypy/annotation/classdef.py
pypy/branch/jit-unroll-loops/pypy/annotation/description.py
pypy/branch/jit-unroll-loops/pypy/annotation/dictdef.py
pypy/branch/jit-unroll-loops/pypy/annotation/listdef.py
pypy/branch/jit-unroll-loops/pypy/annotation/model.py
pypy/branch/jit-unroll-loops/pypy/annotation/policy.py
pypy/branch/jit-unroll-loops/pypy/annotation/specialize.py
pypy/branch/jit-unroll-loops/pypy/annotation/test/test_annrpython.py
pypy/branch/jit-unroll-loops/pypy/annotation/test/test_model.py
pypy/branch/jit-unroll-loops/pypy/annotation/unaryop.py
pypy/branch/jit-unroll-loops/pypy/config/pypyoption.py
pypy/branch/jit-unroll-loops/pypy/config/test/test_pypyoption.py
pypy/branch/jit-unroll-loops/pypy/conftest.py
pypy/branch/jit-unroll-loops/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt
pypy/branch/jit-unroll-loops/pypy/doc/docindex.txt
pypy/branch/jit-unroll-loops/pypy/doc/interpreter-optimizations.txt
pypy/branch/jit-unroll-loops/pypy/interpreter/astcompiler/test/test_compiler.py
pypy/branch/jit-unroll-loops/pypy/interpreter/baseobjspace.py
pypy/branch/jit-unroll-loops/pypy/interpreter/function.py
pypy/branch/jit-unroll-loops/pypy/interpreter/gateway.py
pypy/branch/jit-unroll-loops/pypy/interpreter/pycode.py
pypy/branch/jit-unroll-loops/pypy/interpreter/pyframe.py
pypy/branch/jit-unroll-loops/pypy/interpreter/pyopcode.py
pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/pytokenize.py
pypy/branch/jit-unroll-loops/pypy/interpreter/test/test_function.py
pypy/branch/jit-unroll-loops/pypy/interpreter/typedef.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/detect_cpu.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/runner.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/descr.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/gc.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/llmodel.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/symbolic.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_descr.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_gc.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/model.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/test/runner_test.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/test/test_ll_random.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/test/test_random.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/assembler.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/regalloc.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/regloc.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/runner.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/rx86.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_assembler.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_gc_integration.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_regalloc.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_runner.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_string.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_zrpy_gc.py
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_ztranslation.py
pypy/branch/jit-unroll-loops/pypy/jit/codewriter/call.py
pypy/branch/jit-unroll-loops/pypy/jit/codewriter/codewriter.py
pypy/branch/jit-unroll-loops/pypy/jit/codewriter/effectinfo.py
pypy/branch/jit-unroll-loops/pypy/jit/codewriter/format.py
pypy/branch/jit-unroll-loops/pypy/jit/codewriter/jtransform.py
pypy/branch/jit-unroll-loops/pypy/jit/codewriter/support.py
pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_codewriter.py
pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_effectinfo.py
pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_jtransform.py
pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_list.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/blackhole.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/executor.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/graphpage.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/history.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/jitdriver.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize_nopspec.py (contents, props changed)
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (props changed)
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resoperation.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/simple_optimize.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_jitdriver.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_logger.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_nopspec.py (props changed)
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_oparser.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizefindnode.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_string.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualref.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_warmspot.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/virtualref.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmspot.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmstate.py
pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit.py
pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py
pypy/branch/jit-unroll-loops/pypy/jit/tl/spli/interpreter.py
pypy/branch/jit-unroll-loops/pypy/jit/tl/tla/test_tla.py
pypy/branch/jit-unroll-loops/pypy/jit/tool/loopviewer.py
pypy/branch/jit-unroll-loops/pypy/jit/tool/showstats.py
pypy/branch/jit-unroll-loops/pypy/jit/tool/traceviewer.py
pypy/branch/jit-unroll-loops/pypy/module/__builtin__/interp_classobj.py
pypy/branch/jit-unroll-loops/pypy/module/__builtin__/test/test_classobj.py
pypy/branch/jit-unroll-loops/pypy/module/_bisect/test/ (props changed)
pypy/branch/jit-unroll-loops/pypy/module/_ffi/test/ (props changed)
pypy/branch/jit-unroll-loops/pypy/module/_rawffi/__init__.py
pypy/branch/jit-unroll-loops/pypy/module/_rawffi/array.py
pypy/branch/jit-unroll-loops/pypy/module/_rawffi/callback.py
pypy/branch/jit-unroll-loops/pypy/module/_rawffi/interp_rawffi.py
pypy/branch/jit-unroll-loops/pypy/module/_rawffi/structure.py
pypy/branch/jit-unroll-loops/pypy/module/_rawffi/test/test__rawffi.py
pypy/branch/jit-unroll-loops/pypy/module/_socket/test/test_sock_app.py
pypy/branch/jit-unroll-loops/pypy/module/_sre/interp_sre.py
pypy/branch/jit-unroll-loops/pypy/module/_weakref/interp__weakref.py
pypy/branch/jit-unroll-loops/pypy/module/_winreg/interp_winreg.py
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile (props changed)
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/intimg.c (props changed)
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/intimgtst.c (props changed)
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/intimgtst.py (props changed)
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/loop.c (props changed)
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/sum.c (props changed)
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/sumtst.c (props changed)
pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/sumtst.py (props changed)
pypy/branch/jit-unroll-loops/pypy/module/array/interp_array.py
pypy/branch/jit-unroll-loops/pypy/module/array/test/test_array_old.py (props changed)
pypy/branch/jit-unroll-loops/pypy/module/bz2/interp_bz2.py
pypy/branch/jit-unroll-loops/pypy/module/cpyext/classobject.py
pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_cpyext.py
pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_unicodeobject.py
pypy/branch/jit-unroll-loops/pypy/module/gc/interp_gc.py
pypy/branch/jit-unroll-loops/pypy/module/gc/referents.py
pypy/branch/jit-unroll-loops/pypy/module/gc/test/test_gc.py
pypy/branch/jit-unroll-loops/pypy/module/imp/importing.py
pypy/branch/jit-unroll-loops/pypy/module/imp/test/test_import.py
pypy/branch/jit-unroll-loops/pypy/module/parser/test/test_parser.py
pypy/branch/jit-unroll-loops/pypy/module/pypyjit/__init__.py
pypy/branch/jit-unroll-loops/pypy/module/pypyjit/interp_jit.py
pypy/branch/jit-unroll-loops/pypy/module/pypyjit/policy.py
pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py
pypy/branch/jit-unroll-loops/pypy/module/signal/interp_signal.py
pypy/branch/jit-unroll-loops/pypy/module/sys/__init__.py
pypy/branch/jit-unroll-loops/pypy/module/sys/state.py
pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c
pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py
pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py
pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py
pypy/branch/jit-unroll-loops/pypy/module/thread/ll_thread.py
pypy/branch/jit-unroll-loops/pypy/objspace/descroperation.py
pypy/branch/jit-unroll-loops/pypy/objspace/flow/flowcontext.py
pypy/branch/jit-unroll-loops/pypy/objspace/flow/test/test_objspace.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/callmethod.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/celldict.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/dictmultiobject.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/dicttype.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/fake.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/frozensettype.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/inttype.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/listobject.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/longtype.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/model.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/objspace.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/proxyobject.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/setobject.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/settype.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/smallintobject.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/stringobject.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/stringtype.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/strutil.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_dictmultiobject.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_longobject.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_setobject.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_strutil.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_typeobject.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_versionedtype.py
pypy/branch/jit-unroll-loops/pypy/objspace/std/typeobject.py
pypy/branch/jit-unroll-loops/pypy/objspace/test/test_descroperation.py
pypy/branch/jit-unroll-loops/pypy/rlib/_rsocket_rffi.py
pypy/branch/jit-unroll-loops/pypy/rlib/jit.py
pypy/branch/jit-unroll-loops/pypy/rlib/objectmodel.py
pypy/branch/jit-unroll-loops/pypy/rlib/rbigint.py
pypy/branch/jit-unroll-loops/pypy/rlib/rgc.py
pypy/branch/jit-unroll-loops/pypy/rlib/rmmap.py
pypy/branch/jit-unroll-loops/pypy/rlib/rsre/rsre_char.py
pypy/branch/jit-unroll-loops/pypy/rlib/rsre/rsre_core.py
pypy/branch/jit-unroll-loops/pypy/rlib/rsre/test/test_match.py
pypy/branch/jit-unroll-loops/pypy/rlib/rstring.py
pypy/branch/jit-unroll-loops/pypy/rlib/test/test_jit.py
pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rbigint.py
pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rdynload.py
pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rsocket.py
pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rstring.py
pypy/branch/jit-unroll-loops/pypy/rlib/test/test_runicode.py
pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rzlib.py
pypy/branch/jit-unroll-loops/pypy/rpython/llinterp.py
pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/ll2ctypes.py
pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llarena.py
pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llmemory.py
pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lloperation.py
pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lltype.py
pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rbuilder.py
pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rclass.py
pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rdict.py
pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rffi.py
pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_ll2ctypes.py
pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_llmemory.py
pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_lltype.py
pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_rffi.py
pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/base.py
pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/generation.py
pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/markcompact.py
pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/marksweep.py
pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/minimark.py
pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/minimarkpage.py
pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/semispace.py
pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/test/test_direct.py
pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/asmgcroot.py
pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/framework.py
pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/transform.py
pypy/branch/jit-unroll-loops/pypy/rpython/memory/gcwrapper.py
pypy/branch/jit-unroll-loops/pypy/rpython/memory/support.py
pypy/branch/jit-unroll-loops/pypy/rpython/memory/test/test_gc.py
pypy/branch/jit-unroll-loops/pypy/rpython/memory/test/test_transformed_gc.py
pypy/branch/jit-unroll-loops/pypy/rpython/module/ll_time.py
pypy/branch/jit-unroll-loops/pypy/rpython/rbuilder.py
pypy/branch/jit-unroll-loops/pypy/rpython/rbuiltin.py
pypy/branch/jit-unroll-loops/pypy/rpython/rpbc.py
pypy/branch/jit-unroll-loops/pypy/rpython/rtyper.py
pypy/branch/jit-unroll-loops/pypy/rpython/test/test_llinterp.py
pypy/branch/jit-unroll-loops/pypy/rpython/test/test_nongc.py
pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rbuilder.py
pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rclass.py
pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rdict.py
pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rptr.py
pypy/branch/jit-unroll-loops/pypy/tool/release/ (props changed)
pypy/branch/jit-unroll-loops/pypy/tool/release/__init__.py (props changed)
pypy/branch/jit-unroll-loops/pypy/tool/release/force-builds.py (props changed)
pypy/branch/jit-unroll-loops/pypy/tool/release/make_release.py (props changed)
pypy/branch/jit-unroll-loops/pypy/tool/release/test/ (props changed)
pypy/branch/jit-unroll-loops/pypy/tool/release/test/__init__.py (props changed)
pypy/branch/jit-unroll-loops/pypy/tool/release/test/test_make_release.py (props changed)
pypy/branch/jit-unroll-loops/pypy/translator/c/funcgen.py
pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/instruction.py
pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/test/elf/track5.s
pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/test/test_trackgcroot.py
pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/trackgcroot.py
pypy/branch/jit-unroll-loops/pypy/translator/c/src/g_include.h
pypy/branch/jit-unroll-loops/pypy/translator/c/src/main.h
pypy/branch/jit-unroll-loops/pypy/translator/c/src/signals.h
pypy/branch/jit-unroll-loops/pypy/translator/c/src/stack.h
pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_genc.py
pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_lltyped.py
pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_newgc.py
pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_standalone.py
pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_typed.py
pypy/branch/jit-unroll-loops/pypy/translator/driver.py
pypy/branch/jit-unroll-loops/pypy/translator/goal/ann_override.py
pypy/branch/jit-unroll-loops/pypy/translator/goal/app_main.py
pypy/branch/jit-unroll-loops/pypy/translator/goal/translate.py
pypy/branch/jit-unroll-loops/pypy/translator/jvm/test/test_class.py
pypy/branch/jit-unroll-loops/pypy/translator/oosupport/test_template/class_.py
pypy/branch/jit-unroll-loops/pypy/translator/platform/darwin.py
pypy/branch/jit-unroll-loops/pypy/translator/transform.py
Log:
svn merge -r77505:HEAD svn+ssh://hakanardo at codespeak.net/svn/pypy/trunk
Modified: pypy/branch/jit-unroll-loops/dotviewer/drawgraph.py
==============================================================================
--- pypy/branch/jit-unroll-loops/dotviewer/drawgraph.py (original)
+++ pypy/branch/jit-unroll-loops/dotviewer/drawgraph.py Sat Oct 30 16:41:17 2010
@@ -423,20 +423,43 @@
else:
for line in lines:
raw_line = line.replace('\\l','').replace('\r','') or ' '
- img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
- w, h = img.get_size()
- if w>wmax: wmax = w
- if raw_line.strip():
- if line.endswith('\\l'):
- def cmd(img=img, y=hmax):
- img.draw(xleft, ytop+y)
- elif line.endswith('\r'):
- def cmd(img=img, y=hmax, w=w):
- img.draw(xright-w, ytop+y)
- else:
- def cmd(img=img, y=hmax, w=w):
- img.draw(xcenter-w//2, ytop+y)
+ if '\f' in raw_line: # grayed out parts of the line
+ imgs = []
+ graytext = True
+ h = 16
+ w_total = 0
+ for linepart in raw_line.split('\f'):
+ graytext = not graytext
+ if not linepart.strip():
+ continue
+ if graytext:
+ fgcolor = (128, 160, 160)
+ else:
+ fgcolor = (0, 0, 0)
+ img = TextSnippet(self, linepart, fgcolor, bgcolor)
+ imgs.append((w_total, img))
+ w, h = img.get_size()
+ w_total += w
+ if w_total > wmax: wmax = w_total
+ def cmd(imgs=imgs, y=hmax):
+ for x, img in imgs:
+ img.draw(xleft+x, ytop+y)
commands.append(cmd)
+ else:
+ img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor)
+ w, h = img.get_size()
+ if w>wmax: wmax = w
+ if raw_line.strip():
+ if line.endswith('\\l'):
+ def cmd(img=img, y=hmax):
+ img.draw(xleft, ytop+y)
+ elif line.endswith('\r'):
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xright-w, ytop+y)
+ else:
+ def cmd(img=img, y=hmax, w=w):
+ img.draw(xcenter-w//2, ytop+y)
+ commands.append(cmd)
hmax += h
#hmax += 8
Modified: pypy/branch/jit-unroll-loops/lib-python/conftest.py
==============================================================================
--- pypy/branch/jit-unroll-loops/lib-python/conftest.py (original)
+++ pypy/branch/jit-unroll-loops/lib-python/conftest.py Sat Oct 30 16:41:17 2010
@@ -144,7 +144,7 @@
RegrTest('test_binhex.py'),
RegrTest('test_binop.py', core=True),
- RegrTest('test_bisect.py', core=True),
+ RegrTest('test_bisect.py', core=True, usemodules='_bisect'),
RegrTest('test_bool.py', core=True),
RegrTest('test_bsddb.py', skip="unsupported extension module"),
RegrTest('test_bsddb185.py', skip="unsupported extension module"),
@@ -464,7 +464,12 @@
RegrTest('test_coding.py'),
RegrTest('test_complex_args.py'),
RegrTest('test_contextlib.py', usemodules="thread"),
- RegrTest('test_ctypes.py', usemodules="_rawffi"),
+ # we skip test ctypes, since we adapted it massively in order
+ # to test what we want to support. There are real failures,
+ # but it's about missing features that we don't want to support
+ # now
+ RegrTest('test_ctypes.py', usemodules="_rawffi",
+ skip="missing features that we don't want to support now"),
RegrTest('test_defaultdict.py'),
RegrTest('test_email_renamed.py'),
RegrTest('test_exception_variations.py'),
Modified: pypy/branch/jit-unroll-loops/lib-python/modified-2.5.2/test/test_descr.py
==============================================================================
--- pypy/branch/jit-unroll-loops/lib-python/modified-2.5.2/test/test_descr.py (original)
+++ pypy/branch/jit-unroll-loops/lib-python/modified-2.5.2/test/test_descr.py Sat Oct 30 16:41:17 2010
@@ -2028,7 +2028,9 @@
except TypeError, msg:
verify(str(msg).find("weak reference") >= 0)
else:
- verify(0, "weakref.ref(no) should be illegal")
+ # in PyPy it is (sometimes) possible to take a weakref here
+ #verify(0, "weakref.ref(no) should be illegal")
+ pass
class Weak(object):
__slots__ = ['foo', '__weakref__']
yes = Weak()
Modified: pypy/branch/jit-unroll-loops/lib_pypy/_ctypes/function.py
==============================================================================
--- pypy/branch/jit-unroll-loops/lib_pypy/_ctypes/function.py (original)
+++ pypy/branch/jit-unroll-loops/lib_pypy/_ctypes/function.py Sat Oct 30 16:41:17 2010
@@ -171,7 +171,7 @@
return self._build_result(restype, resbuffer, argtypes, argsandobjs)
def _getfuncptr(self, argtypes, restype, thisarg=None):
- if self._ptr is not None:
+ if self._ptr is not None and argtypes is self._argtypes_:
return self._ptr
if restype is None or not isinstance(restype, _CDataMeta):
import ctypes
Modified: pypy/branch/jit-unroll-loops/lib_pypy/_ctypes/structure.py
==============================================================================
--- pypy/branch/jit-unroll-loops/lib_pypy/_ctypes/structure.py (original)
+++ pypy/branch/jit-unroll-loops/lib_pypy/_ctypes/structure.py Sat Oct 30 16:41:17 2010
@@ -7,7 +7,7 @@
def round_up(size, alignment):
return (size + alignment - 1) & -alignment
-def size_alignment_pos(fields):
+def size_alignment_pos(fields, is_union=False):
import ctypes
size = 0
alignment = 1
@@ -15,14 +15,19 @@
for fieldname, ctype in fields:
fieldsize = ctypes.sizeof(ctype)
fieldalignment = ctypes.alignment(ctype)
- size = round_up(size, fieldalignment)
alignment = max(alignment, fieldalignment)
- pos.append(size)
- size += fieldsize
+ if is_union:
+ pos.append(0)
+ size = max(size, fieldsize)
+ else:
+ size = round_up(size, fieldalignment)
+ pos.append(size)
+ size += fieldsize
size = round_up(size, alignment)
return size, alignment, pos
-def names_and_fields(_fields_, superclass, zero_offset=False, anon=None):
+def names_and_fields(_fields_, superclass, zero_offset=False, anon=None,
+ is_union=False):
if isinstance(_fields_, tuple):
_fields_ = list(_fields_)
for _, tp in _fields_:
@@ -36,7 +41,7 @@
rawfields = [(name, ctype._ffishape)
for name, ctype in all_fields]
if not zero_offset:
- _, _, pos = size_alignment_pos(all_fields)
+ _, _, pos = size_alignment_pos(all_fields, is_union)
else:
pos = [0] * len(all_fields)
fields = {}
@@ -73,8 +78,8 @@
# ________________________________________________________________
-def _set_shape(tp, rawfields):
- tp._ffistruct = _rawffi.Structure(rawfields)
+def _set_shape(tp, rawfields, is_union=False):
+ tp._ffistruct = _rawffi.Structure(rawfields, is_union)
tp._ffiargshape = tp._ffishape = (tp._ffistruct, 1)
tp._fficompositesize = tp._ffistruct.size
@@ -92,13 +97,14 @@
raise AttributeError("Structure or union cannot contain itself")
self._names, rawfields, self._fieldtypes = names_and_fields(
value, self.__bases__[0], False,
- self.__dict__.get('_anonymous_', None))
+ self.__dict__.get('_anonymous_', None), self._is_union)
_CDataMeta.__setattr__(self, '_fields_', value)
- _set_shape(self, rawfields)
+ _set_shape(self, rawfields, self._is_union)
return
_CDataMeta.__setattr__(self, name, value)
-class StructureMeta(_CDataMeta):
+class StructOrUnionMeta(_CDataMeta):
+
def __new__(self, name, cls, typedict):
res = type.__new__(self, name, cls, typedict)
if '_fields_' in typedict:
@@ -109,8 +115,8 @@
raise AttributeError("Anonymous field not found")
res._names, rawfields, res._fieldtypes = names_and_fields(
typedict['_fields_'], cls[0], False,
- typedict.get('_anonymous_', None))
- _set_shape(res, rawfields)
+ typedict.get('_anonymous_', None), self._is_union)
+ _set_shape(res, rawfields, self._is_union)
return res
@@ -150,8 +156,8 @@
res.__dict__['_index'] = -1
return res
-class Structure(_CData):
- __metaclass__ = StructureMeta
+class StructOrUnion(_CData):
+ __metaclass__ = StructOrUnionMeta
def __new__(cls, *args, **kwds):
if not hasattr(cls, '_ffistruct'):
@@ -213,3 +219,10 @@
def _get_buffer_value(self):
return self._buffer.buffer
+
+
+class StructureMeta(StructOrUnionMeta):
+ _is_union = False
+
+class Structure(StructOrUnion):
+ __metaclass__ = StructureMeta
Modified: pypy/branch/jit-unroll-loops/lib_pypy/_ctypes/union.py
==============================================================================
--- pypy/branch/jit-unroll-loops/lib_pypy/_ctypes/union.py (original)
+++ pypy/branch/jit-unroll-loops/lib_pypy/_ctypes/union.py Sat Oct 30 16:41:17 2010
@@ -1,118 +1,7 @@
+from _ctypes import structure
+class UnionMeta(structure.StructOrUnionMeta):
+ _is_union = True
-import _rawffi
-from _ctypes.basics import _CData, _CDataMeta, store_reference, keepalive_key
-from _ctypes.basics import ensure_objects
-from _ctypes.structure import round_up, names_and_fields, struct_getattr,\
- struct_setattr
-
-
-def _set_shape(tp):
- size = tp._sizeofinstances()
- alignment = tp._alignmentofinstances()
- tp._ffiopaque = _rawffi.Structure((size, alignment)) # opaque
- tp._ffiargshape = tp._ffishape = (tp._ffiopaque, 1)
- tp._fficompositesize = tp._ffiopaque.size
- # we need to create an array of size one for each
- # of our elements
- tp._ffiarrays = {}
- for name, field in tp._fieldtypes.iteritems():
- tp._ffiarrays[name] = _rawffi.Array(field.ctype._ffishape)
-
-class UnionMeta(_CDataMeta):
- def __new__(self, name, cls, typedict):
- res = type.__new__(self, name, cls, typedict)
- if '_fields_' in typedict:
- res._names, rawfields, res._fieldtypes = names_and_fields(
- typedict['_fields_'], cls[0], True,
- typedict.get('_anonymous_', None))
- _set_shape(res)
-
- def __init__(self): # don't allow arguments by now
- if not hasattr(self, '_ffiarrays'):
- raise TypeError("Cannot instantiate union, has no type")
- # malloc size
- size = self.__class__._sizeofinstances()
- self.__dict__['_objects'] = {}
- self.__dict__['_buffer'] = self._ffiopaque(autofree=True)
- res.__init__ = __init__
- return res
-
- def _sizeofinstances(self):
- if not hasattr(self, '_size_'):
- self._size_ = max([field.size for field in
- self._fieldtypes.values()] + [0])
- return self._size_
-
- def _alignmentofinstances(self):
- from ctypes import alignment
- if not hasattr(self, '_alignment_'):
- self._alignment_ = max([alignment(field.ctype) for field in
- self._fieldtypes.values()] + [1])
- return self._alignment_
-
- __getattr__ = struct_getattr
-
- def __setattr__(self, name, value):
- if name == '_fields_':
- if self.__dict__.get('_fields_', None):
- raise AttributeError("_fields_ is final")
- if self in [v for k, v in value]:
- raise AttributeError("Union cannot contain itself")
- self._names, rawfields, self._fieldtypes = names_and_fields(
- value, self.__bases__[0], True,
- self.__dict__.get('_anonymous_', None))
- _CDataMeta.__setattr__(self, '_fields_', value)
- _set_shape(self)
- _CDataMeta.__setattr__(self, name, value)
-
- def _CData_output(self, resarray, base=None, index=-1):
- res = self.__new__(self)
- ffiopaque = self._ffiopaque.fromaddress(resarray.buffer)
- res.__dict__['_buffer'] = ffiopaque
- res.__dict__['_base'] = base
- res.__dict__['_index'] = index
- return res
-
- def _CData_retval(self, resbuffer):
- res = self.__new__(self)
- res.__dict__['_buffer'] = resbuffer
- res.__dict__['_base'] = None
- res.__dict__['_index'] = -1
- return res
-
-
-class Union(_CData):
+class Union(structure.StructOrUnion):
__metaclass__ = UnionMeta
-
- def __getattr__(self, name):
- try:
- field = self._fieldtypes[name]
- except KeyError:
- raise AttributeError(name)
- fieldtype = field.ctype
- val = self._ffiarrays[name].fromaddress(self._buffer.buffer, 1)
- offset = field.num
- return fieldtype._CData_output(val, self, offset)
-
- def __setattr__(self, name, value):
- try:
- field = self._fieldtypes[name]
- except KeyError:
- raise AttributeError(name)
- fieldtype = field.ctype
- cobj = fieldtype.from_param(value)
- if ensure_objects(cobj) is not None:
- key = keepalive_key(field.num)
- store_reference(self, key, cobj._objects)
- arg = cobj._get_buffer_value()
- if fieldtype._fficompositesize is not None:
- from ctypes import memmove
- dest = self._buffer.buffer
- memmove(dest, arg, fieldtype._fficompositesize)
- else:
- buf = self._ffiarrays[name].fromaddress(self._buffer.buffer, 1)
- buf[0] = arg
-
- def _get_buffer_value(self):
- return self._buffer.buffer
Modified: pypy/branch/jit-unroll-loops/pypy/annotation/bookkeeper.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/annotation/bookkeeper.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/annotation/bookkeeper.py Sat Oct 30 16:41:17 2010
@@ -24,7 +24,7 @@
from pypy.rpython import extregistry
from pypy.tool.identity_dict import identity_dict
-class Stats:
+class Stats(object):
def __init__(self, bookkeeper):
self.bookkeeper = bookkeeper
@@ -137,7 +137,7 @@
def consider_dict_delitem(self, dic):
return dic
-class Bookkeeper:
+class Bookkeeper(object):
"""The log of choices that have been made while analysing the operations.
It ensures that the same 'choice objects' will be returned if we ask
again during reflowing. Like ExecutionContext, there is an implicit
@@ -736,7 +736,7 @@
return True
# for parsing call arguments
-class RPythonCallsSpace:
+class RPythonCallsSpace(object):
"""Pseudo Object Space providing almost no real operation.
For the Arguments class: if it really needs other operations, it means
that the call pattern is too complex for R-Python.
Modified: pypy/branch/jit-unroll-loops/pypy/annotation/builtin.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/annotation/builtin.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/annotation/builtin.py Sat Oct 30 16:41:17 2010
@@ -423,7 +423,7 @@
from pypy.annotation.model import SomePtr
from pypy.rpython.lltypesystem import lltype
-def malloc(s_T, s_n=None, s_flavor=None, s_zero=None):
+def malloc(s_T, s_n=None, s_flavor=None, s_zero=None, s_track_allocation=None):
assert (s_n is None or s_n.knowntype == int
or issubclass(s_n.knowntype, pypy.rlib.rarithmetic.base_int))
assert s_T.is_constant()
@@ -438,13 +438,15 @@
r = SomePtr(lltype.typeOf(p))
else:
assert s_flavor.is_constant()
+ assert s_track_allocation is None or s_track_allocation.is_constant()
# not sure how to call malloc() for the example 'p' in the
# presence of s_extraargs
r = SomePtr(lltype.Ptr(s_T.const))
return r
-def free(s_p, s_flavor):
+def free(s_p, s_flavor, s_track_allocation=None):
assert s_flavor.is_constant()
+ assert s_track_allocation is None or s_track_allocation.is_constant()
# same problem as in malloc(): some flavors are not easy to
# malloc-by-example
#T = s_p.ll_ptrtype.TO
Modified: pypy/branch/jit-unroll-loops/pypy/annotation/classdef.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/annotation/classdef.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/annotation/classdef.py Sat Oct 30 16:41:17 2010
@@ -58,7 +58,7 @@
# same name in all subclasses of A, if any. (Parent class attributes can
# be visible in reads from instances of subclasses.)
-class Attribute:
+class Attribute(object):
# readonly-ness
# SomeThing-ness
# NB. an attribute is readonly if it is a constant class attribute.
@@ -402,7 +402,7 @@
# ____________________________________________________________
-class InstanceSource:
+class InstanceSource(object):
instance_level = True
def __init__(self, bookkeeper, obj):
Modified: pypy/branch/jit-unroll-loops/pypy/annotation/description.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/annotation/description.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/annotation/description.py Sat Oct 30 16:41:17 2010
@@ -6,7 +6,7 @@
from pypy.tool.sourcetools import valid_identifier
from pypy.tool.pairtype import extendabletype
-class CallFamily:
+class CallFamily(object):
"""A family of Desc objects that could be called from common call sites.
The call families are conceptually a partition of all (callable) Desc
objects, where the equivalence relation is the transitive closure of
@@ -51,7 +51,7 @@
self.total_calltable_size += 1
-class FrozenAttrFamily:
+class FrozenAttrFamily(object):
"""A family of FrozenDesc objects that have any common 'getattr' sites.
The attr families are conceptually a partition of FrozenDesc objects,
where the equivalence relation is the transitive closure of:
@@ -80,7 +80,7 @@
self.attrs[attrname] = s_value
-class ClassAttrFamily:
+class ClassAttrFamily(object):
"""A family of ClassDesc objects that have common 'getattr' sites for a
given attribute name. The attr families are conceptually a partition
of ClassDesc objects, where the equivalence relation is the transitive
Modified: pypy/branch/jit-unroll-loops/pypy/annotation/dictdef.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/annotation/dictdef.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/annotation/dictdef.py Sat Oct 30 16:41:17 2010
@@ -77,7 +77,7 @@
dictdef.dictvalue = self
-class DictDef:
+class DictDef(object):
"""A dict definition remembers how general the keys and values in that
particular dict have to be. Every dict creation makes a new DictDef,
and the union of two dicts merges the DictKeys and DictValues that each
Modified: pypy/branch/jit-unroll-loops/pypy/annotation/listdef.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/annotation/listdef.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/annotation/listdef.py Sat Oct 30 16:41:17 2010
@@ -6,7 +6,7 @@
class TooLateForChange(Exception):
pass
-class ListItem:
+class ListItem(object):
mutated = False # True for lists mutated after creation
resized = False # True for lists resized after creation
range_step = None # the step -- only for lists only created by a range()
@@ -117,7 +117,7 @@
return updated
-class ListDef:
+class ListDef(object):
"""A list definition remembers how general the items in that particular
list have to be. Every list creation makes a new ListDef, and the union
of two lists merges the ListItems that each ListDef stores."""
Modified: pypy/branch/jit-unroll-loops/pypy/annotation/model.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/annotation/model.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/annotation/model.py Sat Oct 30 16:41:17 2010
@@ -574,11 +574,11 @@
NUMBER = object()
annotation_to_ll_map = [
+ (SomeSingleFloat(), lltype.SingleFloat),
(s_None, lltype.Void), # also matches SomeImpossibleValue()
(s_Bool, lltype.Bool),
(SomeInteger(knowntype=r_ulonglong), NUMBER),
(SomeFloat(), lltype.Float),
- (SomeSingleFloat(), lltype.SingleFloat),
(SomeChar(), lltype.Char),
(SomeUnicodeCodePoint(), lltype.UniChar),
(SomeAddress(), llmemory.Address),
Modified: pypy/branch/jit-unroll-loops/pypy/annotation/policy.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/annotation/policy.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/annotation/policy.py Sat Oct 30 16:41:17 2010
@@ -1,4 +1,4 @@
-# base annotation policy for overrides and specialization
+# base annotation policy for specialization
from pypy.annotation.specialize import default_specialize as default
from pypy.annotation.specialize import specialize_argvalue, specialize_argtype, specialize_arglistitemtype
from pypy.annotation.specialize import memo
@@ -41,7 +41,7 @@
if directive is None:
return pol.default_specialize
- # specialize|override:name[(args)]
+ # specialize[(args)]
directive_parts = directive.split('(', 1)
if len(directive_parts) == 1:
[name] = directive_parts
@@ -60,14 +60,6 @@
except AttributeError:
raise AttributeError("%r specialize tag not defined in annotation"
"policy %s" % (name, pol))
- if directive.startswith('override:'):
- # different signature: override__xyz(*args_s)
- if parms:
- raise Exception, "override:* specialisations don't support parameters"
- def specialize_override(funcdesc, args_s):
- funcdesc.overridden = True
- return specializer(*args_s)
- return specialize_override
else:
if not parms:
return specializer
@@ -92,9 +84,5 @@
from pypy.rpython.annlowlevel import LowLevelAnnotatorPolicy
return LowLevelAnnotatorPolicy.specialize__ll_and_arg(*args)
- def override__ignore(pol, *args):
- bk = getbookkeeper()
- return bk.immutablevalue(None)
-
class StrictAnnotatorPolicy(AnnotatorPolicy):
allow_someobjects = False
Modified: pypy/branch/jit-unroll-loops/pypy/annotation/specialize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/annotation/specialize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/annotation/specialize.py Sat Oct 30 16:41:17 2010
@@ -100,7 +100,7 @@
# ____________________________________________________________________________
# specializations
-class MemoTable:
+class MemoTable(object):
def __init__(self, funcdesc, args, value):
self.funcdesc = funcdesc
self.table = {args: value}
Modified: pypy/branch/jit-unroll-loops/pypy/annotation/test/test_annrpython.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/annotation/test/test_annrpython.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/annotation/test/test_annrpython.py Sat Oct 30 16:41:17 2010
@@ -1,4 +1,4 @@
-
+from __future__ import with_statement
import autopath
import py.test
import sys
@@ -766,28 +766,6 @@
s = a.build_types(f, [list])
assert s.classdef is a.bookkeeper.getuniqueclassdef(IndexError) # KeyError ignored because l is a list
- def test_overrides(self):
- excs = []
- def record_exc(e):
- """NOT_RPYTHON"""
- excs.append(sys.exc_info)
- record_exc._annspecialcase_ = "override:record_exc"
- def g():
- pass
- def f():
- try:
- g()
- except Exception, e:
- record_exc(e)
- class MyAnnotatorPolicy(policy.AnnotatorPolicy):
-
- def override__record_exc(pol, s_e):
- return a.bookkeeper.immutablevalue(None)
-
- a = self.RPythonAnnotator(policy=MyAnnotatorPolicy())
- s = a.build_types(f, [])
- assert s.const is None
-
def test_freeze_protocol(self):
class Stuff:
def __init__(self, flag):
@@ -3359,6 +3337,26 @@
s = a.build_types(f, [])
assert isinstance(s, annmodel.SomeChar)
+ def test_context_manager(self):
+ class C:
+ def __init__(self):
+ pass
+ def __enter__(self):
+ self.x = 1
+ def __exit__(self, *args):
+ self.x = 3
+ def f():
+ c = C()
+ with c:
+ pass
+ return c.x
+
+ a = self.RPythonAnnotator()
+ s = a.build_types(f, [])
+ assert isinstance(s, annmodel.SomeInteger)
+ # not a constant: both __enter__ and __exit__ have been annotated
+ assert not s.is_constant()
+
def g(n):
return [0,1,2,n]
Modified: pypy/branch/jit-unroll-loops/pypy/annotation/test/test_model.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/annotation/test/test_model.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/annotation/test/test_model.py Sat Oct 30 16:41:17 2010
@@ -128,7 +128,7 @@
assert isinstance(s_p, SomeOOInstance) and s_p.ootype == C
def test_annotation_to_lltype():
- from pypy.rlib.rarithmetic import r_uint
+ from pypy.rlib.rarithmetic import r_uint, r_singlefloat
s_i = SomeInteger()
s_pos = SomeInteger(nonneg=True)
s_1 = SomeInteger(nonneg=True); s_1.const = 1
@@ -151,6 +151,9 @@
C = ootype.Instance('C', ROOT, {})
ref = SomeOOInstance(C)
assert annotation_to_lltype(ref) == C
+ s_singlefloat = SomeSingleFloat()
+ s_singlefloat.const = r_singlefloat(0.0)
+ assert annotation_to_lltype(s_singlefloat) == lltype.SingleFloat
def test_ll_union():
PS1 = lltype.Ptr(lltype.GcStruct('s'))
Modified: pypy/branch/jit-unroll-loops/pypy/annotation/unaryop.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/annotation/unaryop.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/annotation/unaryop.py Sat Oct 30 16:41:17 2010
@@ -434,6 +434,9 @@
def method_clear(dct):
pass
+ def method_popitem(dct):
+ return dct.getanyitem('items')
+
def _can_only_throw(dic, *ignore):
if dic1.dictdef.dictkey.custom_eq_hash:
return None # r_dict: can throw anything
Modified: pypy/branch/jit-unroll-loops/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/config/pypyoption.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/config/pypyoption.py Sat Oct 30 16:41:17 2010
@@ -30,7 +30,8 @@
"rctime" , "select", "zipimport", "_lsprof",
"crypt", "signal", "_rawffi", "termios", "zlib",
"struct", "md5", "sha", "bz2", "_minimal_curses", "cStringIO",
- "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array"]
+ "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array",
+ "_bisect"]
))
working_oo_modules = default_modules.copy()
@@ -73,9 +74,10 @@
}
module_import_dependencies = {
- # no _rawffi if importing pypy.rlib.libffi raises ImportError
+ # no _rawffi if importing pypy.rlib.clibffi raises ImportError
# or CompilationError
- "_rawffi" : ["pypy.rlib.libffi"],
+ "_rawffi" : ["pypy.rlib.clibffi"],
+ "_ffi" : ["pypy.rlib.clibffi"],
"zlib" : ["pypy.rlib.rzlib"],
"bz2" : ["pypy.module.bz2.interp_bz2"],
@@ -198,6 +200,9 @@
BoolOption("withstrslice", "use strings optimized for slicing",
default=False),
+ BoolOption("withstrbuf", "use strings optimized for addition (ver 2)",
+ default=False),
+
BoolOption("withprebuiltchar",
"use prebuilt single-character string objects",
default=False),
@@ -210,7 +215,8 @@
BoolOption("withrope", "use ropes as the string implementation",
default=False,
requires=[("objspace.std.withstrslice", False),
- ("objspace.std.withstrjoin", False)],
+ ("objspace.std.withstrjoin", False),
+ ("objspace.std.withstrbuf", False)],
suggests=[("objspace.std.withprebuiltchar", True),
("objspace.std.sharesmallstr", True)]),
@@ -224,19 +230,17 @@
requires=[("objspace.opcodes.CALL_LIKELY_BUILTIN", False),
("objspace.honor__builtins__", False)]),
- BoolOption("withsharingdict",
- "use dictionaries that share the keys part",
- default=False),
-
BoolOption("withdictmeasurement",
"create huge files with masses of information "
"about dictionaries",
default=False),
- BoolOption("withinlineddict",
- "make instances more compact by revoming a level of indirection",
+ BoolOption("withmapdict",
+ "make instances really small but slow without the JIT",
default=False,
- requires=[("objspace.std.withshadowtracking", False)]),
+ requires=[("objspace.std.getattributeshortcut", True),
+ ("objspace.std.withtypeversion", True),
+ ]),
BoolOption("withrangelist",
"enable special range list implementation that does not "
@@ -251,12 +255,6 @@
# weakrefs needed, because of get_subclasses()
requires=[("translation.rweakref", True)]),
- BoolOption("withshadowtracking",
- "track whether an instance attribute shadows a type"
- " attribute",
- default=False,
- requires=[("objspace.std.withtypeversion", True),
- ("translation.rweakref", True)]),
BoolOption("withmethodcache",
"try to cache method lookups",
default=False,
@@ -328,9 +326,6 @@
config.objspace.std.suggest(optimized_list_getitem=True)
config.objspace.std.suggest(getattributeshortcut=True)
config.objspace.std.suggest(newshortcut=True)
- if type_system != 'ootype':
- config.objspace.std.suggest(withsharingdict=True)
- config.objspace.std.suggest(withinlineddict=True)
# extra costly optimizations only go in level 3
if level == '3':
@@ -343,7 +338,7 @@
config.objspace.std.suggest(withprebuiltint=True)
config.objspace.std.suggest(withrangelist=True)
config.objspace.std.suggest(withprebuiltchar=True)
- config.objspace.std.suggest(withinlineddict=True)
+ config.objspace.std.suggest(withmapdict=True)
config.objspace.std.suggest(withstrslice=True)
config.objspace.std.suggest(withstrjoin=True)
# xxx other options? ropes maybe?
@@ -359,6 +354,7 @@
# extra optimizations with the JIT
if level == 'jit':
config.objspace.std.suggest(withcelldict=True)
+ config.objspace.std.suggest(withmapdict=True)
def enable_allworkingmodules(config):
Modified: pypy/branch/jit-unroll-loops/pypy/config/test/test_pypyoption.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/config/test/test_pypyoption.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/config/test/test_pypyoption.py Sat Oct 30 16:41:17 2010
@@ -47,7 +47,7 @@
def test_set_pypy_opt_level():
conf = get_pypy_config()
set_pypy_opt_level(conf, '2')
- assert conf.objspace.std.withsharingdict
+ assert conf.objspace.std.newshortcut
conf = get_pypy_config()
set_pypy_opt_level(conf, '0')
assert not conf.objspace.std.newshortcut
@@ -59,7 +59,6 @@
assert not conf.objspace.std.withtypeversion
assert not conf.objspace.std.withmethodcache
- assert not conf.objspace.std.withshadowtracking
def test_check_documentation():
def check_file_exists(fn):
Modified: pypy/branch/jit-unroll-loops/pypy/conftest.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/conftest.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/conftest.py Sat Oct 30 16:41:17 2010
@@ -7,6 +7,7 @@
from inspect import isclass, getmro
from pypy.tool.udir import udir
from pypy.tool.autopath import pypydir
+from pypy.tool import leakfinder
# pytest settings
pytest_plugins = "resultlog",
@@ -354,7 +355,14 @@
def runtest(self):
try:
- super(IntTestFunction, self).runtest()
+ leakfinder.start_tracking_allocations()
+ try:
+ super(IntTestFunction, self).runtest()
+ finally:
+ if leakfinder.TRACK_ALLOCATIONS:
+ leaks = leakfinder.stop_tracking_allocations(False)
+ else:
+ leaks = None # stop_tracking_allocations() already called
except OperationError, e:
check_keyboard_interrupt(e)
raise
@@ -373,6 +381,8 @@
_pygame_imported = True
assert option.view, ("should not invoke Pygame "
"if conftest.option.view is False")
+ if leaks: # check for leaks, but only if the test passed so far
+ raise leakfinder.MallocMismatch(leaks)
class AppTestFunction(PyPyTestFunction):
def _prunetraceback(self, traceback):
Modified: pypy/branch/jit-unroll-loops/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt (original)
+++ pypy/branch/jit-unroll-loops/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt Sat Oct 30 16:41:17 2010
@@ -5,8 +5,6 @@
case. So far, this only works for calls with no keyword, no ``*arg``
and no ``**arg`` but it would be easy to extend.
-Gives the best results combined with :config:`objspace.std.withshadowtracking`.
-
For more information, see the section in `Standard Interpreter Optimizations`_.
.. _`Standard Interpreter Optimizations`: ../interpreter-optimizations.html#lookup-method-call-method
Modified: pypy/branch/jit-unroll-loops/pypy/doc/docindex.txt
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/doc/docindex.txt (original)
+++ pypy/branch/jit-unroll-loops/pypy/doc/docindex.txt Sat Oct 30 16:41:17 2010
@@ -84,7 +84,7 @@
PyPy's own tests `summary`_, daily updated, run through BuildBot infrastructure.
You can also find CPython's compliance tests run with compiled ``pypy-c``
-exeuctables there.
+executables there.
information dating from early 2007:
Modified: pypy/branch/jit-unroll-loops/pypy/doc/interpreter-optimizations.txt
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/doc/interpreter-optimizations.txt (original)
+++ pypy/branch/jit-unroll-loops/pypy/doc/interpreter-optimizations.txt Sat Oct 30 16:41:17 2010
@@ -153,8 +153,8 @@
dicts:
the representation of the instance dict contains only a list of values.
-You can enable this feature with the :config:`objspace.std.withsharingdict`
-option.
+A more advanced version of sharing dicts, called *map dicts,* is available
+with the :config:`objspace.std.withmapdict` option.
Builtin-Shadowing
+++++++++++++++++
@@ -219,8 +219,7 @@
shadowing the class attribute. If we know that there is no shadowing (since
instance dict tells us that) we can save this lookup on the instance dictionary.
-You can enable this feature with the :config:`objspace.std.withshadowtracking`
-option.
+*This was deprecated and is no longer available.*
Method Caching
Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/astcompiler/test/test_compiler.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/interpreter/astcompiler/test/test_compiler.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/interpreter/astcompiler/test/test_compiler.py Sat Oct 30 16:41:17 2010
@@ -41,7 +41,7 @@
co_expr = compile(evalexpr, '', 'eval')
space = self.space
pyco_expr = PyCode._from_code(space, co_expr)
- w_res = pyco_expr.exec_code(space, w_dict, w_dict)
+ w_res = pyco_expr.exec_host_bytecode(space, w_dict, w_dict)
res = space.str_w(space.repr(w_res))
if not isinstance(expected, float):
assert res == repr(expected)
Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/baseobjspace.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/interpreter/baseobjspace.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/interpreter/baseobjspace.py Sat Oct 30 16:41:17 2010
@@ -36,13 +36,10 @@
return space.finditem_str(w_dict, attr)
return None
- def getdictvalue_attr_is_in_class(self, space, attr):
- return self.getdictvalue(space, attr)
-
- def setdictvalue(self, space, attr, w_value, shadows_type=True):
+ def setdictvalue(self, space, attr, w_value):
w_dict = self.getdict()
if w_dict is not None:
- space.setitem_str(w_dict, attr, w_value, shadows_type)
+ space.setitem_str(w_dict, attr, w_value)
return True
return False
@@ -168,6 +165,20 @@
def _call_builtin_destructor(self):
pass # method overridden in typedef.py
+ # hooks that the mapdict implementations needs:
+ def _get_mapdict_map(self):
+ return None
+ def _set_mapdict_map(self, map):
+ raise NotImplementedError
+ def _mapdict_read_storage(self, index):
+ raise NotImplementedError
+ def _mapdict_write_storage(self, index, value):
+ raise NotImplementedError
+ def _mapdict_storage_length(self):
+ raise NotImplementedError
+ def _set_mapdict_storage_and_map(self, storage, map):
+ raise NotImplementedError
+
class Wrappable(W_Root):
"""A subclass of Wrappable is an internal, interpreter-level class
@@ -643,7 +654,7 @@
"""shortcut for space.int_w(space.hash(w_obj))"""
return self.int_w(self.hash(w_obj))
- def setitem_str(self, w_obj, key, w_value, shadows_type=True):
+ def setitem_str(self, w_obj, key, w_value):
return self.setitem(w_obj, self.wrap(key), w_value)
def finditem_str(self, w_obj, key):
@@ -725,7 +736,7 @@
def unpackiterable(self, w_iterable, expected_length=-1):
"""Unpack an iterable object into a real (interpreter-level) list.
- Raise a real (subclass of) ValueError if the length is wrong."""
+ Raise an OperationError(w_ValueError) if the length is wrong."""
w_iterator = self.iter(w_iterable)
items = []
while True:
Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/function.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/interpreter/function.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/interpreter/function.py Sat Oct 30 16:41:17 2010
@@ -430,8 +430,11 @@
self.w_class = w_class # possibly space.w_None
def descr_method__new__(space, w_subtype, w_function, w_instance, w_class=None):
- if space.is_w( w_instance, space.w_None ):
+ if space.is_w(w_instance, space.w_None):
w_instance = None
+ if w_instance is None and space.is_w(w_class, space.w_None):
+ raise OperationError(space.w_TypeError,
+ space.wrap("unbound methods must have class"))
method = space.allocate_instance(Method, w_subtype)
Method.__init__(method, space, w_function, w_instance, w_class)
return space.wrap(method)
@@ -586,12 +589,14 @@
w_klass = space.type(w_obj)
return space.wrap(Method(space, self.w_function, w_klass, space.w_None))
- def descr_classmethod__new__(space, w_type, w_function):
+ def descr_classmethod__new__(space, w_subtype, w_function):
if not space.is_true(space.callable(w_function)):
typename = space.type(w_function).getname(space, '?')
raise operationerrfmt(space.w_TypeError,
"'%s' object is not callable", typename)
- return space.wrap(ClassMethod(w_function))
+ instance = space.allocate_instance(ClassMethod, w_subtype)
+ instance.__init__(w_function)
+ return space.wrap(instance)
class FunctionWithFixedCode(Function):
can_change_code = False
Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/gateway.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/interpreter/gateway.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/interpreter/gateway.py Sat Oct 30 16:41:17 2010
@@ -28,7 +28,7 @@
# internal non-translatable parts:
import py
-class SignatureBuilder:
+class SignatureBuilder(object):
"NOT_RPYTHON"
def __init__(self, func=None, argnames=None, varargname=None,
kwargname=None, name = None):
@@ -51,7 +51,7 @@
#________________________________________________________________
-class UnwrapSpecRecipe:
+class UnwrapSpecRecipe(object):
"NOT_RPYTHON"
bases_order = [Wrappable, W_Root, ObjSpace, Arguments, object]
Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/pycode.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/interpreter/pycode.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/interpreter/pycode.py Sat Oct 30 16:41:17 2010
@@ -117,6 +117,10 @@
self._compute_flatcall()
+ if self.space.config.objspace.std.withmapdict:
+ from pypy.objspace.std.mapdict import init_mapdict_cache
+ init_mapdict_cache(self)
+
def _freeze_(self):
if (self.magic == cpython_magic and
'__pypy__' not in sys.builtin_module_names):
@@ -253,6 +257,12 @@
tuple(self.co_freevars),
tuple(self.co_cellvars) )
+ def exec_host_bytecode(self, w_dict, w_globals, w_locals):
+ from pypy.interpreter.pyframe import CPythonFrame
+ frame = CPythonFrame(self.space, self, w_globals, None)
+ frame.setdictscope(w_locals)
+ return frame.run()
+
def dump(self):
"""A dis.dis() dump of the code object."""
co = self._to_code()
Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/pyframe.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/interpreter/pyframe.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/interpreter/pyframe.py Sat Oct 30 16:41:17 2010
@@ -13,6 +13,7 @@
from pypy.rlib.rarithmetic import intmask
from pypy.rlib import jit, rstack
from pypy.tool import stdlib_opcode
+from pypy.tool.stdlib_opcode import host_bytecode_spec
# Define some opcodes used
g = globals()
@@ -140,7 +141,8 @@
# the following 'assert' is an annotation hint: it hides from
# the annotator all methods that are defined in PyFrame but
# overridden in the {,Host}FrameClass subclasses of PyFrame.
- assert isinstance(self, self.space.FrameClass)
+ assert (isinstance(self, self.space.FrameClass) or
+ not self.space.config.translating)
executioncontext = self.space.getexecutioncontext()
executioncontext.enter(self)
try:
@@ -634,6 +636,18 @@
return space.wrap(self.builtin is not space.builtin)
return space.w_False
+class CPythonFrame(PyFrame):
+ """
+ Execution of host (CPython) opcodes.
+ """
+
+ bytecode_spec = host_bytecode_spec
+ opcode_method_names = host_bytecode_spec.method_names
+ opcodedesc = host_bytecode_spec.opcodedesc
+ opdescmap = host_bytecode_spec.opdescmap
+ HAVE_ARGUMENT = host_bytecode_spec.HAVE_ARGUMENT
+
+
# ____________________________________________________________
def get_block_class(opname):
Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/pyopcode.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/interpreter/pyopcode.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/interpreter/pyopcode.py Sat Oct 30 16:41:17 2010
@@ -164,6 +164,9 @@
next_instr = block.handle(self, unroller)
return next_instr
+ def call_contextmanager_exit_function(self, w_func, w_typ, w_val, w_tb):
+ return self.space.call_function(w_func, w_typ, w_val, w_tb)
+
@jit.unroll_safe
def dispatch_bytecode(self, co_code, next_instr, ec):
space = self.space
@@ -710,9 +713,14 @@
def LOAD_ATTR(self, nameindex, next_instr):
"obj.attributename"
- w_attributename = self.getname_w(nameindex)
w_obj = self.popvalue()
- w_value = self.space.getattr(w_obj, w_attributename)
+ if (self.space.config.objspace.std.withmapdict
+ and not jit.we_are_jitted()):
+ from pypy.objspace.std.mapdict import LOAD_ATTR_caching
+ w_value = LOAD_ATTR_caching(self.getcode(), w_obj, nameindex)
+ else:
+ w_attributename = self.getname_w(nameindex)
+ w_value = self.space.getattr(w_obj, w_attributename)
self.pushvalue(w_value)
LOAD_ATTR._always_inline_ = True
@@ -868,23 +876,43 @@
def WITH_CLEANUP(self, oparg, next_instr):
# see comment in END_FINALLY for stack state
- w_exitfunc = self.popvalue()
- w_unroller = self.peekvalue(2)
+ # This opcode changed a lot between CPython versions
+ if (self.pycode.magic >= 0xa0df2ef
+ # Implementation since 2.7a0: 62191 (introduce SETUP_WITH)
+ or self.pycode.magic >= 0xa0df2d1):
+ # implementation since 2.6a1: 62161 (WITH_CLEANUP optimization)
+ self.popvalue()
+ self.popvalue()
+ w_unroller = self.popvalue()
+ w_exitfunc = self.popvalue()
+ self.pushvalue(w_unroller)
+ self.pushvalue(self.space.w_None)
+ self.pushvalue(self.space.w_None)
+ elif self.pycode.magic >= 0xa0df28c:
+ # Implementation since 2.5a0: 62092 (changed WITH_CLEANUP opcode)
+ w_exitfunc = self.popvalue()
+ w_unroller = self.peekvalue(2)
+ else:
+ raise NotImplementedError("WITH_CLEANUP for CPython <= 2.4")
+
unroller = self.space.interpclass_w(w_unroller)
if isinstance(unroller, SApplicationException):
operr = unroller.operr
- w_result = self.space.call_function(w_exitfunc,
- operr.w_type,
- operr.get_w_value(self.space),
- operr.application_traceback)
+ w_traceback = self.space.wrap(operr.application_traceback)
+ w_result = self.call_contextmanager_exit_function(
+ w_exitfunc,
+ operr.w_type,
+ operr.get_w_value(self.space),
+ w_traceback)
if self.space.is_true(w_result):
# __exit__() returned True -> Swallow the exception.
self.settopvalue(self.space.w_None, 2)
else:
- self.space.call_function(w_exitfunc,
- self.space.w_None,
- self.space.w_None,
- self.space.w_None)
+ self.call_contextmanager_exit_function(
+ w_exitfunc,
+ self.space.w_None,
+ self.space.w_None,
+ self.space.w_None)
@jit.unroll_safe
def call_function(self, oparg, w_star=None, w_starstar=None):
@@ -1225,10 +1253,18 @@
frame.pushvalue(frame.space.w_None)
return self.handlerposition # jump to the handler
+class WithBlock(FinallyBlock):
+
+ def really_handle(self, frame, unroller):
+ if (frame.space.full_exceptions and
+ isinstance(unroller, SApplicationException)):
+ unroller.operr.normalize_exception(frame.space)
+ return FinallyBlock.really_handle(self, frame, unroller)
block_classes = {'SETUP_LOOP': LoopBlock,
'SETUP_EXCEPT': ExceptBlock,
- 'SETUP_FINALLY': FinallyBlock}
+ 'SETUP_FINALLY': FinallyBlock,
+ 'SETUP_WITH': WithBlock}
### helpers written at the application-level ###
# Some of these functions are expected to be generally useful if other
Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/pytokenize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/pytokenize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/pytokenize.py Sat Oct 30 16:41:17 2010
@@ -21,40 +21,42 @@
__all__ = [ "tokenize" ]
# ______________________________________________________________________
-# Automatically generated DFA's (with one or two hand tweeks):
-pseudoStatesAccepts = [True, True, True, True, True, True, True, True,
- True, True, False, True, True, True, False, False,
- False, False, True, False, False, True, True, False,
- True, False, True, False, True, False, True, False,
- False, False, True, False, False, False, True]
-
-pseudoStates = [
- {'\t': 0, '\n': 13, '\x0c': 0, '\r': 14, ' ': 0, '!': 10,
- '"': 16, '#': 18, '%': 12, '&': 12,
- "'": 15, '(': 13, ')': 13, '*': 7,
- '+': 12, ',': 13, '-': 12, '.': 6,
- '/': 11, '0': 4, '1': 5, '2': 5,
- '3': 5, '4': 5, '5': 5, '6': 5,
- '7': 5, '8': 5, '9': 5, ':': 13,
- ';': 13, '<': 9, '=': 12, '>': 8,
- '@': 13, 'A': 1,
- 'B': 1, 'C': 1, 'D': 1, 'E': 1,
- 'F': 1, 'G': 1, 'H': 1, 'I': 1,
- 'J': 1, 'K': 1, 'L': 1, 'M': 1,
- 'N': 1, 'O': 1, 'P': 1, 'Q': 1,
- 'R': 2, 'S': 1, 'T': 1, 'U': 3,
- 'V': 1, 'W': 1, 'X': 1, 'Y': 1,
- 'Z': 1, '[': 13, '\\': 17, ']': 13,
- '^': 12, '_': 1, '`': 13, 'a': 1,
- 'b': 1, 'c': 1, 'd': 1, 'e': 1,
- 'f': 1, 'g': 1, 'h': 1, 'i': 1,
- 'j': 1, 'k': 1, 'l': 1, 'm': 1,
- 'n': 1, 'o': 1, 'p': 1, 'q': 1,
- 'r': 2, 's': 1, 't': 1, 'u': 3,
- 'v': 1, 'w': 1, 'x': 1, 'y': 1,
- 'z': 1, '{': 13, '|': 12, '}': 13,
- '~': 13},
+# Automatically generated DFA's
+accepts = [True, True, True, True, True, True, True, True,
+ True, True, False, True, True, True, False, False,
+ False, False, True, True, True, False, True,
+ False, True, False, True, False, False, True,
+ False, False, False, False, True, False, False,
+ False, True]
+states = [
+ # 0
+ {'\t': 0, '\n': 13, '\x0c': 0,
+ '\r': 14, ' ': 0, '!': 10, '"': 16,
+ '#': 18, '%': 12, '&': 12, "'": 15,
+ '(': 13, ')': 13, '*': 7, '+': 12,
+ ',': 13, '-': 12, '.': 6, '/': 11,
+ '0': 4, '1': 5, '2': 5, '3': 5,
+ '4': 5, '5': 5, '6': 5, '7': 5,
+ '8': 5, '9': 5, ':': 13, ';': 13,
+ '<': 9, '=': 12, '>': 8, '@': 13,
+ 'A': 1, 'B': 1, 'C': 1, 'D': 1,
+ 'E': 1, 'F': 1, 'G': 1, 'H': 1,
+ 'I': 1, 'J': 1, 'K': 1, 'L': 1,
+ 'M': 1, 'N': 1, 'O': 1, 'P': 1,
+ 'Q': 1, 'R': 2, 'S': 1, 'T': 1,
+ 'U': 3, 'V': 1, 'W': 1, 'X': 1,
+ 'Y': 1, 'Z': 1, '[': 13, '\\': 17,
+ ']': 13, '^': 12, '_': 1, '`': 13,
+ 'a': 1, 'b': 1, 'c': 1, 'd': 1,
+ 'e': 1, 'f': 1, 'g': 1, 'h': 1,
+ 'i': 1, 'j': 1, 'k': 1, 'l': 1,
+ 'm': 1, 'n': 1, 'o': 1, 'p': 1,
+ 'q': 1, 'r': 2, 's': 1, 't': 1,
+ 'u': 3, 'v': 1, 'w': 1, 'x': 1,
+ 'y': 1, 'z': 1, '{': 13, '|': 12,
+ '}': 13, '~': 13},
+ # 1
{'0': 1, '1': 1, '2': 1, '3': 1,
'4': 1, '5': 1, '6': 1, '7': 1,
'8': 1, '9': 1, 'A': 1, 'B': 1,
@@ -71,7 +73,7 @@
'p': 1, 'q': 1, 'r': 1, 's': 1,
't': 1, 'u': 1, 'v': 1, 'w': 1,
'x': 1, 'y': 1, 'z': 1},
-
+ # 2
{'"': 16, "'": 15, '0': 1, '1': 1,
'2': 1, '3': 1, '4': 1, '5': 1,
'6': 1, '7': 1, '8': 1, '9': 1,
@@ -89,7 +91,7 @@
'r': 1, 's': 1, 't': 1, 'u': 1,
'v': 1, 'w': 1, 'x': 1, 'y': 1,
'z': 1},
-
+ # 3
{'"': 16, "'": 15, '0': 1, '1': 1,
'2': 1, '3': 1, '4': 1, '5': 1,
'6': 1, '7': 1, '8': 1, '9': 1,
@@ -107,158 +109,182 @@
'r': 2, 's': 1, 't': 1, 'u': 1,
'v': 1, 'w': 1, 'x': 1, 'y': 1,
'z': 1},
-
- {'.': 24, '0': 22, '1': 22, '2': 22,
- '3': 22, '4': 22, '5': 22, '6': 22,
- '7': 22, '8': 23, '9': 23, 'E': 25,
- 'J': 13, 'L': 13, 'X': 21, 'e': 25,
- 'j': 13, 'l': 13, 'x': 21},
-
- {'.': 24, '0': 5, '1': 5, '2': 5,
+ # 4
+ {'.': 22, '0': 20, '1': 20, '2': 20,
+ '3': 20, '4': 20, '5': 20, '6': 20,
+ '7': 20, '8': 21, '9': 21, 'E': 23,
+ 'J': 13, 'L': 13, 'X': 19, 'e': 23,
+ 'j': 13, 'l': 13, 'x': 19},
+ # 5
+ {'.': 22, '0': 5, '1': 5, '2': 5,
'3': 5, '4': 5, '5': 5, '6': 5,
- '7': 5, '8': 5, '9': 5, 'E': 25,
- 'J': 13, 'L': 13, 'e': 25, 'j': 13,
+ '7': 5, '8': 5, '9': 5, 'E': 23,
+ 'J': 13, 'L': 13, 'e': 23, 'j': 13,
'l': 13},
-
- {'0': 26, '1': 26, '2': 26, '3': 26,
- '4': 26, '5': 26, '6': 26, '7': 26,
- '8': 26, '9': 26},
-
+ # 6
+ {'0': 24, '1': 24, '2': 24, '3': 24,
+ '4': 24, '5': 24, '6': 24, '7': 24,
+ '8': 24, '9': 24},
+ # 7
{'*': 12, '=': 13},
-
+ # 8
{'=': 13, '>': 12},
-
- {'=': 13, '<': 12, '>': 13},
-
+ # 9
+ {'<': 12, '=': 13, '>': 13},
+ # 10
{'=': 13},
-
- {'=': 13, '/': 12},
-
+ # 11
+ {'/': 12, '=': 13},
+ # 12
{'=': 13},
-
+ # 13
{},
-
+ # 14
{'\n': 13},
-
- {automata.DEFAULT: 19, '\n': 27, '\\': 29, "'": 28},
-
- {automata.DEFAULT: 20, '"': 30, '\n': 27, '\\': 31},
-
+ # 15
+ {automata.DEFAULT: 28, '\n': 25, "'": 26, '\\': 27},
+ # 16
+ {automata.DEFAULT: 31, '\n': 25, '"': 29, '\\': 30},
+ # 17
{'\n': 13, '\r': 14},
-
- {automata.DEFAULT: 18, '\n': 27, '\r': 27},
-
- {automata.DEFAULT: 19, '\n': 27, '\\': 29, "'": 13},
-
- {automata.DEFAULT: 20, '"': 13, '\n': 27, '\\': 31},
-
- {'0': 21, '1': 21, '2': 21, '3': 21,
- '4': 21, '5': 21, '6': 21, '7': 21,
- '8': 21, '9': 21, 'A': 21, 'B': 21,
- 'C': 21, 'D': 21, 'E': 21, 'F': 21,
- 'L': 13, 'a': 21, 'b': 21, 'c': 21,
- 'd': 21, 'e': 21, 'f': 21, 'l': 13},
-
- {'.': 24, '0': 22, '1': 22, '2': 22,
- '3': 22, '4': 22, '5': 22, '6': 22,
- '7': 22, '8': 23, '9': 23, 'E': 25,
- 'J': 13, 'L': 13, 'e': 25, 'j': 13,
+ # 18
+ {automata.DEFAULT: 18, '\n': 25, '\r': 25},
+ # 19
+ {'0': 19, '1': 19, '2': 19, '3': 19,
+ '4': 19, '5': 19, '6': 19, '7': 19,
+ '8': 19, '9': 19, 'A': 19, 'B': 19,
+ 'C': 19, 'D': 19, 'E': 19, 'F': 19,
+ 'L': 13, 'a': 19, 'b': 19, 'c': 19,
+ 'd': 19, 'e': 19, 'f': 19, 'l': 13},
+ # 20
+ {'.': 22, '0': 20, '1': 20, '2': 20,
+ '3': 20, '4': 20, '5': 20, '6': 20,
+ '7': 20, '8': 21, '9': 21, 'E': 23,
+ 'J': 13, 'L': 13, 'e': 23, 'j': 13,
'l': 13},
-
- {'.': 24, '0': 23, '1': 23, '2': 23,
- '3': 23, '4': 23, '5': 23, '6': 23,
- '7': 23, '8': 23, '9': 23, 'E': 25,
- 'J': 13, 'e': 25, 'j': 13},
-
- {'0': 24, '1': 24, '2': 24, '3': 24,
- '4': 24, '5': 24, '6': 24, '7': 24,
- '8': 24, '9': 24, 'E': 32, 'J': 13,
+ # 21
+ {'.': 22, '0': 21, '1': 21, '2': 21,
+ '3': 21, '4': 21, '5': 21, '6': 21,
+ '7': 21, '8': 21, '9': 21, 'E': 23,
+ 'J': 13, 'e': 23, 'j': 13},
+ # 22
+ {'0': 22, '1': 22, '2': 22, '3': 22,
+ '4': 22, '5': 22, '6': 22, '7': 22,
+ '8': 22, '9': 22, 'E': 32, 'J': 13,
'e': 32, 'j': 13},
-
+ # 23
{'+': 33, '-': 33, '0': 34, '1': 34,
'2': 34, '3': 34, '4': 34, '5': 34,
'6': 34, '7': 34, '8': 34, '9': 34},
-
- {'0': 26, '1': 26, '2': 26, '3': 26,
- '4': 26, '5': 26, '6': 26, '7': 26,
- '8': 26, '9': 26, 'E': 32, 'J': 13,
+ # 24
+ {'0': 24, '1': 24, '2': 24, '3': 24,
+ '4': 24, '5': 24, '6': 24, '7': 24,
+ '8': 24, '9': 24, 'E': 32, 'J': 13,
'e': 32, 'j': 13},
-
+ # 25
{},
-
+ # 26
{"'": 13},
-
+ # 27
{automata.DEFAULT: 35, '\n': 13, '\r': 14},
-
+ # 28
+ {automata.DEFAULT: 28, '\n': 25, "'": 13, '\\': 27},
+ # 29
{'"': 13},
-
+ # 30
{automata.DEFAULT: 36, '\n': 13, '\r': 14},
-
+ # 31
+ {automata.DEFAULT: 31, '\n': 25, '"': 13, '\\': 30},
+ # 32
{'+': 37, '-': 37, '0': 38, '1': 38,
'2': 38, '3': 38, '4': 38, '5': 38,
'6': 38, '7': 38, '8': 38, '9': 38},
-
-
+ # 33
{'0': 34, '1': 34, '2': 34, '3': 34,
'4': 34, '5': 34, '6': 34, '7': 34,
'8': 34, '9': 34},
-
+ # 34
{'0': 34, '1': 34, '2': 34, '3': 34,
'4': 34, '5': 34, '6': 34, '7': 34,
'8': 34, '9': 34, 'J': 13, 'j': 13},
-
- {automata.DEFAULT: 35, '\n': 27, '\\': 29, "'": 13},
-
- {automata.DEFAULT: 36, '"': 13, '\n': 27, '\\': 31},
-
+ # 35
+ {automata.DEFAULT: 35, '\n': 25, "'": 13, '\\': 27},
+ # 36
+ {automata.DEFAULT: 36, '\n': 25, '"': 13, '\\': 30},
+ # 37
{'0': 38, '1': 38, '2': 38, '3': 38,
'4': 38, '5': 38, '6': 38, '7': 38,
'8': 38, '9': 38},
-
+ # 38
{'0': 38, '1': 38, '2': 38, '3': 38,
'4': 38, '5': 38, '6': 38, '7': 38,
'8': 38, '9': 38, 'J': 13, 'j': 13},
]
+pseudoDFA = automata.DFA(states, accepts)
-pseudoDFA = automata.DFA(pseudoStates, pseudoStatesAccepts)
-
-double3StatesAccepts = [False, False, False, False, False, True]
-double3States = [
+accepts = [False, False, False, False, False, True]
+states = [
+ # 0
{automata.DEFAULT: 0, '"': 1, '\\': 2},
+ # 1
{automata.DEFAULT: 4, '"': 3, '\\': 2},
+ # 2
{automata.DEFAULT: 4},
+ # 3
{automata.DEFAULT: 4, '"': 5, '\\': 2},
+ # 4
{automata.DEFAULT: 4, '"': 1, '\\': 2},
+ # 5
{automata.DEFAULT: 4, '"': 5, '\\': 2},
]
-double3DFA = automata.NonGreedyDFA(double3States, double3StatesAccepts)
+double3DFA = automata.NonGreedyDFA(states, accepts)
-single3StatesAccepts = [False, False, False, False, False, True]
-single3States = [
- {automata.DEFAULT: 0, '\\': 2, "'": 1},
- {automata.DEFAULT: 4, '\\': 2, "'": 3},
+accepts = [False, False, False, False, False, True]
+states = [
+ # 0
+ {automata.DEFAULT: 0, "'": 1, '\\': 2},
+ # 1
+ {automata.DEFAULT: 4, "'": 3, '\\': 2},
+ # 2
{automata.DEFAULT: 4},
- {automata.DEFAULT: 4, '\\': 2, "'": 5},
- {automata.DEFAULT: 4, '\\': 2, "'": 1},
- {automata.DEFAULT: 4, '\\': 2, "'": 5},
+ # 3
+ {automata.DEFAULT: 4, "'": 5, '\\': 2},
+ # 4
+ {automata.DEFAULT: 4, "'": 1, '\\': 2},
+ # 5
+ {automata.DEFAULT: 4, "'": 5, '\\': 2},
]
-single3DFA = automata.NonGreedyDFA(single3States, single3StatesAccepts)
+single3DFA = automata.NonGreedyDFA(states, accepts)
-singleStatesAccepts = [False, True, False]
-singleStates = [
- {automata.DEFAULT: 0, '\\': 2, "'": 1},
+accepts = [False, True, False, False]
+states = [
+ # 0
+ {automata.DEFAULT: 0, "'": 1, '\\': 2},
+ # 1
{},
- {automata.DEFAULT: 0},
+ # 2
+ {automata.DEFAULT: 3},
+ # 3
+ {automata.DEFAULT: 3, "'": 1, '\\': 2},
]
-singleDFA = automata.DFA(singleStates, singleStatesAccepts)
+singleDFA = automata.DFA(states, accepts)
-doubleStatesAccepts = [False, True, False]
-doubleStates = [
+accepts = [False, True, False, False]
+states = [
+ # 0
{automata.DEFAULT: 0, '"': 1, '\\': 2},
+ # 1
{},
- {automata.DEFAULT: 0},
+ # 2
+ {automata.DEFAULT: 3},
+ # 3
+ {automata.DEFAULT: 3, '"': 1, '\\': 2},
]
-doubleDFA = automata.DFA(doubleStates, doubleStatesAccepts)
+doubleDFA = automata.DFA(states, accepts)
+
+
+#_______________________________________________________________________
+# End of automatically generated DFA's
endDFAs = {"'" : singleDFA,
'"' : doubleDFA,
Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/test/test_function.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/interpreter/test/test_function.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/interpreter/test/test_function.py Sat Oct 30 16:41:17 2010
@@ -482,6 +482,11 @@
raises(TypeError, m, MyInst(None))
raises(TypeError, m, MyInst(42))
+ def test_invalid_creation(self):
+ import new
+ def f(): pass
+ raises(TypeError, new.instancemethod, f, None)
+
class TestMethod:
def setup_method(self, method):
Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/typedef.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/interpreter/typedef.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/interpreter/typedef.py Sat Oct 30 16:41:17 2010
@@ -133,6 +133,13 @@
typedef = cls.typedef
if wants_dict and typedef.hasdict:
wants_dict = False
+ if config.objspace.std.withmapdict and not typedef.hasdict:
+ # mapdict only works if the type does not already have a dict
+ if wants_del:
+ parentcls = get_unique_interplevel_subclass(config, cls, True, True,
+ False, True)
+ return _usersubclswithfeature(config, parentcls, "del")
+ return _usersubclswithfeature(config, cls, "user", "dict", "weakref", "slots")
# Forest of if's - see the comment above.
if wants_del:
if wants_dict:
@@ -186,10 +193,21 @@
def add(Proto):
for key, value in Proto.__dict__.items():
- if not key.startswith('__') or key == '__del__':
+ if (not key.startswith('__') and not key.startswith('_mixin_')
+ or key == '__del__'):
+ if hasattr(value, "func_name"):
+ value = func_with_new_name(value, value.func_name)
body[key] = value
+ if (config.objspace.std.withmapdict and "dict" in features):
+ from pypy.objspace.std.mapdict import BaseMapdictObject, ObjectMixin
+ add(BaseMapdictObject)
+ add(ObjectMixin)
+ body["user_overridden_class"] = True
+ features = ()
+
if "user" in features: # generic feature needed by all subcls
+
class Proto(object):
user_overridden_class = True
@@ -245,16 +263,10 @@
return self.slots_w[index]
add(Proto)
- wantdict = "dict" in features
- if wantdict and config.objspace.std.withinlineddict:
- from pypy.objspace.std.objectobject import W_ObjectObject
- from pypy.objspace.std.inlinedict import make_mixin
- if supercls is W_ObjectObject:
- Mixin = make_mixin(config)
- add(Mixin)
- wantdict = False
-
- if wantdict:
+ if "dict" in features:
+ base_user_setup = supercls.user_setup.im_func
+ if "user_setup" in body:
+ base_user_setup = body["user_setup"]
class Proto(object):
def getdict(self):
return self.w__dict__
@@ -263,24 +275,13 @@
self.w__dict__ = check_new_dictionary(space, w_dict)
def user_setup(self, space, w_subtype):
- self.space = space
- self.w__class__ = w_subtype
self.w__dict__ = space.newdict(
instance=True, classofinstance=w_subtype)
- self.user_setup_slots(w_subtype.nslots)
+ base_user_setup(self, space, w_subtype)
def setclass(self, space, w_subtype):
# only used by descr_set___class__
self.w__class__ = w_subtype
- if space.config.objspace.std.withshadowtracking:
- self.w__dict__.set_shadows_anything()
-
- def getdictvalue_attr_is_in_class(self, space, name):
- w_dict = self.w__dict__
- if space.config.objspace.std.withshadowtracking:
- if not w_dict.shadows_anything():
- return None
- return space.finditem_str(w_dict, name)
add(Proto)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/detect_cpu.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/detect_cpu.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/detect_cpu.py Sat Oct 30 16:41:17 2010
@@ -23,11 +23,6 @@
mach = os.popen('uname -m', 'r').read().strip()
if not mach:
raise ProcessorAutodetectError, "cannot run 'uname -m'"
- if mach == 'x86_64':
- if sys.maxint == 2147483647:
- mach = 'x86' # it's a 64-bit processor but in 32-bits mode, maybe
- else:
- assert sys.maxint == 2 ** 63 - 1
try:
return {'i386': 'x86',
'i486': 'x86',
@@ -36,17 +31,21 @@
'i86pc': 'x86', # Solaris/Intel
'x86': 'x86', # Apple
'Power Macintosh': 'ppc',
- 'x86_64': 'x86_64',
+ 'x86_64': 'x86',
}[mach]
except KeyError:
raise ProcessorAutodetectError, "unsupported processor '%s'" % mach
def autodetect():
model = autodetect_main_model()
- if model == 'x86':
- from pypy.jit.backend.x86.detect_sse2 import detect_sse2
- if not detect_sse2():
- model = 'x86-without-sse2'
+ if sys.maxint == 2**63-1:
+ model += '_64'
+ else:
+ assert sys.maxint == 2**31-1
+ if model == 'x86':
+ from pypy.jit.backend.x86.detect_sse2 import detect_sse2
+ if not detect_sse2():
+ model = 'x86-without-sse2'
return model
def getcpuclassname(backend_name="auto"):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py Sat Oct 30 16:41:17 2010
@@ -10,7 +10,7 @@
BoxInt, BoxPtr, BoxObj, BoxFloat,
REF, INT, FLOAT)
from pypy.jit.codewriter import heaptracker
-from pypy.rpython.lltypesystem import lltype, llmemory, rclass, rstr
+from pypy.rpython.lltypesystem import lltype, llmemory, rclass, rstr, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.module.support import LLSupport, OOSupport
from pypy.rpython.llinterp import LLException
@@ -129,7 +129,7 @@
'arraylen_gc' : (('ref',), 'int'),
'call' : (('ref', 'varargs'), 'intorptr'),
'call_assembler' : (('varargs',), 'intorptr'),
- 'cond_call_gc_wb' : (('ptr',), None),
+ 'cond_call_gc_wb' : (('ptr', 'ptr'), None),
'oosend' : (('varargs',), 'intorptr'),
'oosend_pure' : (('varargs',), 'intorptr'),
'guard_true' : (('bool',), None),
@@ -305,12 +305,12 @@
loop = _from_opaque(loop)
loop.operations.append(Operation(opnum))
-def compile_add_descr(loop, ofs, type):
+def compile_add_descr(loop, ofs, type, arg_types):
from pypy.jit.backend.llgraph.runner import Descr
loop = _from_opaque(loop)
op = loop.operations[-1]
assert isinstance(type, str) and len(type) == 1
- op.descr = Descr(ofs, type)
+ op.descr = Descr(ofs, type, arg_types=arg_types)
def compile_add_loop_token(loop, descr):
if we_are_translated():
@@ -801,7 +801,7 @@
else:
raise TypeError(x)
try:
- return _do_call_common(func, args_in_order)
+ return _do_call_common(func, args_in_order, calldescr)
except LLException, lle:
_last_exception = lle
d = {'v': None,
@@ -810,7 +810,7 @@
FLOAT: 0.0}
return d[calldescr.typeinfo]
- def op_cond_call_gc_wb(self, descr, a):
+ def op_cond_call_gc_wb(self, descr, a, b):
py.test.skip("cond_call_gc_wb not supported")
def op_oosend(self, descr, obj, *args):
@@ -1018,6 +1018,9 @@
if isinstance(TYPE, lltype.Ptr):
if isinstance(x, (int, long, llmemory.AddressAsInt)):
x = llmemory.cast_int_to_adr(x)
+ if TYPE is rffi.VOIDP:
+ # assume that we want a "C-style" cast, without typechecking the value
+ return rffi.cast(TYPE, x)
return llmemory.cast_adr_to_ptr(x, TYPE)
elif TYPE == llmemory.Address:
if isinstance(x, (int, long, llmemory.AddressAsInt)):
@@ -1411,10 +1414,26 @@
def do_call_pushfloat(x):
_call_args_f.append(x)
-def _do_call_common(f, args_in_order=None):
+kind2TYPE = {
+ 'i': lltype.Signed,
+ 'f': lltype.Float,
+ 'v': lltype.Void,
+ }
+
+def _do_call_common(f, args_in_order=None, calldescr=None):
ptr = llmemory.cast_int_to_adr(f).ptr
- FUNC = lltype.typeOf(ptr).TO
- ARGS = FUNC.ARGS
+ PTR = lltype.typeOf(ptr)
+ if PTR == rffi.VOIDP:
+ # it's a pointer to a C function, so we don't have a precise
+ # signature: create one from the descr
+ ARGS = map(kind2TYPE.get, calldescr.arg_types)
+ RESULT = kind2TYPE[calldescr.typeinfo]
+ FUNC = lltype.FuncType(ARGS, RESULT)
+ func_to_call = rffi.cast(lltype.Ptr(FUNC), ptr)
+ else:
+ FUNC = PTR.TO
+ ARGS = FUNC.ARGS
+ func_to_call = ptr._obj._callable
args = cast_call_args(ARGS, _call_args_i, _call_args_r, _call_args_f,
args_in_order)
del _call_args_i[:]
@@ -1426,7 +1445,7 @@
result = llinterp.eval_graph(ptr._obj.graph, args)
# ^^^ may raise, in which case we get an LLException
else:
- result = ptr._obj._callable(*args)
+ result = func_to_call(*args)
return result
def do_call_void(f):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/runner.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/runner.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/runner.py Sat Oct 30 16:41:17 2010
@@ -118,13 +118,13 @@
self._descrs[key] = descr
return descr
- def compile_bridge(self, faildescr, inputargs, operations):
+ def compile_bridge(self, faildescr, inputargs, operations, log=True):
c = llimpl.compile_start()
self._compile_loop_or_bridge(c, inputargs, operations)
old, oldindex = faildescr._compiled_fail
llimpl.compile_redirect_fail(old, oldindex, c)
- def compile_loop(self, inputargs, operations, loopdescr):
+ def compile_loop(self, inputargs, operations, loopdescr, log=True):
"""In a real assembler backend, this should assemble the given
list of operations. Here we just generate a similar CompiledLoop
instance. The code here is RPython, whereas the code in llimpl
@@ -154,7 +154,7 @@
llimpl.compile_add(c, op.getopnum())
descr = op.getdescr()
if isinstance(descr, Descr):
- llimpl.compile_add_descr(c, descr.ofs, descr.typeinfo)
+ llimpl.compile_add_descr(c, descr.ofs, descr.typeinfo, descr.arg_types)
if isinstance(descr, history.LoopToken) and op.getopnum() != rop.JUMP:
llimpl.compile_add_loop_token(c, descr)
if self.is_oo and isinstance(descr, (OODescr, MethDescr)):
@@ -297,6 +297,18 @@
return self.getdescr(0, token[0], extrainfo=extrainfo,
arg_types=''.join(arg_types))
+ def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo=None):
+ from pypy.jit.backend.llsupport.ffisupport import get_ffi_type_kind
+ arg_types = []
+ for arg in ffi_args:
+ kind = get_ffi_type_kind(arg)
+ if kind != history.VOID:
+ arg_types.append(kind)
+ reskind = get_ffi_type_kind(ffi_result)
+ return self.getdescr(0, reskind, extrainfo=extrainfo,
+ arg_types=''.join(arg_types))
+
+
def grab_exc_value(self):
return llimpl.grab_exc_value()
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/descr.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/descr.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/descr.py Sat Oct 30 16:41:17 2010
@@ -82,6 +82,7 @@
_is_pointer_field = False # unless overridden by GcPtrFieldDescr
_is_float_field = False # unless overridden by FloatFieldDescr
+ _is_field_signed = False # unless overridden by XxxFieldDescr
def is_pointer_field(self):
return self._is_pointer_field
@@ -89,6 +90,9 @@
def is_float_field(self):
return self._is_float_field
+ def is_field_signed(self):
+ return self._is_field_signed
+
def repr_of_descr(self):
return '<%s %s %s>' % (self._clsname, self.name, self.offset)
@@ -105,7 +109,7 @@
def getFieldDescrClass(TYPE):
return getDescrClass(TYPE, BaseFieldDescr, GcPtrFieldDescr,
NonGcPtrFieldDescr, 'Field', 'get_field_size',
- '_is_float_field')
+ '_is_float_field', '_is_field_signed')
def get_field_descr(gccache, STRUCT, fieldname):
cache = gccache._cache_field
@@ -144,6 +148,7 @@
_is_array_of_pointers = False # unless overridden by GcPtrArrayDescr
_is_array_of_floats = False # unless overridden by FloatArrayDescr
+ _is_item_signed = False # unless overridden by XxxArrayDescr
def is_array_of_pointers(self):
return self._is_array_of_pointers
@@ -151,6 +156,9 @@
def is_array_of_floats(self):
return self._is_array_of_floats
+ def is_item_signed(self):
+ return self._is_item_signed
+
def repr_of_descr(self):
return '<%s>' % self._clsname
@@ -186,12 +194,12 @@
def getArrayDescrClass(ARRAY):
return getDescrClass(ARRAY.OF, BaseArrayDescr, GcPtrArrayDescr,
NonGcPtrArrayDescr, 'Array', 'get_item_size',
- '_is_array_of_floats')
+ '_is_array_of_floats', '_is_item_signed')
def getArrayNoLengthDescrClass(ARRAY):
return getDescrClass(ARRAY.OF, BaseArrayNoLengthDescr, GcPtrArrayNoLengthDescr,
NonGcPtrArrayNoLengthDescr, 'ArrayNoLength', 'get_item_size',
- '_is_array_of_floats')
+ '_is_array_of_floats', '_is_item_signed')
def get_array_descr(gccache, ARRAY):
cache = gccache._cache_array
@@ -242,6 +250,9 @@
def get_result_size(self, translate_support_code):
raise NotImplementedError
+ def is_result_signed(self):
+ return False # unless overridden
+
def create_call_stub(self, rtyper, RESULT):
def process(c):
arg = 'args_%s[%d]' % (c, seen[c])
@@ -307,6 +318,30 @@
_return_type = history.INT
call_stub = staticmethod(lambda func, args_i, args_r, args_f: 0)
+ _is_result_signed = False # can be overridden in XxxCallDescr
+ def is_result_signed(self):
+ return self._is_result_signed
+
+class DynamicIntCallDescr(BaseIntCallDescr):
+ """
+ calldescr that works for every integer type, by explicitly passing it the
+ size of the result. Used only by get_call_descr_dynamic
+ """
+ _clsname = 'DynamicIntCallDescr'
+
+ def __init__(self, arg_classes, result_size, result_sign, extrainfo=None):
+ BaseIntCallDescr.__init__(self, arg_classes, extrainfo)
+ assert isinstance(result_sign, bool)
+ self._result_size = chr(result_size)
+ self._result_sign = result_sign
+
+ def get_result_size(self, translate_support_code):
+ return ord(self._result_size)
+
+ def is_result_signed(self):
+ return self._result_sign
+
+
class NonGcPtrCallDescr(BaseIntCallDescr):
_clsname = 'NonGcPtrCallDescr'
def get_result_size(self, translate_support_code):
@@ -341,7 +376,8 @@
return FloatCallDescr
return getDescrClass(RESULT, BaseIntCallDescr, GcPtrCallDescr,
NonGcPtrCallDescr, 'Call', 'get_result_size',
- Ellipsis) # <= floatattrname should not be used here
+ Ellipsis, # <= floatattrname should not be used here
+ '_is_result_signed')
def get_call_descr(gccache, ARGS, RESULT, extrainfo=None):
arg_classes = []
@@ -368,7 +404,8 @@
# ____________________________________________________________
def getDescrClass(TYPE, BaseDescr, GcPtrDescr, NonGcPtrDescr,
- nameprefix, methodname, floatattrname, _cache={}):
+ nameprefix, methodname, floatattrname, signedattrname,
+ _cache={}):
if isinstance(TYPE, lltype.Ptr):
if TYPE.TO._gckind == 'gc':
return GcPtrDescr
@@ -388,6 +425,8 @@
#
if TYPE is lltype.Float:
setattr(Descr, floatattrname, True)
+ elif TYPE is not lltype.Bool and rffi.cast(TYPE, -1) == -1:
+ setattr(Descr, signedattrname, True)
#
_cache[nameprefix, TYPE] = Descr
return Descr
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/gc.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/gc.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/gc.py Sat Oct 30 16:41:17 2010
@@ -158,7 +158,7 @@
# used to avoid too many duplications in the GCREF_LISTs.
self.hashtable = lltype.malloc(self.HASHTABLE,
self.HASHTABLE_SIZE+1,
- flavor='raw')
+ flavor='raw', track_allocation=False)
dummy = lltype.direct_ptradd(lltype.direct_arrayitems(self.hashtable),
self.HASHTABLE_SIZE)
dummy = llmemory.cast_ptr_to_adr(dummy)
@@ -252,14 +252,15 @@
def _enlarge_gcmap(self):
newlength = 250 + self._gcmap_maxlength * 2
- newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw')
+ newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw',
+ track_allocation=False)
oldgcmap = self._gcmap
for i in range(self._gcmap_curlength):
newgcmap[i] = oldgcmap[i]
self._gcmap = newgcmap
self._gcmap_maxlength = newlength
if oldgcmap:
- lltype.free(oldgcmap, flavor='raw')
+ lltype.free(oldgcmap, flavor='raw', track_allocation=False)
def get_basic_shape(self, is_64_bit=False):
# XXX: Should this code even really know about stack frame layout of
@@ -308,7 +309,8 @@
# them inside bigger arrays) and we never try to share them.
length = len(shape)
compressed = lltype.malloc(self.CALLSHAPE_ARRAY, length,
- flavor='raw')
+ flavor='raw',
+ track_allocation=False) # memory leak
for i in range(length):
compressed[length-1-i] = rffi.cast(rffi.UCHAR, shape[i])
return llmemory.cast_ptr_to_adr(compressed)
@@ -404,7 +406,7 @@
self.GC_MALLOC_BASIC = lltype.Ptr(lltype.FuncType(
[lltype.Signed, lltype.Signed], llmemory.GCREF))
self.WB_FUNCPTR = lltype.Ptr(lltype.FuncType(
- [llmemory.Address], lltype.Void))
+ [llmemory.Address, llmemory.Address], lltype.Void))
self.write_barrier_descr = WriteBarrierDescr(self)
#
def malloc_array(itemsize, tid, num_elem):
@@ -550,7 +552,8 @@
# the GC, and call it immediately
llop1 = self.llop1
funcptr = llop1.get_write_barrier_failing_case(self.WB_FUNCPTR)
- funcptr(llmemory.cast_ptr_to_adr(gcref_struct))
+ funcptr(llmemory.cast_ptr_to_adr(gcref_struct),
+ llmemory.cast_ptr_to_adr(gcref_newptr))
def rewrite_assembler(self, cpu, operations):
# Perform two kinds of rewrites in parallel:
@@ -589,7 +592,7 @@
v = op.getarg(1)
if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and
bool(v.value)): # store a non-NULL
- self._gen_write_barrier(newops, op.getarg(0))
+ self._gen_write_barrier(newops, op.getarg(0), v)
op = op.copy_and_change(rop.SETFIELD_RAW)
# ---------- write barrier for SETARRAYITEM_GC ----------
if op.getopnum() == rop.SETARRAYITEM_GC:
@@ -598,15 +601,15 @@
bool(v.value)): # store a non-NULL
# XXX detect when we should produce a
# write_barrier_from_array
- self._gen_write_barrier(newops, op.getarg(0))
+ self._gen_write_barrier(newops, op.getarg(0), v)
op = op.copy_and_change(rop.SETARRAYITEM_RAW)
# ----------
newops.append(op)
del operations[:]
operations.extend(newops)
- def _gen_write_barrier(self, newops, v_base):
- args = [v_base]
+ def _gen_write_barrier(self, newops, v_base, v_value):
+ args = [v_base, v_value]
newops.append(ResOperation(rop.COND_CALL_GC_WB, args, None,
descr=self.write_barrier_descr))
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/llmodel.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/llmodel.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/llmodel.py Sat Oct 30 16:41:17 2010
@@ -17,6 +17,7 @@
from pypy.jit.backend.llsupport.descr import get_call_descr
from pypy.jit.backend.llsupport.descr import BaseIntCallDescr, GcPtrCallDescr
from pypy.jit.backend.llsupport.descr import FloatCallDescr, VoidCallDescr
+from pypy.jit.backend.llsupport.ffisupport import get_call_descr_dynamic
from pypy.rpython.annlowlevel import cast_instance_to_base_ptr
@@ -82,7 +83,8 @@
# read back by the machine code reading at the address given by
# pos_exception() and pos_exc_value().
_exception_emulator = lltype.malloc(rffi.CArray(lltype.Signed), 2,
- zero=True, flavor='raw')
+ zero=True, flavor='raw',
+ immortal=True)
self._exception_emulator = _exception_emulator
def _store_exception(lle):
@@ -210,7 +212,8 @@
assert isinstance(fielddescr, BaseFieldDescr)
ofs = fielddescr.offset
size = fielddescr.get_field_size(self.translate_support_code)
- return ofs, size
+ sign = fielddescr.is_field_signed()
+ return ofs, size, sign
unpack_fielddescr_size._always_inline_ = True
def arraydescrof(self, A):
@@ -225,12 +228,16 @@
assert isinstance(arraydescr, BaseArrayDescr)
ofs = arraydescr.get_base_size(self.translate_support_code)
size = arraydescr.get_item_size(self.translate_support_code)
- return ofs, size
+ sign = arraydescr.is_item_signed()
+ return ofs, size, sign
unpack_arraydescr_size._always_inline_ = True
def calldescrof(self, FUNC, ARGS, RESULT, extrainfo=None):
return get_call_descr(self.gc_ll_descr, ARGS, RESULT, extrainfo)
+ def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo=None):
+ return get_call_descr_dynamic(ffi_args, ffi_result, extrainfo)
+
def get_overflow_error(self):
ovf_vtable = self.cast_adr_to_int(self._ovf_error_vtable)
ovf_inst = lltype.cast_opaque_ptr(llmemory.GCREF,
@@ -252,15 +259,21 @@
@specialize.argtype(2)
def bh_getarrayitem_gc_i(self, arraydescr, gcref, itemindex):
- ofs, size = self.unpack_arraydescr_size(arraydescr)
+ ofs, size, sign = self.unpack_arraydescr_size(arraydescr)
# --- start of GC unsafe code (no GC operation!) ---
items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for STYPE, UTYPE, itemsize in unroll_basic_sizes:
if size == itemsize:
- items = rffi.cast(rffi.CArrayPtr(TYPE), items)
- val = items[itemindex]
+ if sign:
+ items = rffi.cast(rffi.CArrayPtr(STYPE), items)
+ val = items[itemindex]
+ val = rffi.cast(lltype.Signed, val)
+ else:
+ items = rffi.cast(rffi.CArrayPtr(UTYPE), items)
+ val = items[itemindex]
+ val = rffi.cast(lltype.Signed, val)
# --- end of GC unsafe code ---
- return rffi.cast(lltype.Signed, val)
+ return val
else:
raise NotImplementedError("size = %d" % size)
@@ -285,10 +298,10 @@
@specialize.argtype(2)
def bh_setarrayitem_gc_i(self, arraydescr, gcref, itemindex, newvalue):
- ofs, size = self.unpack_arraydescr_size(arraydescr)
+ ofs, size, sign = self.unpack_arraydescr_size(arraydescr)
# --- start of GC unsafe code (no GC operation!) ---
items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for TYPE, _, itemsize in unroll_basic_sizes:
if size == itemsize:
items = rffi.cast(rffi.CArrayPtr(TYPE), items)
items[itemindex] = rffi.cast(TYPE, newvalue)
@@ -339,14 +352,22 @@
@specialize.argtype(1)
def _base_do_getfield_i(self, struct, fielddescr):
- ofs, size = self.unpack_fielddescr_size(fielddescr)
+ ofs, size, sign = self.unpack_fielddescr_size(fielddescr)
# --- start of GC unsafe code (no GC operation!) ---
fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for STYPE, UTYPE, itemsize in unroll_basic_sizes:
if size == itemsize:
- val = rffi.cast(rffi.CArrayPtr(TYPE), fieldptr)[0]
+ # Note that in the common case where size==sizeof(Signed),
+ # both cases of what follows are doing the same thing.
+ # But gcc is clever enough to figure this out :-)
+ if sign:
+ val = rffi.cast(rffi.CArrayPtr(STYPE), fieldptr)[0]
+ val = rffi.cast(lltype.Signed, val)
+ else:
+ val = rffi.cast(rffi.CArrayPtr(UTYPE), fieldptr)[0]
+ val = rffi.cast(lltype.Signed, val)
# --- end of GC unsafe code ---
- return rffi.cast(lltype.Signed, val)
+ return val
else:
raise NotImplementedError("size = %d" % size)
@@ -378,10 +399,10 @@
@specialize.argtype(1)
def _base_do_setfield_i(self, struct, fielddescr, newvalue):
- ofs, size = self.unpack_fielddescr_size(fielddescr)
+ ofs, size, sign = self.unpack_fielddescr_size(fielddescr)
# --- start of GC unsafe code (no GC operation!) ---
fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs)
- for TYPE, itemsize in unroll_basic_sizes:
+ for TYPE, _, itemsize in unroll_basic_sizes:
if size == itemsize:
fieldptr = rffi.cast(rffi.CArrayPtr(TYPE), fieldptr)
fieldptr[0] = rffi.cast(TYPE, newvalue)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/symbolic.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/symbolic.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/symbolic.py Sat Oct 30 16:41:17 2010
@@ -69,8 +69,9 @@
SIZEOF_INT = get_size(rffi.INT, False)
SIZEOF_FLOAT = get_size(lltype.Float, False)
-unroll_basic_sizes = unrolling_iterable([(lltype.Signed, WORD),
- (lltype.Char, SIZEOF_CHAR),
- (rffi.SHORT, SIZEOF_SHORT),
- (rffi.INT, SIZEOF_INT)])
+unroll_basic_sizes = unrolling_iterable([
+ (lltype.Signed, lltype.Unsigned, WORD),
+ (rffi.SIGNEDCHAR, lltype.Char, SIZEOF_CHAR),
+ (rffi.SHORT, rffi.USHORT, SIZEOF_SHORT),
+ (rffi.INT, rffi.UINT, SIZEOF_INT)])
# does not contain Float ^^^ which must be special-cased
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_descr.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_descr.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_descr.py Sat Oct 30 16:41:17 2010
@@ -83,6 +83,18 @@
assert descr_f.is_float_field()
+def test_get_field_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ descr_x = get_field_descr(c2, S, 'x')
+ assert descr_x.is_field_signed() == signed
+
+
def test_get_array_descr():
U = lltype.Struct('U')
T = lltype.GcStruct('T')
@@ -164,6 +176,25 @@
assert descr.get_base_size(False) == 0
assert descr.get_ofs_length(False) == -1
+
+def test_get_array_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ A = lltype.GcArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ arraydescr = get_array_descr(c2, A)
+ assert arraydescr.is_item_signed() == signed
+ #
+ RA = rffi.CArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ arraydescr = get_array_descr(c2, RA)
+ assert arraydescr.is_item_signed() == signed
+
+
def test_get_call_descr_not_translated():
c0 = GcCache(False)
descr1 = get_call_descr(c0, [lltype.Char, lltype.Signed], lltype.Char)
@@ -219,6 +250,17 @@
extrainfo = descr3.get_extra_info()
assert extrainfo is None
+def test_get_call_descr_sign():
+ for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False),
+ (rffi.SHORT, True), (rffi.USHORT, False),
+ (rffi.INT, True), (rffi.UINT, False),
+ (rffi.LONG, True), (rffi.ULONG, False)]:
+ A = lltype.GcArray(RESTYPE)
+ for tsc in [False, True]:
+ c2 = GcCache(tsc)
+ descr1 = get_call_descr(c2, [], RESTYPE)
+ assert descr1.is_result_signed() == signed
+
def test_repr_of_descr():
c0 = GcCache(False)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_gc.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_gc.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_gc.py Sat Oct 30 16:41:17 2010
@@ -141,8 +141,8 @@
repr(offset_to_length), p))
return p
- def _write_barrier_failing_case(self, adr_struct):
- self.record.append(('barrier', adr_struct))
+ def _write_barrier_failing_case(self, adr_struct, adr_newptr):
+ self.record.append(('barrier', adr_struct, adr_newptr))
def get_write_barrier_failing_case(self, FPTRTYPE):
return llhelper(FPTRTYPE, self._write_barrier_failing_case)
@@ -239,6 +239,7 @@
s_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
r_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, r)
s_adr = llmemory.cast_ptr_to_adr(s)
+ r_adr = llmemory.cast_ptr_to_adr(r)
#
s_hdr.tid &= ~gc_ll_descr.GCClass.JIT_WB_IF_FLAG
gc_ll_descr.do_write_barrier(s_gcref, r_gcref)
@@ -246,7 +247,7 @@
#
s_hdr.tid |= gc_ll_descr.GCClass.JIT_WB_IF_FLAG
gc_ll_descr.do_write_barrier(s_gcref, r_gcref)
- assert self.llop1.record == [('barrier', s_adr)]
+ assert self.llop1.record == [('barrier', s_adr, r_adr)]
def test_gen_write_barrier(self):
gc_ll_descr = self.gc_ll_descr
@@ -254,11 +255,13 @@
#
newops = []
v_base = BoxPtr()
- gc_ll_descr._gen_write_barrier(newops, v_base)
+ v_value = BoxPtr()
+ gc_ll_descr._gen_write_barrier(newops, v_base, v_value)
assert llop1.record == []
assert len(newops) == 1
assert newops[0].getopnum() == rop.COND_CALL_GC_WB
assert newops[0].getarg(0) == v_base
+ assert newops[0].getarg(1) == v_value
assert newops[0].result is None
wbdescr = newops[0].getdescr()
assert isinstance(wbdescr.jit_wb_if_flag, int)
@@ -358,6 +361,7 @@
#
assert operations[0].getopnum() == rop.COND_CALL_GC_WB
assert operations[0].getarg(0) == v_base
+ assert operations[0].getarg(1) == v_value
assert operations[0].result is None
#
assert operations[1].getopnum() == rop.SETFIELD_RAW
@@ -381,6 +385,7 @@
#
assert operations[0].getopnum() == rop.COND_CALL_GC_WB
assert operations[0].getarg(0) == v_base
+ assert operations[0].getarg(1) == v_value
assert operations[0].result is None
#
assert operations[1].getopnum() == rop.SETARRAYITEM_RAW
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/model.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/model.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/model.py Sat Oct 30 16:41:17 2010
@@ -33,14 +33,14 @@
pass
- def compile_loop(self, inputargs, operations, looptoken):
+ def compile_loop(self, inputargs, operations, looptoken, log=True):
"""Assemble the given loop.
Extra attributes should be put in the LoopToken to
point to the compiled loop in assembler.
"""
raise NotImplementedError
- def compile_bridge(self, faildescr, inputargs, operations):
+ def compile_bridge(self, faildescr, inputargs, operations, log=True):
"""Assemble the bridge.
The FailDescr is the descr of the original guard that failed.
"""
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/test/runner_test.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/test/runner_test.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/test/runner_test.py Sat Oct 30 16:41:17 2010
@@ -9,12 +9,13 @@
ConstObj, BoxFloat, ConstFloat)
from pypy.jit.metainterp.resoperation import ResOperation, rop
from pypy.jit.metainterp.typesystem import deref
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi, rclass
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.llinterp import LLException
from pypy.jit.codewriter import heaptracker
+from pypy.rlib.rarithmetic import intmask
class Runner(object):
@@ -421,6 +422,7 @@
assert x == 3.5 - 42
def test_call(self):
+ from pypy.rlib.libffi import types
def func_int(a, b):
return a + b
@@ -428,23 +430,31 @@
return chr(ord(c) + ord(c1))
functions = [
- (func_int, lltype.Signed, 655360),
- (func_int, rffi.SHORT, 1213),
- (func_char, lltype.Char, 12)
+ (func_int, lltype.Signed, types.sint, 655360),
+ (func_int, rffi.SHORT, types.sint16, 1213),
+ (func_char, lltype.Char, types.uchar, 12)
]
- for func, TP, num in functions:
+ for func, TP, ffi_type, num in functions:
cpu = self.cpu
#
FPTR = self.Ptr(self.FuncType([TP, TP], TP))
func_ptr = llhelper(FPTR, func)
FUNC = deref(FPTR)
- calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
funcbox = self.get_funcbox(cpu, func_ptr)
+ # first, try it with the "normal" calldescr
+ calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
res = self.execute_operation(rop.CALL,
[funcbox, BoxInt(num), BoxInt(num)],
'int', descr=calldescr)
assert res.value == 2 * num
+ # then, try it with the dynamic calldescr
+ dyn_calldescr = cpu.calldescrof_dynamic([ffi_type, ffi_type], ffi_type)
+ res = self.execute_operation(rop.CALL,
+ [funcbox, BoxInt(num), BoxInt(num)],
+ 'int', descr=dyn_calldescr)
+ assert res.value == 2 * num
+
if cpu.supports_floats:
def func(f0, f1, f2, f3, f4, f5, f6, i0, i1, f7, f8, f9):
@@ -507,6 +517,24 @@
'int', descr=calldescr)
assert res.value == func_ints(*args)
+ def test_call_to_c_function(self):
+ from pypy.rlib.libffi import CDLL, types, ArgChain
+ from pypy.rpython.lltypesystem.ll2ctypes import libc_name
+ libc = CDLL(libc_name)
+ c_tolower = libc.getpointer('tolower', [types.uchar], types.sint)
+ argchain = ArgChain().arg(ord('A'))
+ assert c_tolower.call(argchain, rffi.INT) == ord('a')
+
+ func_adr = llmemory.cast_ptr_to_adr(c_tolower.funcsym)
+ funcbox = ConstInt(heaptracker.adr2int(func_adr))
+ calldescr = self.cpu.calldescrof_dynamic([types.uchar], types.sint)
+ res = self.execute_operation(rop.CALL,
+ [funcbox, BoxInt(ord('A'))],
+ 'int',
+ descr=calldescr)
+ assert res.value == ord('a')
+
+
def test_field_basic(self):
t_box, T_box = self.alloc_instance(self.T)
fielddescr = self.cpu.fielddescrof(self.S, 'value')
@@ -833,6 +861,23 @@
length_box], 'void')
assert self.look_string(r_box) == "!??cdef?!"
+ def test_copyunicodecontent(self):
+ s_box = self.alloc_unicode(u"abcdef")
+ for s_box in [s_box, s_box.constbox()]:
+ for srcstart_box in [BoxInt(2), ConstInt(2)]:
+ for dststart_box in [BoxInt(3), ConstInt(3)]:
+ for length_box in [BoxInt(4), ConstInt(4)]:
+ for r_box_is_const in [False, True]:
+ r_box = self.alloc_unicode(u"!???????!")
+ if r_box_is_const:
+ r_box = r_box.constbox()
+ self.execute_operation(rop.COPYUNICODECONTENT,
+ [s_box, r_box,
+ srcstart_box,
+ dststart_box,
+ length_box], 'void')
+ assert self.look_unicode(r_box) == u"!??cdef?!"
+
def test_do_unicode_basic(self):
u = self.cpu.bh_newunicode(5)
self.cpu.bh_unicodesetitem(u, 4, 123)
@@ -1227,6 +1272,10 @@
u_box = BoxPtr(lltype.cast_opaque_ptr(llmemory.GCREF, u))
return u_box
+ def look_unicode(self, unicode_box):
+ u = unicode_box.getref(lltype.Ptr(rstr.UNICODE))
+ return u''.join(u.chars)
+
def test_casts(self):
py.test.skip("xxx fix or kill")
@@ -1283,6 +1332,7 @@
descr=fd)
res = self.execute_operation(get_op, [s_box], 'int', descr=fd)
assert res.getint() == 32
+ lltype.free(s, flavor='raw')
def test_new_with_vtable(self):
cpu = self.cpu
@@ -1427,12 +1477,12 @@
assert not excvalue
def test_cond_call_gc_wb(self):
- def func_void(a):
- record.append(a)
+ def func_void(a, b):
+ record.append((a, b))
record = []
#
S = lltype.GcStruct('S', ('tid', lltype.Signed))
- FUNC = self.FuncType([lltype.Ptr(S)], lltype.Void)
+ FUNC = self.FuncType([lltype.Ptr(S), lltype.Signed], lltype.Void)
func_ptr = llhelper(lltype.Ptr(FUNC), func_void)
funcbox = self.get_funcbox(self.cpu, func_ptr)
class WriteBarrierDescr(AbstractDescr):
@@ -1453,10 +1503,10 @@
sgcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
del record[:]
self.execute_operation(rop.COND_CALL_GC_WB,
- [BoxPtr(sgcref)],
+ [BoxPtr(sgcref), ConstInt(-2121)],
'void', descr=WriteBarrierDescr())
if cond:
- assert record == [s]
+ assert record == [(s, -2121)]
else:
assert record == []
@@ -1980,6 +2030,196 @@
assert self.cpu.get_latest_value_float(0) == 13.5
assert called
+ def test_short_result_of_getfield_direct(self):
+ # Test that a getfield that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ descrfld_x = cpu.fielddescrof(S, 'x')
+ s = lltype.malloc(S)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ s.x = rffi.cast(RESTYPE, value)
+ x = cpu.bh_getfield_gc_i(lltype.cast_opaque_ptr(llmemory.GCREF, s),
+ descrfld_x)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_getfield_compiled(self):
+ # Test that a getfield that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ S = lltype.GcStruct('S', ('x', RESTYPE))
+ descrfld_x = cpu.fielddescrof(S, 'x')
+ s = lltype.malloc(S)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ s.x = rffi.cast(RESTYPE, value)
+ s_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, s)
+ res = self.execute_operation(rop.GETFIELD_GC, [BoxPtr(s_gcref)],
+ 'int', descr=descrfld_x)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
+ def test_short_result_of_getarrayitem_direct(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = lltype.GcArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ x = cpu.bh_getarrayitem_gc_i(
+ descrarray, lltype.cast_opaque_ptr(llmemory.GCREF, a), 3)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_getarrayitem_compiled(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = lltype.GcArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, a)
+ res = self.execute_operation(rop.GETARRAYITEM_GC,
+ [BoxPtr(a_gcref), BoxInt(3)],
+ 'int', descr=descrarray)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
+ def test_short_result_of_getarrayitem_raw_direct(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Direct bh_xxx test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = rffi.CArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5, flavor='raw')
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_rawint = heaptracker.adr2int(llmemory.cast_ptr_to_adr(a))
+ x = cpu.bh_getarrayitem_raw_i(descrarray, a_rawint, 3)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+ lltype.free(a, flavor='raw')
+
+ def test_short_result_of_getarrayitem_raw_compiled(self):
+ # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed
+ # or unsigned, properly gets zero-extended or sign-extended.
+ # Machine code compilation test.
+ cpu = self.cpu
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ A = rffi.CArray(RESTYPE)
+ descrarray = cpu.arraydescrof(A)
+ a = lltype.malloc(A, 5, flavor='raw')
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value))
+ a[3] = rffi.cast(RESTYPE, value)
+ a_rawint = heaptracker.adr2int(llmemory.cast_ptr_to_adr(a))
+ res = self.execute_operation(rop.GETARRAYITEM_RAW,
+ [BoxInt(a_rawint), BoxInt(3)],
+ 'int', descr=descrarray)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+ lltype.free(a, flavor='raw')
+
+ def test_short_result_of_call_direct(self):
+ # Test that calling a function that returns a CHAR, SHORT or INT,
+ # signed or unsigned, properly gets zero-extended or sign-extended.
+ from pypy.translator.tool.cbuild import ExternalCompilationInfo
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ # Tested with a function that intentionally does not cast the
+ # result to RESTYPE, but makes sure that we return the whole
+ # value in eax or rax.
+ eci = ExternalCompilationInfo(separate_module_sources=["""
+ long fn_test_result_of_call(long x)
+ {
+ return x + 1;
+ }
+ """])
+ f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed],
+ RESTYPE, compilation_info=eci, _nowrapper=True)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1))
+ assert intmask(f(value)) == expected
+ #
+ FUNC = self.FuncType([lltype.Signed], RESTYPE)
+ FPTR = self.Ptr(FUNC)
+ calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+ x = self.cpu.bh_call_i(self.get_funcbox(self.cpu, f).value,
+ calldescr, [value], None, None)
+ assert x == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, x, expected))
+
+ def test_short_result_of_call_compiled(self):
+ # Test that calling a function that returns a CHAR, SHORT or INT,
+ # signed or unsigned, properly gets zero-extended or sign-extended.
+ from pypy.translator.tool.cbuild import ExternalCompilationInfo
+ for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR,
+ rffi.SHORT, rffi.USHORT,
+ rffi.INT, rffi.UINT,
+ rffi.LONG, rffi.ULONG]:
+ # Tested with a function that intentionally does not cast the
+ # result to RESTYPE, but makes sure that we return the whole
+ # value in eax or rax.
+ eci = ExternalCompilationInfo(separate_module_sources=["""
+ long fn_test_result_of_call(long x)
+ {
+ return x + 1;
+ }
+ """])
+ f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed],
+ RESTYPE, compilation_info=eci, _nowrapper=True)
+ value = intmask(0xFFEEDDCCBBAA9988)
+ expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1))
+ assert intmask(f(value)) == expected
+ #
+ FUNC = self.FuncType([lltype.Signed], RESTYPE)
+ FPTR = self.Ptr(FUNC)
+ calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT)
+ funcbox = self.get_funcbox(self.cpu, f)
+ res = self.execute_operation(rop.CALL, [funcbox, BoxInt(value)],
+ 'int', descr=calldescr)
+ assert res.value == expected, (
+ "%r: got %r, expected %r" % (RESTYPE, res.value, expected))
+
class OOtypeBackendTest(BaseBackendTest):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/test/test_ll_random.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/test/test_ll_random.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/test/test_ll_random.py Sat Oct 30 16:41:17 2010
@@ -386,6 +386,20 @@
v_string = self.get_string(builder, r)
builder.do(self.opnum, [v_string])
+class AbstractCopyContentOperation(AbstractStringOperation):
+ def produce_into(self, builder, r):
+ v_srcstring = self.get_string(builder, r)
+ v_dststring = self.get_string(builder, r)
+ if v_srcstring.value == v_dststring.value: # because it's not a
+ raise test_random.CannotProduceOperation # memmove(), but memcpy()
+ srclen = len(v_srcstring.getref(self.ptr).chars)
+ dstlen = len(v_dststring.getref(self.ptr).chars)
+ v_length = builder.get_index(min(srclen, dstlen), r)
+ v_srcstart = builder.get_index(srclen - v_length.value + 1, r)
+ v_dststart = builder.get_index(dstlen - v_length.value + 1, r)
+ builder.do(self.opnum, [v_srcstring, v_dststring,
+ v_srcstart, v_dststart, v_length])
+
class StrGetItemOperation(AbstractGetItemOperation, _StrOperation):
pass
@@ -404,6 +418,13 @@
class UnicodeLenOperation(AbstractStringLenOperation, _UnicodeOperation):
pass
+class CopyStrContentOperation(AbstractCopyContentOperation, _StrOperation):
+ pass
+
+class CopyUnicodeContentOperation(AbstractCopyContentOperation,
+ _UnicodeOperation):
+ pass
+
# there are five options in total:
# 1. non raising call and guard_no_exception
@@ -577,6 +598,8 @@
OPERATIONS.append(UnicodeSetItemOperation(rop.UNICODESETITEM))
OPERATIONS.append(StrLenOperation(rop.STRLEN))
OPERATIONS.append(UnicodeLenOperation(rop.UNICODELEN))
+ OPERATIONS.append(CopyStrContentOperation(rop.COPYSTRCONTENT))
+ OPERATIONS.append(CopyUnicodeContentOperation(rop.COPYUNICODECONTENT))
for i in range(2):
OPERATIONS.append(GuardClassOperation(rop.GUARD_CLASS))
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/test/test_random.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/test/test_random.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/test/test_random.py Sat Oct 30 16:41:17 2010
@@ -1,7 +1,7 @@
import py, sys
from pypy.rlib.rarithmetic import intmask, LONG_BIT
from pypy.rpython.lltypesystem import llmemory
-from pypy.jit.backend.test import conftest as demo_conftest
+from pypy.jit.backend import conftest as demo_conftest
from pypy.jit.metainterp.history import BasicFailDescr, TreeLoop
from pypy.jit.metainterp.history import BoxInt, ConstInt, LoopToken
from pypy.jit.metainterp.history import BoxPtr, ConstPtr
@@ -102,7 +102,7 @@
elif isinstance(v, ConstFloat):
args.append('ConstFloat(%r)' % v.value)
elif isinstance(v, ConstInt):
- args.append('ConstInt(%d)' % v.value)
+ args.append('ConstInt(%s)' % v.value)
else:
raise NotImplementedError(v)
if op.getdescr() is None:
@@ -113,7 +113,7 @@
except AttributeError:
descrstr = ', descr=...'
print >>s, ' ResOperation(rop.%s, [%s], %s%s),' % (
- opname[op.opnum], ', '.join(args), names[op.result], descrstr)
+ opname[op.getopnum()], ', '.join(args), names[op.result], descrstr)
#if getattr(op, 'suboperations', None) is not None:
# subops.append(op)
@@ -189,7 +189,7 @@
v.value)
print >>s, ' op = cpu.execute_token(looptoken)'
if self.should_fail_by is None:
- fail_args = self.loop.operations[-1].args
+ fail_args = self.loop.operations[-1].getarglist()
else:
fail_args = self.should_fail_by.getfailargs()
for i, v in enumerate(fail_args):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/assembler.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/assembler.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/assembler.py Sat Oct 30 16:41:17 2010
@@ -8,7 +8,8 @@
from pypy.rpython.annlowlevel import llhelper
from pypy.tool.uid import fixid
from pypy.jit.backend.x86.regalloc import (RegAlloc, X86RegisterManager,
- X86XMMRegisterManager, get_ebp_ofs)
+ X86XMMRegisterManager, get_ebp_ofs,
+ _get_scale)
from pypy.jit.backend.x86.arch import (FRAME_FIXED_SIZE, FORCE_INDEX_OFS, WORD,
IS_X86_32, IS_X86_64)
@@ -22,7 +23,8 @@
X86_64_SCRATCH_REG,
X86_64_XMM_SCRATCH_REG,
RegLoc, StackLoc, ConstFloatLoc,
- ImmedLoc, AddressLoc, imm)
+ ImmedLoc, AddressLoc, imm,
+ imm0, imm1)
from pypy.rlib.objectmodel import we_are_translated, specialize
from pypy.jit.backend.x86 import rx86, regloc, codebuf
@@ -244,12 +246,13 @@
f = open_file_as_stream(output_log, "w")
for i in range(len(self.loop_run_counters)):
name, struct = self.loop_run_counters[i]
- f.write(str(struct.i) + " " * (8 - len(str(struct.i))) + name + "\n")
+ f.write(str(name) + ":" + str(struct.i) + "\n")
f.close()
def _build_float_constants(self):
# 44 bytes: 32 bytes for the data, and up to 12 bytes for alignment
- addr = lltype.malloc(rffi.CArray(lltype.Char), 44, flavor='raw')
+ addr = lltype.malloc(rffi.CArray(lltype.Char), 44, flavor='raw',
+ track_allocation=False)
if not we_are_translated():
self._keepalive_malloced_float_consts = addr
float_constants = rffi.cast(lltype.Signed, addr)
@@ -292,7 +295,7 @@
self.mc.RET()
self.mc.done()
- def assemble_loop(self, inputargs, operations, looptoken):
+ def assemble_loop(self, inputargs, operations, looptoken, log):
"""adds the following attributes to looptoken:
_x86_loop_code (an integer giving an address)
_x86_bootstrap_code (an integer giving an address)
@@ -307,10 +310,11 @@
self.setup()
funcname = self._find_debug_merge_point(operations)
-
+ if log:
+ self._register_counter()
+ operations = self._inject_debugging_code(operations)
regalloc = RegAlloc(self, self.cpu.translate_support_code)
- operations = self._inject_debugging_code(operations)
arglocs = regalloc.prepare_loop(inputargs, operations, looptoken)
looptoken._x86_arglocs = arglocs
@@ -337,13 +341,16 @@
self.mc.end_function()
self.write_pending_failure_recoveries()
- def assemble_bridge(self, faildescr, inputargs, operations):
+ def assemble_bridge(self, faildescr, inputargs, operations, log):
if not we_are_translated():
# Arguments should be unique
assert len(set(inputargs)) == len(inputargs)
self.setup()
funcname = self._find_debug_merge_point(operations)
+ if log:
+ self._register_counter()
+ operations = self._inject_debugging_code(operations)
arglocs = self.rebuild_faillocs_from_descr(
faildescr._x86_failure_recovery_bytecode)
@@ -398,11 +405,14 @@
else:
funcname = "" % len(self.loop_run_counters)
# invent the counter, so we don't get too confused
+ return funcname
+
+ def _register_counter(self):
if self._debug:
- struct = lltype.malloc(DEBUG_COUNTER, flavor='raw')
+ struct = lltype.malloc(DEBUG_COUNTER, flavor='raw',
+ track_allocation=False) # known to leak
struct.i = 0
- self.loop_run_counters.append((funcname, struct))
- return funcname
+ self.loop_run_counters.append((len(self.loop_run_counters), struct))
def patch_jump_for_descr(self, faildescr, adr_new_target):
adr_jump_offset = faildescr._x86_adr_jump_offset
@@ -440,7 +450,7 @@
# self.mc.PUSH(eax)
# adr = rffi.cast(lltype.Signed, self.loop_run_counters[-1][1])
# self.mc.MOV(eax, heap(adr))
- # self.mc.ADD(eax, imm(1))
+ # self.mc.ADD(eax, imm1)
# self.mc.MOV(heap(adr), eax)
# self.mc.POP(eax)
return operations
@@ -711,7 +721,7 @@
self.regalloc_perform_with_guard(None, guard_op, faillocs, arglocs,
resloc, current_depths)
- def load_effective_addr(self, sizereg, baseofs, scale, result, frm=imm(0)):
+ def load_effective_addr(self, sizereg, baseofs, scale, result, frm=imm0):
self.mc.LEA(result, addr_add(frm, sizereg, baseofs, scale))
def _unaryop(asmop):
@@ -973,28 +983,28 @@
def genop_guard_int_is_true(self, op, guard_op, guard_token, arglocs, resloc):
guard_opnum = guard_op.getopnum()
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
if guard_opnum == rop.GUARD_TRUE:
return self.implement_guard(guard_token, 'Z')
else:
return self.implement_guard(guard_token, 'NZ')
def genop_int_is_true(self, op, arglocs, resloc):
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
rl = resloc.lowest8bits()
self.mc.SET_ir(rx86.Conditions['NE'], rl.value)
self.mc.MOVZX8(resloc, rl)
def genop_guard_int_is_zero(self, op, guard_op, guard_token, arglocs, resloc):
guard_opnum = guard_op.getopnum()
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
if guard_opnum == rop.GUARD_TRUE:
return self.implement_guard(guard_token, 'NZ')
else:
return self.implement_guard(guard_token, 'Z')
def genop_int_is_zero(self, op, arglocs, resloc):
- self.mc.CMP(arglocs[0], imm(0))
+ self.mc.CMP(arglocs[0], imm0)
rl = resloc.lowest8bits()
self.mc.SET_ir(rx86.Conditions['E'], rl.value)
self.mc.MOVZX8(resloc, rl)
@@ -1050,50 +1060,66 @@
assert result_loc is eax
self.call(self.malloc_unicode_func_addr, arglocs, eax)
- def genop_getfield_gc(self, op, arglocs, resloc):
- base_loc, ofs_loc, size_loc = arglocs
- assert isinstance(size_loc, ImmedLoc)
+ # ----------
+
+ def load_from_mem(self, resloc, source_addr, size_loc, sign_loc):
assert isinstance(resloc, RegLoc)
size = size_loc.value
-
- source_addr = AddressLoc(base_loc, ofs_loc)
+ sign = sign_loc.value
if resloc.is_xmm:
self.mc.MOVSD(resloc, source_addr)
+ elif size == WORD:
+ self.mc.MOV(resloc, source_addr)
elif size == 1:
- self.mc.MOVZX8(resloc, source_addr)
+ if sign:
+ self.mc.MOVSX8(resloc, source_addr)
+ else:
+ self.mc.MOVZX8(resloc, source_addr)
elif size == 2:
- self.mc.MOVZX16(resloc, source_addr)
+ if sign:
+ self.mc.MOVSX16(resloc, source_addr)
+ else:
+ self.mc.MOVZX16(resloc, source_addr)
+ elif IS_X86_64 and size == 4:
+ if sign:
+ self.mc.MOVSX32(resloc, source_addr)
+ else:
+ self.mc.MOV32(resloc, source_addr) # zero-extending
+ else:
+ not_implemented("load_from_mem size = %d" % size)
+
+ def save_into_mem(self, dest_addr, value_loc, size_loc):
+ size = size_loc.value
+ if isinstance(value_loc, RegLoc) and value_loc.is_xmm:
+ self.mc.MOVSD(dest_addr, value_loc)
+ elif size == 1:
+ self.mc.MOV8(dest_addr, value_loc.lowest8bits())
+ elif size == 2:
+ self.mc.MOV16(dest_addr, value_loc)
elif size == 4:
- # MOV32 is zero-extending on 64-bit, so this is okay
- self.mc.MOV32(resloc, source_addr)
+ self.mc.MOV32(dest_addr, value_loc)
elif IS_X86_64 and size == 8:
- self.mc.MOV(resloc, source_addr)
+ self.mc.MOV(dest_addr, value_loc)
else:
- raise NotImplementedError("getfield size = %d" % size)
+ not_implemented("save_into_mem size = %d" % size)
+
+ def genop_getfield_gc(self, op, arglocs, resloc):
+ base_loc, ofs_loc, size_loc, sign_loc = arglocs
+ assert isinstance(size_loc, ImmedLoc)
+ source_addr = AddressLoc(base_loc, ofs_loc)
+ self.load_from_mem(resloc, source_addr, size_loc, sign_loc)
genop_getfield_raw = genop_getfield_gc
genop_getfield_raw_pure = genop_getfield_gc
genop_getfield_gc_pure = genop_getfield_gc
def genop_getarrayitem_gc(self, op, arglocs, resloc):
- base_loc, ofs_loc, scale, ofs = arglocs
+ base_loc, ofs_loc, size_loc, ofs, sign_loc = arglocs
assert isinstance(ofs, ImmedLoc)
- assert isinstance(scale, ImmedLoc)
- src_addr = addr_add(base_loc, ofs_loc, ofs.value, scale.value)
- if op.result.type == FLOAT:
- self.mc.MOVSD(resloc, src_addr)
- else:
- if scale.value == 0:
- self.mc.MOVZX8(resloc, src_addr)
- elif scale.value == 1:
- self.mc.MOVZX16(resloc, src_addr)
- elif scale.value == 2:
- self.mc.MOV32(resloc, src_addr)
- elif IS_X86_64 and scale.value == 3:
- self.mc.MOV(resloc, src_addr)
- else:
- print "[asmgen]getarrayitem unsupported size: %d" % scale.value
- raise NotImplementedError()
+ assert isinstance(size_loc, ImmedLoc)
+ scale = _get_scale(size_loc.value)
+ src_addr = addr_add(base_loc, ofs_loc, ofs.value, scale)
+ self.load_from_mem(resloc, src_addr, size_loc, sign_loc)
genop_getarrayitem_gc_pure = genop_getarrayitem_gc
genop_getarrayitem_raw = genop_getarrayitem_gc
@@ -1101,40 +1127,16 @@
def genop_discard_setfield_gc(self, op, arglocs):
base_loc, ofs_loc, size_loc, value_loc = arglocs
assert isinstance(size_loc, ImmedLoc)
- size = size_loc.value
dest_addr = AddressLoc(base_loc, ofs_loc)
- if isinstance(value_loc, RegLoc) and value_loc.is_xmm:
- self.mc.MOVSD(dest_addr, value_loc)
- elif IS_X86_64 and size == 8:
- self.mc.MOV(dest_addr, value_loc)
- elif size == 4:
- self.mc.MOV32(dest_addr, value_loc)
- elif size == 2:
- self.mc.MOV16(dest_addr, value_loc)
- elif size == 1:
- self.mc.MOV8(dest_addr, value_loc.lowest8bits())
- else:
- print "[asmgen]setfield addr size %d" % size
- raise NotImplementedError("Addr size %d" % size)
+ self.save_into_mem(dest_addr, value_loc, size_loc)
def genop_discard_setarrayitem_gc(self, op, arglocs):
- base_loc, ofs_loc, value_loc, scale_loc, baseofs = arglocs
+ base_loc, ofs_loc, value_loc, size_loc, baseofs = arglocs
assert isinstance(baseofs, ImmedLoc)
- assert isinstance(scale_loc, ImmedLoc)
- dest_addr = AddressLoc(base_loc, ofs_loc, scale_loc.value, baseofs.value)
- if op.getarg(2).type == FLOAT:
- self.mc.MOVSD(dest_addr, value_loc)
- else:
- if IS_X86_64 and scale_loc.value == 3:
- self.mc.MOV(dest_addr, value_loc)
- elif scale_loc.value == 2:
- self.mc.MOV32(dest_addr, value_loc)
- elif scale_loc.value == 1:
- self.mc.MOV16(dest_addr, value_loc)
- elif scale_loc.value == 0:
- self.mc.MOV8(dest_addr, value_loc.lowest8bits())
- else:
- raise NotImplementedError("scale = %d" % scale_loc.value)
+ assert isinstance(size_loc, ImmedLoc)
+ scale = _get_scale(size_loc.value)
+ dest_addr = AddressLoc(base_loc, ofs_loc, scale, baseofs.value)
+ self.save_into_mem(dest_addr, value_loc, size_loc)
def genop_discard_strsetitem(self, op, arglocs):
base_loc, ofs_loc, val_loc = arglocs
@@ -1201,7 +1203,7 @@
def genop_guard_guard_no_exception(self, ign_1, guard_op, guard_token,
locs, ign_2):
- self.mc.CMP(heap(self.cpu.pos_exception()), imm(0))
+ self.mc.CMP(heap(self.cpu.pos_exception()), imm0)
return self.implement_guard(guard_token, 'NZ')
def genop_guard_guard_exception(self, ign_1, guard_op, guard_token,
@@ -1213,8 +1215,8 @@
addr = self.implement_guard(guard_token, 'NE')
if resloc is not None:
self.mc.MOV(resloc, heap(self.cpu.pos_exc_value()))
- self.mc.MOV(heap(self.cpu.pos_exception()), imm(0))
- self.mc.MOV(heap(self.cpu.pos_exc_value()), imm(0))
+ self.mc.MOV(heap(self.cpu.pos_exception()), imm0)
+ self.mc.MOV(heap(self.cpu.pos_exc_value()), imm0)
return addr
def _gen_guard_overflow(self, guard_op, guard_token):
@@ -1224,8 +1226,8 @@
elif guard_opnum == rop.GUARD_OVERFLOW:
return self.implement_guard(guard_token, 'NO')
else:
- print "int_xxx_ovf followed by", guard_op.getopname()
- raise AssertionError
+ not_implemented("int_xxx_ovf followed by %s" %
+ guard_op.getopname())
def genop_guard_int_add_ovf(self, op, guard_op, guard_token, arglocs, result_loc):
self.genop_int_add(op, arglocs, result_loc)
@@ -1288,7 +1290,7 @@
def genop_guard_guard_nonnull_class(self, ign_1, guard_op,
guard_token, locs, ign_2):
self.mc.ensure_bytes_available(256)
- self.mc.CMP(locs[0], imm(1))
+ self.mc.CMP(locs[0], imm1)
# Patched below
self.mc.J_il8(rx86.Conditions['B'], 0)
jb_location = self.mc.get_relative_pos()
@@ -1637,25 +1639,34 @@
sizeloc = arglocs[0]
assert isinstance(sizeloc, ImmedLoc)
size = sizeloc.value
+ signloc = arglocs[1]
if isinstance(op.getarg(0), Const):
x = imm(op.getarg(0).getint())
else:
- x = arglocs[1]
+ x = arglocs[2]
if x is eax:
tmp = ecx
else:
tmp = eax
- self._emit_call(x, arglocs, 2, tmp=tmp)
+ self._emit_call(x, arglocs, 3, tmp=tmp)
+
+ if IS_X86_32 and isinstance(resloc, StackLoc) and resloc.width == 8:
+ self.mc.FSTP_b(resloc.value) # float return
+ elif size == WORD:
+ assert resloc is eax or resloc is xmm0 # a full word
+ elif size == 0:
+ pass # void return
+ else:
+ # use the code in load_from_mem to do the zero- or sign-extension
+ assert resloc is eax
+ if size == 1:
+ srcloc = eax.lowest8bits()
+ else:
+ srcloc = eax
+ self.load_from_mem(eax, srcloc, sizeloc, signloc)
- if isinstance(resloc, StackLoc) and resloc.width == 8 and IS_X86_32:
- self.mc.FSTP_b(resloc.value)
- elif size == 1:
- self.mc.AND_ri(eax.value, 0xff)
- elif size == 2:
- self.mc.AND_ri(eax.value, 0xffff)
-
def genop_guard_call_may_force(self, op, guard_op, guard_token,
arglocs, result_loc):
faildescr = guard_op.getdescr()
@@ -1780,11 +1791,12 @@
self.mc.PUSH_i32(loc.getint())
if IS_X86_64:
- # We clobber this register to pass the arguments, but that's
+ # We clobber these registers to pass the arguments, but that's
# okay, because consider_cond_call_gc_wb makes sure that any
# caller-save registers with values in them are present in arglocs,
# so they are saved on the stack above and restored below
self.mc.MOV_rs(edi.value, 0)
+ self.mc.MOV_rs(esi.value, 8)
# misaligned stack in the call, but it's ok because the write barrier
# is not going to call anything more. Also, this assumes that the
@@ -1807,20 +1819,16 @@
self.mc.LEA_rb(resloc.value, FORCE_INDEX_OFS)
def not_implemented_op_discard(self, op, arglocs):
- msg = "not implemented operation: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation: %s" % op.getopname())
def not_implemented_op(self, op, arglocs, resloc):
- msg = "not implemented operation with res: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation with res: %s" %
+ op.getopname())
def not_implemented_op_guard(self, op, guard_op,
failaddr, arglocs, resloc):
- msg = "not implemented operation (guard): %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation (guard): %s" %
+ op.getopname())
def mark_gc_roots(self):
gcrootmap = self.cpu.gc_ll_descr.gcrootmap
@@ -1904,3 +1912,7 @@
def heap(addr):
return AddressLoc(ImmedLoc(addr), ImmedLoc(0), 0, 0)
+
+def not_implemented(msg):
+ os.write(2, '[x86/asm] %s\n' % msg)
+ raise NotImplementedError(msg)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/regalloc.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/regalloc.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/regalloc.py Sat Oct 30 16:41:17 2010
@@ -2,6 +2,7 @@
""" Register allocation scheme.
"""
+import os
from pypy.jit.metainterp.history import (Box, Const, ConstInt, ConstPtr,
ResOperation, BoxPtr,
LoopToken, INT, REF, FLOAT)
@@ -40,12 +41,10 @@
return imm(c.value)
elif isinstance(c, ConstPtr):
if we_are_translated() and c.value and rgc.can_move(c.value):
- print "convert_to_imm: ConstPtr needs special care"
- raise AssertionError
+ not_implemented("convert_to_imm: ConstPtr needs special care")
return imm(rffi.cast(lltype.Signed, c.value))
else:
- print "convert_to_imm: got a %s" % c
- raise AssertionError
+ not_implemented("convert_to_imm: got a %s" % c)
class X86_64_RegisterManager(X86RegisterManager):
# r11 omitted because it's used as scratch
@@ -70,8 +69,9 @@
def _get_new_array(self):
n = self.BASE_CONSTANT_SIZE
+ # known to leak
self.cur_array = lltype.malloc(rffi.CArray(lltype.Float), n,
- flavor='raw')
+ flavor='raw', track_allocation=False)
self.cur_array_free = n
_get_new_array._dont_inline_ = True
@@ -359,8 +359,8 @@
if op.is_ovf():
if (operations[i + 1].getopnum() != rop.GUARD_NO_OVERFLOW and
operations[i + 1].getopnum() != rop.GUARD_OVERFLOW):
- print "int_xxx_ovf not followed by guard_(no)_overflow"
- raise AssertionError
+ not_implemented("int_xxx_ovf not followed by "
+ "guard_(no)_overflow")
return True
return False
if (operations[i + 1].getopnum() != rop.GUARD_TRUE and
@@ -412,8 +412,8 @@
arg = op.getarg(j)
if isinstance(arg, Box):
if arg not in start_live:
- print "Bogus arg in operation %d at %d" % (op.getopnum(), i)
- raise AssertionError
+ not_implemented("Bogus arg in operation %d at %d" %
+ (op.getopnum(), i))
longevity[arg] = (start_live[arg], i)
if op.is_guard():
for arg in op.getfailargs():
@@ -421,8 +421,8 @@
continue
assert isinstance(arg, Box)
if arg not in start_live:
- print "Bogus arg in guard %d at %d" % (op.getopnum(), i)
- raise AssertionError
+ not_implemented("Bogus arg in guard %d at %d" %
+ (op.getopnum(), i))
longevity[arg] = (start_live[arg], i)
for arg in inputargs:
if arg not in longevity:
@@ -667,7 +667,13 @@
assert isinstance(calldescr, BaseCallDescr)
assert len(calldescr.arg_classes) == op.numargs() - 1
size = calldescr.get_result_size(self.translate_support_code)
- self._call(op, [imm(size)] + [self.loc(op.getarg(i)) for i in range(op.numargs())],
+ sign = calldescr.is_result_signed()
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self._call(op, [imm(size), sign_loc] +
+ [self.loc(op.getarg(i)) for i in range(op.numargs())],
guard_not_forced_op=guard_not_forced_op)
def consider_call(self, op):
@@ -688,7 +694,7 @@
self.rm._sync_var(op.getarg(vable_index))
vable = self.fm.loc(op.getarg(vable_index))
else:
- vable = imm(0)
+ vable = imm0
self._call(op, [imm(size), vable] +
[self.loc(op.getarg(i)) for i in range(op.numargs())],
guard_not_forced_op=guard_op)
@@ -696,9 +702,13 @@
def consider_cond_call_gc_wb(self, op):
assert op.result is None
args = op.getarglist()
+ loc_newvalue = self.rm.make_sure_var_in_reg(op.getarg(1), args)
+ # ^^^ we force loc_newvalue in a reg (unless it's a Const),
+ # because it will be needed anyway by the following setfield_gc.
+ # It avoids loading it twice from the memory.
loc_base = self.rm.make_sure_var_in_reg(op.getarg(0), args,
imm_fine=False)
- arglocs = [loc_base]
+ arglocs = [loc_base, loc_newvalue]
# add eax, ecx and edx as extra "arguments" to ensure they are
# saved and restored. Fish in self.rm to know which of these
# registers really need to be saved (a bit of a hack). Moreover,
@@ -774,15 +784,11 @@
loc = self.loc(op.getarg(0))
return self._call(op, [loc])
# boehm GC (XXX kill the following code at some point)
- ofs_items, itemsize, ofs = symbolic.get_array_token(rstr.UNICODE, self.translate_support_code)
- if itemsize == 4:
- return self._malloc_varsize(ofs_items, ofs, 2, op.getarg(0),
- op.result)
- elif itemsize == 2:
- return self._malloc_varsize(ofs_items, ofs, 1, op.getarg(0),
- op.result)
- else:
- assert False, itemsize
+ ofs_items, _, ofs = symbolic.get_array_token(rstr.UNICODE,
+ self.translate_support_code)
+ scale = self._get_unicode_item_scale()
+ return self._malloc_varsize(ofs_items, ofs, scale, op.getarg(0),
+ op.result)
def _malloc_varsize(self, ofs_items, ofs_length, scale, v, res_v):
# XXX kill this function at some point
@@ -814,8 +820,9 @@
arglocs.append(self.loc(op.getarg(0)))
return self._call(op, arglocs)
# boehm GC (XXX kill the following code at some point)
- scale_of_field, basesize, ofs_length, _ = (
+ itemsize, basesize, ofs_length, _, _ = (
self._unpack_arraydescr(op.getdescr()))
+ scale_of_field = _get_scale(itemsize)
return self._malloc_varsize(basesize, ofs_length, scale_of_field,
op.getarg(0), op.result)
@@ -825,21 +832,19 @@
ofs = arraydescr.get_base_size(self.translate_support_code)
size = arraydescr.get_item_size(self.translate_support_code)
ptr = arraydescr.is_array_of_pointers()
- scale = 0
- while (1 << scale) < size:
- scale += 1
- assert (1 << scale) == size
- return scale, ofs, ofs_length, ptr
+ sign = arraydescr.is_item_signed()
+ return size, ofs, ofs_length, ptr, sign
def _unpack_fielddescr(self, fielddescr):
assert isinstance(fielddescr, BaseFieldDescr)
ofs = fielddescr.offset
size = fielddescr.get_field_size(self.translate_support_code)
ptr = fielddescr.is_pointer_field()
- return imm(ofs), imm(size), ptr
+ sign = fielddescr.is_field_signed()
+ return imm(ofs), imm(size), ptr, sign
def consider_setfield_gc(self, op):
- ofs_loc, size_loc, ptr = self._unpack_fielddescr(op.getdescr())
+ ofs_loc, size_loc, _, _ = self._unpack_fielddescr(op.getdescr())
assert isinstance(size_loc, ImmedLoc)
if size_loc.value == 1:
need_lower_byte = True
@@ -866,10 +871,10 @@
consider_unicodesetitem = consider_strsetitem
def consider_setarrayitem_gc(self, op):
- scale, ofs, _, ptr = self._unpack_arraydescr(op.getdescr())
+ itemsize, ofs, _, _, _ = self._unpack_arraydescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
- if scale == 0:
+ if itemsize == 1:
need_lower_byte = True
else:
need_lower_byte = False
@@ -878,30 +883,39 @@
ofs_loc = self.rm.make_sure_var_in_reg(op.getarg(1), args)
self.possibly_free_vars(args)
self.PerformDiscard(op, [base_loc, ofs_loc, value_loc,
- imm(scale), imm(ofs)])
+ imm(itemsize), imm(ofs)])
consider_setarrayitem_raw = consider_setarrayitem_gc
def consider_getfield_gc(self, op):
- ofs_loc, size_loc, _ = self._unpack_fielddescr(op.getdescr())
+ ofs_loc, size_loc, _, sign = self._unpack_fielddescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
self.rm.possibly_free_vars(args)
result_loc = self.force_allocate_reg(op.result)
- self.Perform(op, [base_loc, ofs_loc, size_loc], result_loc)
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self.Perform(op, [base_loc, ofs_loc, size_loc, sign_loc], result_loc)
consider_getfield_raw = consider_getfield_gc
consider_getfield_raw_pure = consider_getfield_gc
consider_getfield_gc_pure = consider_getfield_gc
def consider_getarrayitem_gc(self, op):
- scale, ofs, _, _ = self._unpack_arraydescr(op.getdescr())
+ itemsize, ofs, _, _, sign = self._unpack_arraydescr(op.getdescr())
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args)
ofs_loc = self.rm.make_sure_var_in_reg(op.getarg(1), args)
self.rm.possibly_free_vars_for_op(op)
result_loc = self.force_allocate_reg(op.result)
- self.Perform(op, [base_loc, ofs_loc, imm(scale), imm(ofs)], result_loc)
+ if sign:
+ sign_loc = imm1
+ else:
+ sign_loc = imm0
+ self.Perform(op, [base_loc, ofs_loc, imm(itemsize), imm(ofs),
+ sign_loc], result_loc)
consider_getarrayitem_raw = consider_getarrayitem_gc
consider_getarrayitem_gc_pure = consider_getarrayitem_gc
@@ -955,41 +969,85 @@
consider_unicodegetitem = consider_strgetitem
def consider_copystrcontent(self, op):
+ self._consider_copystrcontent(op, is_unicode=False)
+
+ def consider_copyunicodecontent(self, op):
+ self._consider_copystrcontent(op, is_unicode=True)
+
+ def _consider_copystrcontent(self, op, is_unicode):
# compute the source address
args = op.getarglist()
base_loc = self.rm.make_sure_var_in_reg(args[0], args)
ofs_loc = self.rm.make_sure_var_in_reg(args[2], args)
+ assert args[0] is not args[1] # forbidden case of aliasing
self.rm.possibly_free_var(args[0])
- self.rm.possibly_free_var(args[2])
+ if args[3] is not args[2] is not args[4]: # MESS MESS MESS: don't free
+ self.rm.possibly_free_var(args[2]) # it if ==args[3] or args[4]
srcaddr_box = TempBox()
- srcaddr_loc = self.rm.force_allocate_reg(srcaddr_box)
- self._gen_address_inside_string(base_loc, ofs_loc, srcaddr_loc)
+ forbidden_vars = [args[1], args[3], args[4], srcaddr_box]
+ srcaddr_loc = self.rm.force_allocate_reg(srcaddr_box, forbidden_vars)
+ self._gen_address_inside_string(base_loc, ofs_loc, srcaddr_loc,
+ is_unicode=is_unicode)
# compute the destination address
- base_loc = self.rm.make_sure_var_in_reg(args[1], args)
- ofs_loc = self.rm.make_sure_var_in_reg(args[3], args)
+ base_loc = self.rm.make_sure_var_in_reg(args[1], forbidden_vars)
+ ofs_loc = self.rm.make_sure_var_in_reg(args[3], forbidden_vars)
self.rm.possibly_free_var(args[1])
- self.rm.possibly_free_var(args[3])
+ if args[3] is not args[4]: # more of the MESS described above
+ self.rm.possibly_free_var(args[3])
+ forbidden_vars = [args[4], srcaddr_box]
dstaddr_box = TempBox()
- dstaddr_loc = self.rm.force_allocate_reg(dstaddr_box)
- self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc)
+ dstaddr_loc = self.rm.force_allocate_reg(dstaddr_box, forbidden_vars)
+ self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc,
+ is_unicode=is_unicode)
+ # compute the length in bytes
+ length_box = args[4]
+ length_loc = self.loc(length_box)
+ if is_unicode:
+ self.rm.possibly_free_var(length_box)
+ forbidden_vars = [srcaddr_box, dstaddr_box]
+ bytes_box = TempBox()
+ bytes_loc = self.rm.force_allocate_reg(bytes_box, forbidden_vars)
+ scale = self._get_unicode_item_scale()
+ if not (isinstance(length_loc, ImmedLoc) or
+ isinstance(length_loc, RegLoc)):
+ self.assembler.mov(length_loc, bytes_loc)
+ length_loc = bytes_loc
+ self.assembler.load_effective_addr(length_loc, 0, scale, bytes_loc)
+ length_box = bytes_box
+ length_loc = bytes_loc
# call memcpy()
- length_loc = self.loc(args[4])
self.rm.before_call()
self.xrm.before_call()
self.assembler._emit_call(imm(self.assembler.memcpy_addr),
[dstaddr_loc, srcaddr_loc, length_loc])
- self.rm.possibly_free_var(args[4])
+ self.rm.possibly_free_var(length_box)
self.rm.possibly_free_var(dstaddr_box)
self.rm.possibly_free_var(srcaddr_box)
- def _gen_address_inside_string(self, baseloc, ofsloc, resloc):
+ def _gen_address_inside_string(self, baseloc, ofsloc, resloc, is_unicode):
cpu = self.assembler.cpu
- ofs_items, itemsize, _ = symbolic.get_array_token(rstr.STR,
+ if is_unicode:
+ ofs_items, _, _ = symbolic.get_array_token(rstr.UNICODE,
self.translate_support_code)
- assert itemsize == 1
- self.assembler.load_effective_addr(ofsloc, ofs_items, 0,
+ scale = self._get_unicode_item_scale()
+ else:
+ ofs_items, itemsize, _ = symbolic.get_array_token(rstr.STR,
+ self.translate_support_code)
+ assert itemsize == 1
+ scale = 0
+ self.assembler.load_effective_addr(ofsloc, ofs_items, scale,
resloc, baseloc)
+ def _get_unicode_item_scale(self):
+ _, itemsize, _ = symbolic.get_array_token(rstr.UNICODE,
+ self.translate_support_code)
+ if itemsize == 4:
+ return 2
+ elif itemsize == 2:
+ return 1
+ else:
+ raise AssertionError("bad unicode item size")
+
def consider_jump(self, op):
assembler = self.assembler
assert self.jump_target_descr is None
@@ -1024,6 +1082,9 @@
def consider_debug_merge_point(self, op):
pass
+ def consider_jit_debug(self, op):
+ pass
+
def get_mark_gc_roots(self, gcrootmap):
shape = gcrootmap.get_basic_shape(IS_X86_64)
for v, val in self.fm.frame_bindings.items():
@@ -1043,15 +1104,11 @@
self.Perform(op, [], loc)
def not_implemented_op(self, op):
- msg = "[regalloc] Not implemented operation: %s" % op.getopname()
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation: %s" % op.getopname())
def not_implemented_op_with_guard(self, op, guard_op):
- msg = "[regalloc] Not implemented operation with guard: %s" % (
- op.getopname(),)
- print msg
- raise NotImplementedError(msg)
+ not_implemented("not implemented operation with guard: %s" % (
+ op.getopname(),))
oplist = [RegAlloc.not_implemented_op] * rop._LAST
oplist_with_guard = [RegAlloc.not_implemented_op_with_guard] * rop._LAST
@@ -1085,3 +1142,14 @@
# Returns (ebp-20), (ebp-24), (ebp-28)...
# i.e. the n'th word beyond the fixed frame size.
return -WORD * (FRAME_FIXED_SIZE + position)
+
+def _get_scale(size):
+ assert size == 1 or size == 2 or size == 4 or size == 8
+ if size < 4:
+ return size - 1 # 1, 2 => 0, 1
+ else:
+ return (size >> 2) + 1 # 4, 8 => 2, 3
+
+def not_implemented(msg):
+ os.write(2, '[x86/regalloc] %s\n' % msg)
+ raise NotImplementedError(msg)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/regloc.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/regloc.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/regloc.py Sat Oct 30 16:41:17 2010
@@ -442,8 +442,11 @@
MOV8 = _binaryop('MOV8')
MOV16 = _16_bit_binaryop('MOV')
MOVZX8 = _binaryop('MOVZX8')
+ MOVSX8 = _binaryop('MOVSX8')
MOVZX16 = _binaryop('MOVZX16')
+ MOVSX16 = _binaryop('MOVSX16')
MOV32 = _binaryop('MOV32')
+ MOVSX32 = _binaryop('MOVSX32')
XCHG = _binaryop('XCHG')
PUSH = _unaryop('PUSH')
@@ -473,6 +476,9 @@
else:
return ImmedLoc(x)
+imm0 = imm(0)
+imm1 = imm(1)
+
all_extra_instructions = [name for name in LocationCodeBuilder.__dict__
if name[0].isupper()]
all_extra_instructions.sort()
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/runner.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/runner.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/runner.py Sat Oct 30 16:41:17 2010
@@ -49,11 +49,13 @@
self.assembler.finish_once()
self.profile_agent.shutdown()
- def compile_loop(self, inputargs, operations, looptoken):
- self.assembler.assemble_loop(inputargs, operations, looptoken)
-
- def compile_bridge(self, faildescr, inputargs, operations):
- self.assembler.assemble_bridge(faildescr, inputargs, operations)
+ def compile_loop(self, inputargs, operations, looptoken, log=True):
+ self.assembler.assemble_loop(inputargs, operations, looptoken,
+ log=log)
+
+ def compile_bridge(self, faildescr, inputargs, operations, log=True):
+ self.assembler.assemble_bridge(faildescr, inputargs, operations,
+ log=log)
def set_future_value_int(self, index, intvalue):
self.assembler.fail_boxes_int.setitem(index, intvalue)
@@ -87,7 +89,9 @@
def execute_token(self, executable_token):
addr = executable_token._x86_bootstrap_code
+ #llop.debug_print(lltype.Void, ">>>> Entering", addr)
func = rffi.cast(lltype.Ptr(self.BOOTSTRAP_TP), addr)
+ #llop.debug_print(lltype.Void, "<<<< Back")
fail_index = self._execute_call(func)
return self.get_fail_descr_from_number(fail_index)
@@ -99,10 +103,7 @@
LLInterpreter.current_interpreter = self.debug_ll_interpreter
res = 0
try:
- #llop.debug_print(lltype.Void, ">>>> Entering",
- # rffi.cast(lltype.Signed, func))
res = func()
- #llop.debug_print(lltype.Void, "<<<< Back")
finally:
if not self.translate_support_code:
LLInterpreter.current_interpreter = prev_interpreter
@@ -114,7 +115,8 @@
return CPU386.cast_adr_to_int(adr)
all_null_registers = lltype.malloc(rffi.LONGP.TO, 24,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
def force(self, addr_of_force_index):
TP = rffi.CArrayPtr(lltype.Signed)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/rx86.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/rx86.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/rx86.py Sat Oct 30 16:41:17 2010
@@ -642,7 +642,10 @@
define_modrm_modes('MOV8_*i', [rex_w, '\xC6', orbyte(0<<3)], [immediate(2, 'b')], regtype='BYTE')
define_modrm_modes('MOVZX8_r*', [rex_w, '\x0F\xB6', register(1, 8)], regtype='BYTE')
+define_modrm_modes('MOVSX8_r*', [rex_w, '\x0F\xBE', register(1, 8)], regtype='BYTE')
define_modrm_modes('MOVZX16_r*', [rex_w, '\x0F\xB7', register(1, 8)])
+define_modrm_modes('MOVSX16_r*', [rex_w, '\x0F\xBF', register(1, 8)])
+define_modrm_modes('MOVSX32_r*', [rex_w, '\x63', register(1, 8)])
define_modrm_modes('MOVSD_x*', ['\xF2', rex_nw, '\x0F\x10', register(1,8)], regtype='XMM')
define_modrm_modes('MOVSD_*x', ['\xF2', rex_nw, '\x0F\x11', register(2,8)], regtype='XMM')
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_assembler.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_assembler.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_assembler.py Sat Oct 30 16:41:17 2010
@@ -81,7 +81,8 @@
# also test rebuild_faillocs_from_descr(), which should not
# reproduce the holes at all
- bytecode = lltype.malloc(rffi.UCHARP.TO, len(mc.content), flavor='raw')
+ bytecode = lltype.malloc(rffi.UCHARP.TO, len(mc.content), flavor='raw',
+ immortal=True)
for i in range(len(mc.content)):
assert 0 <= mc.content[i] <= 255
bytecode[i] = rffi.cast(rffi.UCHAR, mc.content[i])
@@ -115,7 +116,8 @@
assert withfloats
value = random.random() - 0.5
# make sure it fits into 64 bits
- tmp = lltype.malloc(rffi.LONGP.TO, 2, flavor='raw')
+ tmp = lltype.malloc(rffi.LONGP.TO, 2, flavor='raw',
+ track_allocation=False)
rffi.cast(rffi.DOUBLEP, tmp)[0] = value
return rffi.cast(rffi.DOUBLEP, tmp)[0], tmp[0], tmp[1]
@@ -152,10 +154,12 @@
# prepare the expected target arrays, the descr_bytecode,
# the 'registers' and the 'stack' arrays according to 'content'
- xmmregisters = lltype.malloc(rffi.LONGP.TO, 16+ACTUAL_CPU.NUM_REGS+1, flavor='raw')
+ xmmregisters = lltype.malloc(rffi.LONGP.TO, 16+ACTUAL_CPU.NUM_REGS+1,
+ flavor='raw', immortal=True)
registers = rffi.ptradd(xmmregisters, 16)
stacklen = baseloc + 10
- stack = lltype.malloc(rffi.LONGP.TO, stacklen, flavor='raw')
+ stack = lltype.malloc(rffi.LONGP.TO, stacklen, flavor='raw',
+ immortal=True)
expected_ints = [0] * len(content)
expected_ptrs = [lltype.nullptr(llmemory.GCREF.TO)] * len(content)
expected_floats = [0.0] * len(content)
@@ -221,7 +225,7 @@
descr_bytecode.append(0x00)
descr_bytecode.append(0xCC) # end marker
descr_bytes = lltype.malloc(rffi.UCHARP.TO, len(descr_bytecode),
- flavor='raw')
+ flavor='raw', immortal=True)
for i in range(len(descr_bytecode)):
assert 0 <= descr_bytecode[i] <= 255
descr_bytes[i] = rffi.cast(rffi.UCHAR, descr_bytecode[i])
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_gc_integration.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_gc_integration.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_gc_integration.py Sat Oct 30 16:41:17 2010
@@ -12,7 +12,7 @@
from pypy.jit.backend.detect_cpu import getcpuclass
from pypy.jit.backend.x86.regalloc import RegAlloc
from pypy.jit.backend.x86.arch import WORD, FRAME_FIXED_SIZE
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.rpython.lltypesystem import lltype, llmemory, rffi
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.lltypesystem import rclass, rstr
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_regalloc.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_regalloc.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_regalloc.py Sat Oct 30 16:41:17 2010
@@ -11,7 +11,7 @@
from pypy.jit.backend.x86.regalloc import RegAlloc, X86RegisterManager,\
FloatConstants, is_comparison_or_ovf_op
from pypy.jit.backend.x86.arch import IS_X86_32, IS_X86_64
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.rpython.lltypesystem import lltype, llmemory, rffi
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.lltypesystem import rclass, rstr
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_runner.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_runner.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_runner.py Sat Oct 30 16:41:17 2010
@@ -10,7 +10,7 @@
from pypy.jit.metainterp.resoperation import rop
from pypy.jit.metainterp.executor import execute
from pypy.jit.backend.test.runner_test import LLtypeBackendTest
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.tool.udir import udir
import ctypes
import sys
@@ -506,8 +506,8 @@
self.cpu.execute_token(ops.token)
# check debugging info
name, struct = self.cpu.assembler.loop_run_counters[0]
- assert name == 'xyz'
+ assert name == 0 # 'xyz'
assert struct.i == 10
self.cpu.finish_once()
lines = py.path.local(self.logfile + ".count").readlines()
- assert lines[0] == '10 xyz\n'
+ assert lines[0] == '0:10\n' # '10 xyz\n'
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_string.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_string.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_string.py Sat Oct 30 16:41:17 2010
@@ -2,8 +2,12 @@
from pypy.jit.metainterp.test import test_string
from pypy.jit.backend.x86.test.test_basic import Jit386Mixin
-class TestString(Jit386Mixin, test_string.StringTests):
+class TestString(Jit386Mixin, test_string.TestLLtype):
# for the individual tests see
# ====> ../../../metainterp/test/test_string.py
- CALL = 'call'
- CALL_PURE = 'call_pure'
+ pass
+
+class TestUnicode(Jit386Mixin, test_string.TestLLtypeUnicode):
+ # for the individual tests see
+ # ====> ../../../metainterp/test/test_string.py
+ pass
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_zrpy_gc.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_zrpy_gc.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_zrpy_gc.py Sat Oct 30 16:41:17 2010
@@ -191,6 +191,33 @@
def run_orig(self, name, n, x):
self.main_allfuncs(name, n, x)
+ def define_libffi_workaround(cls):
+ # XXX: this is a workaround for a bug in database.py. It seems that
+ # the problem is triggered by optimizeopt/fficall.py, and in
+ # particular by the ``cast_base_ptr_to_instance(Func, llfunc)``: in
+ # these tests, that line is the only place where libffi.Func is
+ # referenced.
+ #
+ # The problem occurs because the gctransformer tries to annotate a
+ # low-level helper to call the __del__ of libffi.Func when it's too
+ # late.
+ #
+ # This workaround works by forcing the annotator (and all the rest of
+ # the toolchain) to see libffi.Func in a "proper" context, not just as
+ # the target of cast_base_ptr_to_instance. Note that the function
+ # below is *never* called by any actual test, it's just annotated.
+ #
+ from pypy.rlib.libffi import get_libc_name, CDLL, types, ArgChain
+ libc_name = get_libc_name()
+ def f(n, x, *args):
+ libc = CDLL(libc_name)
+ ptr = libc.getpointer('labs', [types.slong], types.slong)
+ chain = ArgChain()
+ chain.arg(n)
+ n = ptr.call(chain, lltype.Signed)
+ return (n, x) + args
+ return None, f, None
+
def define_compile_framework_1(cls):
# a moving GC. Supports malloc_varsize_nonmovable. Simple test, works
# without write_barriers and root stack enumeration.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_ztranslation.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_ztranslation.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_ztranslation.py Sat Oct 30 16:41:17 2010
@@ -1,4 +1,4 @@
-import py, os
+import py, os, sys
from pypy.tool.udir import udir
from pypy.rlib.jit import JitDriver, OPTIMIZER_FULL, unroll_parameters
from pypy.rlib.jit import PARAMETERS, dont_look_inside
@@ -63,8 +63,32 @@
if k - abs(j): raise ValueError
if k - abs(-j): raise ValueError
return total * 10
+ #
+ from pypy.rpython.lltypesystem import lltype, rffi
+ from pypy.rlib.libffi import types, CDLL, ArgChain
+ from pypy.rlib.test.test_libffi import get_libm_name
+ libm_name = get_libm_name(sys.platform)
+ jitdriver2 = JitDriver(greens=[], reds = ['i', 'func', 'res', 'x'])
+ def libffi_stuff(i, j):
+ lib = CDLL(libm_name)
+ func = lib.getpointer('fabs', [types.double], types.double)
+ res = 0.0
+ x = float(j)
+ while i > 0:
+ jitdriver2.jit_merge_point(i=i, res=res, func=func, x=x)
+ jitdriver2.can_enter_jit(i=i, res=res, func=func, x=x)
+ func = hint(func, promote=True)
+ argchain = ArgChain()
+ argchain.arg(x)
+ res = func.call(argchain, rffi.DOUBLE)
+ i -= 1
+ return res
+ #
+ def main(i, j):
+ return f(i, j) + libffi_stuff(i, j)
+ expected = f(40, -49)
res = self.meta_interp(f, [40, -49])
- assert res == f(40, -49)
+ assert res == expected
def test_direct_assembler_call_translates(self):
class Thing(object):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/call.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/call.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/call.py Sat Oct 30 16:41:17 2010
@@ -277,3 +277,11 @@
return seen.pop()
else:
return None
+
+ def could_be_green_field(self, GTYPE, fieldname):
+ GTYPE_fieldname = (GTYPE, fieldname)
+ for jd in self.jitdrivers_sd:
+ if jd.greenfield_info is not None:
+ if GTYPE_fieldname in jd.greenfield_info.green_fields:
+ return True
+ return False
Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/codewriter.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/codewriter.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/codewriter.py Sat Oct 30 16:41:17 2010
@@ -95,18 +95,18 @@
print '%s:' % (ssarepr.name,)
print format_assembler(ssarepr)
else:
- dir = udir.ensure("jitcodes", dir=1)
- if portal_jitdriver:
- name = "%02d_portal_runner" % (portal_jitdriver.index,)
- elif ssarepr.name and ssarepr.name != '?':
- name = ssarepr.name
- else:
- name = 'unnamed' % id(ssarepr)
- i = 1
- extra = ''
- while name+extra in self._seen_files:
- i += 1
- extra = '.%d' % i
- self._seen_files.add(name+extra)
- dir.join(name+extra).write(format_assembler(ssarepr))
log.dot()
+ dir = udir.ensure("jitcodes", dir=1)
+ if portal_jitdriver:
+ name = "%02d_portal_runner" % (portal_jitdriver.index,)
+ elif ssarepr.name and ssarepr.name != '?':
+ name = ssarepr.name
+ else:
+ name = 'unnamed' % id(ssarepr)
+ i = 1
+ extra = ''
+ while name+extra in self._seen_files:
+ i += 1
+ extra = '.%d' % i
+ self._seen_files.add(name+extra)
+ dir.join(name+extra).write(format_assembler(ssarepr))
Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/effectinfo.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/effectinfo.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/effectinfo.py Sat Oct 30 16:41:17 2010
@@ -18,19 +18,34 @@
# the 'oopspecindex' field is one of the following values:
OS_NONE = 0 # normal case, no oopspec
OS_ARRAYCOPY = 1 # "list.ll_arraycopy"
- OS_STR_CONCAT = 2 # "stroruni.concat"
- OS_UNI_CONCAT = 3 # "stroruni.concat"
- OS_STR_SLICE = 4 # "stroruni.slice"
- OS_UNI_SLICE = 5 # "stroruni.slice"
- OS_STR_EQUAL = 6 # "stroruni.equal"
- OS_UNI_EQUAL = 7 # "stroruni.equal"
- OS_STREQ_SLICE_CHECKNULL = 8 # s2!=NULL and s1[x:x+length]==s2
- OS_STREQ_SLICE_NONNULL = 9 # s1[x:x+length]==s2 (assert s2!=NULL)
- OS_STREQ_SLICE_CHAR = 10 # s1[x:x+length]==char
- OS_STREQ_NONNULL = 11 # s1 == s2 (assert s1!=NULL,s2!=NULL)
- OS_STREQ_NONNULL_CHAR = 12 # s1 == char (assert s1!=NULL)
- OS_STREQ_CHECKNULL_CHAR = 13 # s1!=NULL and s1==char
- OS_STREQ_LENGTHOK = 14 # s1 == s2 (assert len(s1)==len(s2))
+ OS_STR2UNICODE = 2 # "str.str2unicode"
+ #
+ OS_STR_CONCAT = 22 # "stroruni.concat"
+ OS_STR_SLICE = 23 # "stroruni.slice"
+ OS_STR_EQUAL = 24 # "stroruni.equal"
+ OS_STREQ_SLICE_CHECKNULL = 25 # s2!=NULL and s1[x:x+length]==s2
+ OS_STREQ_SLICE_NONNULL = 26 # s1[x:x+length]==s2 (assert s2!=NULL)
+ OS_STREQ_SLICE_CHAR = 27 # s1[x:x+length]==char
+ OS_STREQ_NONNULL = 28 # s1 == s2 (assert s1!=NULL,s2!=NULL)
+ OS_STREQ_NONNULL_CHAR = 29 # s1 == char (assert s1!=NULL)
+ OS_STREQ_CHECKNULL_CHAR = 30 # s1!=NULL and s1==char
+ OS_STREQ_LENGTHOK = 31 # s1 == s2 (assert len(s1)==len(s2))
+ #
+ OS_UNI_CONCAT = 42 #
+ OS_UNI_SLICE = 43 #
+ OS_UNI_EQUAL = 44 #
+ OS_UNIEQ_SLICE_CHECKNULL = 45 #
+ OS_UNIEQ_SLICE_NONNULL = 46 #
+ OS_UNIEQ_SLICE_CHAR = 47 #
+ OS_UNIEQ_NONNULL = 48 # the same for unicode
+ OS_UNIEQ_NONNULL_CHAR = 49 # (must be the same amount as for
+ OS_UNIEQ_CHECKNULL_CHAR = 50 # STR, in the same order)
+ OS_UNIEQ_LENGTHOK = 51 #
+ _OS_offset_uni = OS_UNI_CONCAT - OS_STR_CONCAT
+ #
+ OS_LIBFFI_PREPARE = 60
+ OS_LIBFFI_PUSH_ARG = 61
+ OS_LIBFFI_CALL = 62
def __new__(cls, readonly_descrs_fields,
write_descrs_fields, write_descrs_arrays,
Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/format.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/format.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/format.py Sat Oct 30 16:41:17 2010
@@ -80,7 +80,8 @@
def assert_format(ssarepr, expected):
asm = format_assembler(ssarepr)
- expected = str(py.code.Source(expected)).strip() + '\n'
+ if expected != '':
+ expected = str(py.code.Source(expected)).strip() + '\n'
asmlines = asm.split("\n")
explines = expected.split("\n")
for asm, exp in zip(asmlines, explines):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/jtransform.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/jtransform.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/jtransform.py Sat Oct 30 16:41:17 2010
@@ -172,6 +172,7 @@
def rewrite_op_same_as(self, op): pass
def rewrite_op_cast_pointer(self, op): pass
+ def rewrite_op_cast_opaque_ptr(self, op): pass # rlib.rerased
def rewrite_op_cast_primitive(self, op): pass
def rewrite_op_cast_bool_to_int(self, op): pass
def rewrite_op_cast_bool_to_uint(self, op): pass
@@ -316,8 +317,14 @@
prepare = self._handle_list_call
elif oopspec_name.startswith('stroruni.'):
prepare = self._handle_stroruni_call
+ elif oopspec_name == 'str.str2unicode':
+ prepare = self._handle_str2unicode_call
elif oopspec_name.startswith('virtual_ref'):
prepare = self._handle_virtual_ref_call
+ elif oopspec_name.startswith('jit.'):
+ prepare = self._handle_jit_call
+ elif oopspec_name.startswith('libffi_'):
+ prepare = self._handle_libffi_call
else:
prepare = self.prepare_builtin_call
try:
@@ -427,7 +434,8 @@
op.result)
def rewrite_op_free(self, op):
- assert op.args[1].value == 'raw'
+ flags = op.args[1].value
+ assert flags['flavor'] == 'raw'
ARRAY = op.args[0].concretetype.TO
return self._do_builtin_call(op, 'raw_free', [op.args[0]],
extra = (ARRAY,), extrakey = ARRAY)
@@ -519,7 +527,12 @@
# check for deepfrozen structures that force constant-folding
immut = v_inst.concretetype.TO._immutable_field(c_fieldname.value)
if immut:
- pure = '_pure'
+ if (self.callcontrol is not None and
+ self.callcontrol.could_be_green_field(v_inst.concretetype.TO,
+ c_fieldname.value)):
+ pure = '_greenfield'
+ else:
+ pure = '_pure'
if immut == "[*]":
self.immutable_arrays[op.result] = True
else:
@@ -819,6 +832,8 @@
def rewrite_op_jit_marker(self, op):
key = op.args[0].value
jitdriver = op.args[1].value
+ if not jitdriver.active:
+ return []
return getattr(self, 'handle_jit_marker__%s' % key)(op, jitdriver)
def handle_jit_marker__jit_merge_point(self, op, jitdriver):
@@ -852,6 +867,15 @@
(self.graph,))
return []
+ def _handle_jit_call(self, op, oopspec_name, args):
+ if oopspec_name == 'jit.debug':
+ return SpaceOperation('jit_debug', args, None)
+ elif oopspec_name == 'jit.assert_green':
+ kind = getkind(args[0].concretetype)
+ return SpaceOperation('%s_assert_green' % kind, args, None)
+ else:
+ raise AssertionError("missing support for %r" % oopspec_name)
+
# ----------
# Lists.
@@ -870,17 +894,21 @@
prefix = 'do_resizable_'
ARRAY = LIST.items.TO
if self._array_of_voids(ARRAY):
- raise NotSupported("resizable lists of voids")
- descrs = (self.cpu.arraydescrof(ARRAY),
- self.cpu.fielddescrof(LIST, 'length'),
- self.cpu.fielddescrof(LIST, 'items'),
- self.cpu.sizeof(LIST))
+ prefix += 'void_'
+ descrs = ()
+ else:
+ descrs = (self.cpu.arraydescrof(ARRAY),
+ self.cpu.fielddescrof(LIST, 'length'),
+ self.cpu.fielddescrof(LIST, 'items'),
+ self.cpu.sizeof(LIST))
else:
prefix = 'do_fixed_'
if self._array_of_voids(LIST):
- raise NotSupported("fixed lists of voids")
- arraydescr = self.cpu.arraydescrof(LIST)
- descrs = (arraydescr,)
+ prefix += 'void_'
+ descrs = ()
+ else:
+ arraydescr = self.cpu.arraydescrof(LIST)
+ descrs = (arraydescr,)
#
try:
meth = getattr(self, prefix + oopspec_name.replace('.', '_'))
@@ -919,6 +947,11 @@
descr, args[1]], v_posindex)
return v_posindex, [op0, op1]
+ def _prepare_void_list_getset(self, op):
+ non_negative, can_raise = self._get_list_nonneg_canraise_flags(op)
+ if can_raise:
+ raise NotSupported("list operation can raise")
+
def _get_initial_newlist_length(self, op, args):
# normalize number of arguments to the 'newlist' function
if len(args) > 1:
@@ -990,6 +1023,12 @@
def do_fixed_list_ll_arraycopy(self, op, args, arraydescr):
return self._handle_oopspec_call(op, args, EffectInfo.OS_ARRAYCOPY)
+ def do_fixed_void_list_getitem(self, op, args):
+ self._prepare_void_list_getset(op)
+ return []
+ do_fixed_void_list_getitem_foldable = do_fixed_void_list_getitem
+ do_fixed_void_list_setitem = do_fixed_void_list_getitem
+
# ---------- resizable lists ----------
def do_resizable_newlist(self, op, args, arraydescr, lengthdescr,
@@ -1025,11 +1064,19 @@
return SpaceOperation('getfield_gc_i',
[args[0], lengthdescr], op.result)
+ def do_resizable_void_list_getitem(self, op, args):
+ self._prepare_void_list_getset(op)
+ return []
+ do_resizable_void_list_getitem_foldable = do_resizable_void_list_getitem
+ do_resizable_void_list_setitem = do_resizable_void_list_getitem
+
# ----------
# Strings and Unicodes.
- def _handle_oopspec_call(self, op, args, oopspecindex):
+ def _handle_oopspec_call(self, op, args, oopspecindex, extraeffect=None):
calldescr = self.callcontrol.getcalldescr(op, oopspecindex)
+ if extraeffect:
+ calldescr.get_extra_info().extraeffect = extraeffect
if isinstance(op.args[0].value, str):
pass # for tests only
else:
@@ -1055,28 +1102,32 @@
[c_func] + [varoftype(T) for T in argtypes],
varoftype(resulttype))
calldescr = self.callcontrol.getcalldescr(op, oopspecindex)
- func = heaptracker.adr2int(
- llmemory.cast_ptr_to_adr(c_func.value))
+ if isinstance(c_func.value, str): # in tests only
+ func = c_func.value
+ else:
+ func = heaptracker.adr2int(
+ llmemory.cast_ptr_to_adr(c_func.value))
_callinfo_for_oopspec[oopspecindex] = calldescr, func
def _handle_stroruni_call(self, op, oopspec_name, args):
- if args[0].concretetype.TO == rstr.STR:
+ SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE)
+ if SoU.TO == rstr.STR:
dict = {"stroruni.concat": EffectInfo.OS_STR_CONCAT,
"stroruni.slice": EffectInfo.OS_STR_SLICE,
"stroruni.equal": EffectInfo.OS_STR_EQUAL,
}
- elif args[0].concretetype.TO == rstr.UNICODE:
+ CHR = lltype.Char
+ elif SoU.TO == rstr.UNICODE:
dict = {"stroruni.concat": EffectInfo.OS_UNI_CONCAT,
"stroruni.slice": EffectInfo.OS_UNI_SLICE,
"stroruni.equal": EffectInfo.OS_UNI_EQUAL,
}
+ CHR = lltype.UniChar
else:
assert 0, "args[0].concretetype must be STR or UNICODE"
#
if oopspec_name == "stroruni.equal":
- SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE)
for otherindex, othername, argtypes, resulttype in [
-
(EffectInfo.OS_STREQ_SLICE_CHECKNULL,
"str.eq_slice_checknull",
[SoU, lltype.Signed, lltype.Signed, SoU],
@@ -1087,7 +1138,7 @@
lltype.Signed),
(EffectInfo.OS_STREQ_SLICE_CHAR,
"str.eq_slice_char",
- [SoU, lltype.Signed, lltype.Signed, lltype.Char],
+ [SoU, lltype.Signed, lltype.Signed, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_NONNULL,
"str.eq_nonnull",
@@ -1095,22 +1146,27 @@
lltype.Signed),
(EffectInfo.OS_STREQ_NONNULL_CHAR,
"str.eq_nonnull_char",
- [SoU, lltype.Char],
+ [SoU, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_CHECKNULL_CHAR,
"str.eq_checknull_char",
- [SoU, lltype.Char],
+ [SoU, CHR],
lltype.Signed),
(EffectInfo.OS_STREQ_LENGTHOK,
"str.eq_lengthok",
[SoU, SoU],
lltype.Signed),
]:
+ if args[0].concretetype.TO == rstr.UNICODE:
+ otherindex += EffectInfo._OS_offset_uni
self._register_extra_helper(otherindex, othername,
argtypes, resulttype)
#
return self._handle_oopspec_call(op, args, dict[oopspec_name])
+ def _handle_str2unicode_call(self, op, oopspec_name, args):
+ return self._handle_oopspec_call(op, args, EffectInfo.OS_STR2UNICODE)
+
# ----------
# VirtualRefs.
@@ -1121,6 +1177,23 @@
vrefinfo.JIT_VIRTUAL_REF)
return SpaceOperation(oopspec_name, list(args), op.result)
+ # -----------
+ # rlib.libffi
+
+ def _handle_libffi_call(self, op, oopspec_name, args):
+ if oopspec_name == 'libffi_prepare_call':
+ oopspecindex = EffectInfo.OS_LIBFFI_PREPARE
+ extraeffect = EffectInfo.EF_CANNOT_RAISE
+ elif oopspec_name.startswith('libffi_push_'):
+ oopspecindex = EffectInfo.OS_LIBFFI_PUSH_ARG
+ extraeffect = EffectInfo.EF_CANNOT_RAISE
+ elif oopspec_name.startswith('libffi_call_'):
+ oopspecindex = EffectInfo.OS_LIBFFI_CALL
+ extraeffect = EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE
+ else:
+ assert False, 'unsupported oopspec: %s' % oopspec_name
+ return self._handle_oopspec_call(op, args, oopspecindex, extraeffect)
+
def rewrite_op_jit_force_virtual(self, op):
return self._do_builtin_call(op)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/support.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/support.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/support.py Sat Oct 30 16:41:17 2010
@@ -1,5 +1,5 @@
import sys
-from pypy.rpython.lltypesystem import lltype, rclass
+from pypy.rpython.lltypesystem import lltype, rclass, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython import rlist
from pypy.rpython.lltypesystem import rstr as ll_rstr, rdict as ll_rdict
@@ -8,6 +8,7 @@
from pypy.rpython.ootypesystem import rdict as oo_rdict
from pypy.rpython.llinterp import LLInterpreter
from pypy.rpython.extregistry import ExtRegistryEntry
+from pypy.rpython.annlowlevel import cast_base_ptr_to_instance
from pypy.translator.simplify import get_funcobj
from pypy.translator.unsimplify import split_block
from pypy.objspace.flow.model import Constant
@@ -60,7 +61,7 @@
return rtyper.annotator.translator.graphs[0]
def split_before_jit_merge_point(graph, portalblock, portalopindex):
- """Find the block with 'jit_merge_point' and split just before,
+ """Split the block just before the 'jit_merge_point',
making sure the input args are in the canonical order.
"""
# split the block just before the jit_merge_point()
@@ -217,6 +218,33 @@
else:
return x
+
+# libffi support
+# --------------
+
+def func(llfunc):
+ from pypy.rlib.libffi import Func
+ return cast_base_ptr_to_instance(Func, llfunc)
+
+def _ll_1_libffi_prepare_call(llfunc):
+ return func(llfunc)._prepare()
+
+def _ll_4_libffi_push_int(llfunc, value, ll_args, i):
+ return func(llfunc)._push_int(value, ll_args, i)
+
+def _ll_4_libffi_push_float(llfunc, value, ll_args, i):
+ return func(llfunc)._push_float(value, ll_args, i)
+
+def _ll_3_libffi_call_int(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, rffi.LONG)
+
+def _ll_3_libffi_call_float(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, rffi.DOUBLE)
+
+def _ll_3_libffi_call_void(llfunc, funcsym, ll_args):
+ return func(llfunc)._do_call(funcsym, ll_args, lltype.Void)
+
+
# in the following calls to builtins, the JIT is allowed to look inside:
inline_calls_to = [
('int_floordiv_ovf_zer', [lltype.Signed, lltype.Signed], lltype.Signed),
Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_codewriter.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_codewriter.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_codewriter.py Sat Oct 30 16:41:17 2010
@@ -45,6 +45,7 @@
self.portal_graph = portal_graph
self.portal_runner_ptr = "???"
self.virtualizable_info = None
+ self.greenfield_info = None
def test_loop():
Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_effectinfo.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_effectinfo.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_effectinfo.py Sat Oct 30 16:41:17 2010
@@ -1,7 +1,8 @@
from pypy.rpython.lltypesystem.rclass import OBJECT
from pypy.rpython.lltypesystem import lltype
from pypy.rpython.ootypesystem import ootype
-from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze
+from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze,\
+ EffectInfo
class FakeCPU:
def fielddescrof(self, T, fieldname):
@@ -9,6 +10,14 @@
def arraydescrof(self, A):
return ('arraydescr', A)
+def test_no_oopspec_duplicate():
+ # check that all the various EffectInfo.OS_* have unique values
+ oopspecs = set()
+ for name, value in EffectInfo.__dict__.iteritems():
+ if name.startswith('OS_'):
+ assert value not in oopspecs
+ oopspecs.add(value)
+
def test_include_read_field():
S = lltype.GcStruct("S", ("a", lltype.Signed))
effects = frozenset([("readstruct", lltype.Ptr(S), "a")])
Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_jtransform.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_jtransform.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_jtransform.py Sat Oct 30 16:41:17 2010
@@ -77,7 +77,32 @@
class FakeBuiltinCallControl:
def guess_call_kind(self, op):
return 'builtin'
- def getcalldescr(self, op, oopspecindex):
+ def getcalldescr(self, op, oopspecindex=None):
+ assert oopspecindex is not None # in this test
+ EI = effectinfo.EffectInfo
+ if oopspecindex != EI.OS_ARRAYCOPY:
+ PSTR = lltype.Ptr(rstr.STR)
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ INT = lltype.Signed
+ UNICHAR = lltype.UniChar
+ argtypes = {
+ EI.OS_STR2UNICODE:([PSTR], PUNICODE),
+ EI.OS_STR_CONCAT: ([PSTR, PSTR], PSTR),
+ EI.OS_STR_SLICE: ([PSTR, INT, INT], PSTR),
+ EI.OS_UNI_CONCAT: ([PUNICODE, PUNICODE], PUNICODE),
+ EI.OS_UNI_SLICE: ([PUNICODE, INT, INT], PUNICODE),
+ EI.OS_UNI_EQUAL: ([PUNICODE, PUNICODE], lltype.Bool),
+ EI.OS_UNIEQ_SLICE_CHECKNULL:([PUNICODE, INT, INT, PUNICODE], INT),
+ EI.OS_UNIEQ_SLICE_NONNULL: ([PUNICODE, INT, INT, PUNICODE], INT),
+ EI.OS_UNIEQ_SLICE_CHAR: ([PUNICODE, INT, INT, UNICHAR], INT),
+ EI.OS_UNIEQ_NONNULL: ([PUNICODE, PUNICODE], INT),
+ EI.OS_UNIEQ_NONNULL_CHAR: ([PUNICODE, UNICHAR], INT),
+ EI.OS_UNIEQ_CHECKNULL_CHAR: ([PUNICODE, UNICHAR], INT),
+ EI.OS_UNIEQ_LENGTHOK: ([PUNICODE, PUNICODE], INT),
+ }
+ argtypes = argtypes[oopspecindex]
+ assert argtypes[0] == [v.concretetype for v in op.args[1:]]
+ assert argtypes[1] == op.result.concretetype
return 'calldescr-%d' % oopspecindex
def calldescr_canraise(self, calldescr):
return False
@@ -662,6 +687,80 @@
assert block.operations[1].result is None
assert block.exits[0].args == [v1]
+def test_jit_merge_point_1():
+ class FakeJitDriverSD:
+ index = 42
+ class jitdriver:
+ active = True
+ greens = ['green1', 'green2', 'voidgreen3']
+ reds = ['red1', 'red2', 'voidred3']
+ jd = FakeJitDriverSD()
+ v1 = varoftype(lltype.Signed)
+ v2 = varoftype(lltype.Signed)
+ vvoid1 = varoftype(lltype.Void)
+ v3 = varoftype(lltype.Signed)
+ v4 = varoftype(lltype.Signed)
+ vvoid2 = varoftype(lltype.Void)
+ v5 = varoftype(lltype.Void)
+ op = SpaceOperation('jit_marker',
+ [Constant('jit_merge_point', lltype.Void),
+ Constant(jd.jitdriver, lltype.Void),
+ v1, v2, vvoid1, v3, v4, vvoid2], v5)
+ tr = Transformer()
+ tr.portal_jd = jd
+ oplist = tr.rewrite_operation(op)
+ assert len(oplist) == 6
+ assert oplist[0].opname == '-live-'
+ assert oplist[1].opname == 'int_guard_value'
+ assert oplist[1].args == [v1]
+ assert oplist[2].opname == '-live-'
+ assert oplist[3].opname == 'int_guard_value'
+ assert oplist[3].args == [v2]
+ assert oplist[4].opname == 'jit_merge_point'
+ assert oplist[4].args[0].value == 42
+ assert list(oplist[4].args[1]) == [v1, v2]
+ assert list(oplist[4].args[4]) == [v3, v4]
+ assert oplist[5].opname == '-live-'
+
+def test_getfield_gc():
+ S = lltype.GcStruct('S', ('x', lltype.Char))
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
+def test_getfield_gc_pure():
+ S = lltype.GcStruct('S', ('x', lltype.Char),
+ hints={'immutable': True})
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i_pure'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
+def test_getfield_gc_greenfield():
+ class FakeCC:
+ def get_vinfo(self, v):
+ return None
+ def could_be_green_field(self, S1, name1):
+ assert S1 is S
+ assert name1 == 'x'
+ return True
+ S = lltype.GcStruct('S', ('x', lltype.Char),
+ hints={'immutable': True})
+ v1 = varoftype(lltype.Ptr(S))
+ v2 = varoftype(lltype.Char)
+ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2)
+ op1 = Transformer(FakeCPU(), FakeCC()).rewrite_operation(op)
+ assert op1.opname == 'getfield_gc_i_greenfield'
+ assert op1.args == [v1, ('fielddescr', S, 'x')]
+ assert op1.result == v2
+
def test_int_abs():
v1 = varoftype(lltype.Signed)
v2 = varoftype(lltype.Signed)
@@ -766,6 +865,46 @@
assert op1.args[3] == ListOfKind('ref', [v1])
assert op1.result == v4
+def test_str2unicode():
+ # test that the oopspec is present and correctly transformed
+ PSTR = lltype.Ptr(rstr.STR)
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ FUNC = lltype.FuncType([PSTR], PUNICODE)
+ func = lltype.functionptr(FUNC, 'll_str2unicode',
+ _callable=rstr.LLHelpers.ll_str2unicode)
+ v1 = varoftype(PSTR)
+ v2 = varoftype(PUNICODE)
+ op = SpaceOperation('direct_call', [const(func), v1], v2)
+ tr = Transformer(FakeCPU(), FakeBuiltinCallControl())
+ op1 = tr.rewrite_operation(op)
+ assert op1.opname == 'residual_call_r_r'
+ assert op1.args[0].value == func
+ assert op1.args[1] == 'calldescr-%d' % effectinfo.EffectInfo.OS_STR2UNICODE
+ assert op1.args[2] == ListOfKind('ref', [v1])
+ assert op1.result == v2
+
+def test_unicode_eq_checknull_char():
+ # test that the oopspec is present and correctly transformed
+ PUNICODE = lltype.Ptr(rstr.UNICODE)
+ FUNC = lltype.FuncType([PUNICODE, PUNICODE], lltype.Bool)
+ func = lltype.functionptr(FUNC, 'll_streq',
+ _callable=rstr.LLHelpers.ll_streq)
+ v1 = varoftype(PUNICODE)
+ v2 = varoftype(PUNICODE)
+ v3 = varoftype(lltype.Bool)
+ op = SpaceOperation('direct_call', [const(func), v1, v2], v3)
+ tr = Transformer(FakeCPU(), FakeBuiltinCallControl())
+ op1 = tr.rewrite_operation(op)
+ assert op1.opname == 'residual_call_r_i'
+ assert op1.args[0].value == func
+ assert op1.args[1] == 'calldescr-%d' % effectinfo.EffectInfo.OS_UNI_EQUAL
+ assert op1.args[2] == ListOfKind('ref', [v1, v2])
+ assert op1.result == v3
+ # test that the OS_UNIEQ_* functions are registered
+ cifo = effectinfo._callinfo_for_oopspec
+ assert effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL in cifo
+ assert effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR in cifo
+
def test_list_ll_arraycopy():
from pypy.rlib.rgc import ll_arraycopy
LIST = lltype.GcArray(lltype.Signed)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_list.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_list.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_list.py Sat Oct 30 16:41:17 2010
@@ -19,6 +19,7 @@
class FakeCPU:
class arraydescrof(AbstractDescr):
def __init__(self, ARRAY):
+ assert ARRAY.OF != lltype.Void
self.ARRAY = ARRAY
def __repr__(self):
return ''
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/blackhole.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/blackhole.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/blackhole.py Sat Oct 30 16:41:17 2010
@@ -760,6 +760,20 @@
def bhimpl_debug_fatalerror(msg):
llop.debug_fatalerror(lltype.Void, msg)
+ @arguments("r", "i", "i", "i", "i")
+ def bhimpl_jit_debug(string, arg1=0, arg2=0, arg3=0, arg4=0):
+ pass
+
+ @arguments("i")
+ def bhimpl_int_assert_green(x):
+ pass
+ @arguments("r")
+ def bhimpl_ref_assert_green(x):
+ pass
+ @arguments("f")
+ def bhimpl_float_assert_green(x):
+ pass
+
# ----------
# the main hints and recursive calls
@@ -1073,6 +1087,10 @@
bhimpl_getfield_vable_r = bhimpl_getfield_gc_r
bhimpl_getfield_vable_f = bhimpl_getfield_gc_f
+ bhimpl_getfield_gc_i_greenfield = bhimpl_getfield_gc_i
+ bhimpl_getfield_gc_r_greenfield = bhimpl_getfield_gc_r
+ bhimpl_getfield_gc_f_greenfield = bhimpl_getfield_gc_f
+
@arguments("cpu", "i", "d", returns="i")
def bhimpl_getfield_raw_i(cpu, struct, fielddescr):
return cpu.bh_getfield_raw_i(struct, fielddescr)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py Sat Oct 30 16:41:17 2010
@@ -140,7 +140,6 @@
if not we_are_translated():
show_loop(metainterp_sd)
TreeLoop.check_consistency_of(inputargs, operations)
- pass
metainterp_sd.profiler.start_backend()
debug_start("jit-backend")
try:
@@ -387,7 +386,8 @@
from pypy.jit.metainterp.resume import force_from_resumedata
metainterp_sd = self.metainterp_sd
vinfo = self.jitdriver_sd.virtualizable_info
- all_virtuals = force_from_resumedata(metainterp_sd, self, vinfo)
+ ginfo = self.jitdriver_sd.greenfield_info
+ all_virtuals = force_from_resumedata(metainterp_sd, self, vinfo, ginfo)
# The virtualizable data was stored on the real virtualizable above.
# Handle all_virtuals: keep them for later blackholing from the
# future failure of the GUARD_NOT_FORCED
@@ -616,5 +616,5 @@
ResOperation(rop.FINISH, finishargs, None, descr=jd.portal_finishtoken)
]
operations[1].setfailargs([])
- cpu.compile_loop(inputargs, operations, loop_token)
+ cpu.compile_loop(inputargs, operations, loop_token, log=False)
return loop_token
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/executor.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/executor.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/executor.py Sat Oct 30 16:41:17 2010
@@ -80,6 +80,9 @@
do_call_loopinvariant = do_call
do_call_may_force = do_call
+def do_call_c(cpu, metainterp, argboxes, descr):
+ raise NotImplementedError("Should never be called directly")
+
def do_getarrayitem_gc(cpu, _, arraybox, indexbox, arraydescr):
array = arraybox.getref_base()
index = indexbox.getint()
@@ -205,8 +208,8 @@
def do_copystrcontent(cpu, _, srcbox, dstbox,
srcstartbox, dststartbox, lengthbox):
- src = srcbox.getptr(lltype.Ptr(rstr.STR))
- dst = dstbox.getptr(lltype.Ptr(rstr.STR))
+ src = srcbox.getref(lltype.Ptr(rstr.STR))
+ dst = dstbox.getref(lltype.Ptr(rstr.STR))
srcstart = srcstartbox.getint()
dststart = dststartbox.getint()
length = lengthbox.getint()
@@ -214,8 +217,8 @@
def do_copyunicodecontent(cpu, _, srcbox, dstbox,
srcstartbox, dststartbox, lengthbox):
- src = srcbox.getptr(lltype.Ptr(rstr.UNICODE))
- dst = dstbox.getptr(lltype.Ptr(rstr.UNICODE))
+ src = srcbox.getref(lltype.Ptr(rstr.UNICODE))
+ dst = dstbox.getref(lltype.Ptr(rstr.UNICODE))
srcstart = srcstartbox.getint()
dststart = dststartbox.getint()
length = lengthbox.getint()
@@ -304,6 +307,7 @@
rop.CALL_ASSEMBLER,
rop.COND_CALL_GC_WB,
rop.DEBUG_MERGE_POINT,
+ rop.JIT_DEBUG,
rop.SETARRAYITEM_RAW,
): # list of opcodes never executed by pyjitpl
continue
@@ -428,6 +432,10 @@
if arity == 3:
func = get_execute_funclist(3, False)[opnum]
return func(cpu, metainterp, argboxes[0], argboxes[1], argboxes[2])
+ if arity == 5: # copystrcontent, copyunicodecontent
+ func = get_execute_funclist(5, False)[opnum]
+ return func(cpu, metainterp, argboxes[0], argboxes[1],
+ argboxes[2], argboxes[3], argboxes[4])
raise NotImplementedError
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/graphpage.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/graphpage.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/graphpage.py Sat Oct 30 16:41:17 2010
@@ -153,7 +153,7 @@
opindex = opstartindex
while True:
op = operations[opindex]
- lines.append(repr(op))
+ lines.append(op.repr(graytext=True))
if is_interesting_guard(op):
tgt = op.getdescr()._debug_suboperations[0]
tgt_g, tgt_i = self.all_operations[tgt]
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/history.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/history.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/history.py Sat Oct 30 16:41:17 2010
@@ -698,6 +698,21 @@
return result
_const_ptr_for_string = {}
+def get_const_ptr_for_unicode(s):
+ from pypy.rpython.annlowlevel import llunicode
+ if not we_are_translated():
+ try:
+ return _const_ptr_for_unicode[s]
+ except KeyError:
+ pass
+ if isinstance(s, str):
+ s = unicode(s)
+ result = ConstPtr(lltype.cast_opaque_ptr(llmemory.GCREF, llunicode(s)))
+ if not we_are_translated():
+ _const_ptr_for_unicode[s] = result
+ return result
+_const_ptr_for_unicode = {}
+
# ____________________________________________________________
# The TreeLoop class contains a loop or a generalized loop, i.e. a tree
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/jitdriver.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/jitdriver.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/jitdriver.py Sat Oct 30 16:41:17 2010
@@ -13,8 +13,10 @@
# self.num_red_args ... pypy.jit.metainterp.warmspot
# self.result_type ... pypy.jit.metainterp.warmspot
# self.virtualizable_info... pypy.jit.metainterp.warmspot
+ # self.greenfield_info ... pypy.jit.metainterp.warmspot
# self.warmstate ... pypy.jit.metainterp.warmspot
# self.handle_jitexc_from_bh pypy.jit.metainterp.warmspot
+ # self.no_loop_header ... pypy.jit.metainterp.warmspot
# self.portal_finishtoken... pypy.jit.metainterp.pyjitpl
# self.index ... pypy.jit.codewriter.call
# self.mainjitcode ... pypy.jit.codewriter.call
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize_nopspec.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize_nopspec.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize_nopspec.py Sat Oct 30 16:41:17 2010
@@ -14,6 +14,9 @@
def _optimize_loop(metainterp_sd, old_loop_tokens, loop):
cpu = metainterp_sd.cpu
metainterp_sd.logger_noopt.log_loop(loop.inputargs, loop.operations)
+ # XXX the following lines are probably still needed, to discard invalid
+ # loops. bit silly to run a full perfect specialization and throw the
+ # result away.
finder = PerfectSpecializationFinder(cpu)
finder.find_nodes_loop(loop, False)
if old_loop_tokens:
@@ -31,6 +34,7 @@
def _optimize_bridge(metainterp_sd, old_loop_tokens, bridge):
cpu = metainterp_sd.cpu
metainterp_sd.logger_noopt.log_loop(bridge.inputargs, bridge.operations)
+ # XXX same comment as above applies
finder = BridgeSpecializationFinder(cpu)
finder.find_nodes_bridge(bridge)
if old_loop_tokens:
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py Sat Oct 30 16:41:17 2010
@@ -3,6 +3,7 @@
from pypy.jit.metainterp.optimizeopt.intbounds import OptIntBounds
from pypy.jit.metainterp.optimizeopt.virtualize import OptVirtualize
from pypy.jit.metainterp.optimizeopt.heap import OptHeap
+from pypy.jit.metainterp.optimizeopt.fficall import OptFfiCall
from pypy.jit.metainterp.optimizeopt.string import OptString
from pypy.jit.metainterp.optimizeopt.unroll import OptUnroll
@@ -17,6 +18,7 @@
OptVirtualize(),
OptString(),
OptHeap(),
+ OptFfiCall(),
]
if not_a_bridge:
optimizations.insert(0, OptUnroll())
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py Sat Oct 30 16:41:17 2010
@@ -205,6 +205,7 @@
v1.intbound.make_ge(IntLowerBound(0))
optimize_STRLEN = optimize_ARRAYLEN_GC
+ optimize_UNICODELEN = optimize_ARRAYLEN_GC
def make_int_lt(self, box1, box2):
v1 = self.getvalue(box1)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py Sat Oct 30 16:41:17 2010
@@ -3,6 +3,7 @@
from pypy.jit.metainterp.history import ConstInt
from pypy.jit.metainterp.optimizeutil import _findall
from pypy.jit.metainterp.resoperation import rop, ResOperation
+from pypy.jit.codewriter.effectinfo import EffectInfo
class OptRewrite(Optimization):
"""Rewrite operations into equivalent, cheaper operations.
@@ -244,11 +245,11 @@
self.optimizer.exception_might_have_happened = False
def optimize_CALL_LOOPINVARIANT(self, op):
- funcvalue = self.getvalue(op.getarg(0))
- if not funcvalue.is_constant():
- self.emit_operation(op)
- return
- key = make_hashable_int(op.getarg(0).getint())
+ arg = op.getarg(0)
+ # 'arg' must be a Const, because residual_call in codewriter
+ # expects a compile-time constant
+ assert isinstance(arg, Const)
+ key = make_hashable_int(arg.getint())
resvalue = self.optimizer.loop_invariant_results.get(key, None)
if resvalue is not None:
self.make_equal_to(op.result, resvalue)
@@ -324,8 +325,37 @@
## return
## self.emit_operation(op)
-optimize_ops = _findall(OptRewrite, 'optimize_')
-
+ def optimize_CALL(self, op):
+ # dispatch based on 'oopspecindex' to a method that handles
+ # specifically the given oopspec call. For non-oopspec calls,
+ # oopspecindex is just zero.
+ effectinfo = op.getdescr().get_extra_info()
+ if effectinfo is not None:
+ oopspecindex = effectinfo.oopspecindex
+ if oopspecindex == EffectInfo.OS_ARRAYCOPY:
+ if self._optimize_CALL_ARRAYCOPY(op):
+ return
+ self.emit_operation(op)
+ def _optimize_CALL_ARRAYCOPY(self, op):
+ source_value = self.getvalue(op.getarg(1))
+ dest_value = self.getvalue(op.getarg(2))
+ source_start_box = self.get_constant_box(op.getarg(3))
+ dest_start_box = self.get_constant_box(op.getarg(4))
+ length = self.get_constant_box(op.getarg(5))
+ if (source_value.is_virtual() and source_start_box and dest_start_box
+ and length and dest_value.is_virtual()):
+ # XXX optimize the case where dest value is not virtual,
+ # but we still can avoid a mess
+ source_start = source_start_box.getint()
+ dest_start = dest_start_box.getint()
+ for index in range(length.getint()):
+ val = source_value.getitem(index + source_start)
+ dest_value.setitem(index + dest_start, val)
+ return True
+ if length and length.getint() == 0:
+ return True # 0-length arraycopy
+ return False
+
+optimize_ops = _findall(OptRewrite, 'optimize_')
-
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py Sat Oct 30 16:41:17 2010
@@ -3,6 +3,7 @@
from pypy.jit.metainterp.history import Box, BoxInt, BoxPtr
from pypy.jit.metainterp.history import Const, ConstInt, ConstPtr
from pypy.jit.metainterp.history import get_const_ptr_for_string
+from pypy.jit.metainterp.history import get_const_ptr_for_unicode
from pypy.jit.metainterp.resoperation import rop, ResOperation
from pypy.jit.metainterp.optimizeopt import optimizer, virtualize
from pypy.jit.metainterp.optimizeopt.optimizer import CONST_0, CONST_1
@@ -11,56 +12,106 @@
from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec
from pypy.jit.codewriter import heaptracker
from pypy.rlib.unroll import unrolling_iterable
+from pypy.rlib.objectmodel import specialize, we_are_translated
+
+
+class StrOrUnicode(object):
+ def __init__(self, LLTYPE, hlstr, emptystr, chr,
+ NEWSTR, STRLEN, STRGETITEM, STRSETITEM, COPYSTRCONTENT,
+ OS_offset):
+ self.LLTYPE = LLTYPE
+ self.hlstr = hlstr
+ self.emptystr = emptystr
+ self.chr = chr
+ self.NEWSTR = NEWSTR
+ self.STRLEN = STRLEN
+ self.STRGETITEM = STRGETITEM
+ self.STRSETITEM = STRSETITEM
+ self.COPYSTRCONTENT = COPYSTRCONTENT
+ self.OS_offset = OS_offset
+
+ def _freeze_(self):
+ return True
+
+mode_string = StrOrUnicode(rstr.STR, annlowlevel.hlstr, '', chr,
+ rop.NEWSTR, rop.STRLEN, rop.STRGETITEM,
+ rop.STRSETITEM, rop.COPYSTRCONTENT, 0)
+mode_unicode = StrOrUnicode(rstr.UNICODE, annlowlevel.hlunicode, u'', unichr,
+ rop.NEWUNICODE, rop.UNICODELEN, rop.UNICODEGETITEM,
+ rop.UNICODESETITEM, rop.COPYUNICODECONTENT,
+ EffectInfo._OS_offset_uni)
+
+# ____________________________________________________________
class __extend__(optimizer.OptValue):
"""New methods added to the base class OptValue for this file."""
- def getstrlen(self, newoperations):
- s = self.get_constant_string()
- if s is not None:
- return ConstInt(len(s))
+ def getstrlen(self, newoperations, mode):
+ if mode is mode_string:
+ s = self.get_constant_string_spec(mode_string)
+ if s is not None:
+ return ConstInt(len(s))
else:
- if newoperations is None:
- return None
- self.ensure_nonnull()
- box = self.force_box()
- lengthbox = BoxInt()
- newoperations.append(ResOperation(rop.STRLEN, [box], lengthbox))
- return lengthbox
+ s = self.get_constant_string_spec(mode_unicode)
+ if s is not None:
+ return ConstInt(len(s))
+ if newoperations is None:
+ return None
+ self.ensure_nonnull()
+ box = self.force_box()
+ lengthbox = BoxInt()
+ newoperations.append(ResOperation(mode.STRLEN, [box], lengthbox))
+ return lengthbox
- def get_constant_string(self):
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
if self.is_constant():
- s = self.box.getref(lltype.Ptr(rstr.STR))
- return annlowlevel.hlstr(s)
+ s = self.box.getref(lltype.Ptr(mode.LLTYPE))
+ return mode.hlstr(s)
else:
return None
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
# Copies the pointer-to-string 'self' into the target string
# given by 'targetbox', at the specified offset. Returns the offset
# at the end of the copy.
- lengthbox = self.getstrlen(newoperations)
+ lengthbox = self.getstrlen(newoperations, mode)
srcbox = self.force_box()
return copy_str_content(newoperations, srcbox, targetbox,
- CONST_0, offsetbox, lengthbox)
+ CONST_0, offsetbox, lengthbox, mode)
class VAbstractStringValue(virtualize.AbstractVirtualValue):
- _attrs_ = ()
+ _attrs_ = ('mode',)
+
+ def __init__(self, optimizer, keybox, source_op, mode):
+ virtualize.AbstractVirtualValue.__init__(self, optimizer, keybox,
+ source_op)
+ self.mode = mode
def _really_force(self):
- s = self.get_constant_string()
- if s is not None:
- c_s = get_const_ptr_for_string(s)
- self.make_constant(c_s)
- return
+ if self.mode is mode_string:
+ s = self.get_constant_string_spec(mode_string)
+ if s is not None:
+ c_s = get_const_ptr_for_string(s)
+ self.make_constant(c_s)
+ return
+ else:
+ s = self.get_constant_string_spec(mode_unicode)
+ if s is not None:
+ c_s = get_const_ptr_for_unicode(s)
+ self.make_constant(c_s)
+ return
assert self.source_op is not None
self.box = box = self.source_op.result
newoperations = self.optimizer.newoperations
- lengthbox = self.getstrlen(newoperations)
- newoperations.append(ResOperation(rop.NEWSTR, [lengthbox], box))
- self.string_copy_parts(newoperations, box, CONST_0)
+ lengthbox = self.getstrlen(newoperations, self.mode)
+ op = ResOperation(self.mode.NEWSTR, [lengthbox], box)
+ if not we_are_translated():
+ op.name = 'FORCE'
+ newoperations.append(op)
+ self.string_copy_parts(newoperations, box, CONST_0, self.mode)
class VStringPlainValue(VAbstractStringValue):
@@ -74,7 +125,7 @@
assert 0 <= start <= stop <= len(longerlist)
self._chars = longerlist[start:stop]
- def getstrlen(self, _):
+ def getstrlen(self, _, mode):
if self._lengthbox is None:
self._lengthbox = ConstInt(len(self._chars))
return self._lengthbox
@@ -86,18 +137,21 @@
assert isinstance(charvalue, optimizer.OptValue)
self._chars[index] = charvalue
- def get_constant_string(self):
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
for c in self._chars:
if c is optimizer.CVAL_UNINITIALIZED_ZERO or not c.is_constant():
return None
- return ''.join([chr(c.box.getint()) for c in self._chars])
+ return mode.emptystr.join([mode.chr(c.box.getint())
+ for c in self._chars])
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
for i in range(len(self._chars)):
charbox = self._chars[i].force_box()
- newoperations.append(ResOperation(rop.STRSETITEM, [targetbox,
- offsetbox,
- charbox], None))
+ newoperations.append(ResOperation(mode.STRSETITEM, [targetbox,
+ offsetbox,
+ charbox],
+ None))
offsetbox = _int_add(newoperations, offsetbox, CONST_1)
return offsetbox
@@ -120,7 +174,7 @@
boxes.append(self.box)
def _make_virtual(self, modifier):
- return modifier.make_vstrplain()
+ return modifier.make_vstrplain(self.mode is mode_unicode)
class VStringConcatValue(VAbstractStringValue):
@@ -131,23 +185,24 @@
self.right = right
self.lengthbox = lengthbox
- def getstrlen(self, _):
+ def getstrlen(self, _, mode):
return self.lengthbox
- def get_constant_string(self):
- s1 = self.left.get_constant_string()
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
+ s1 = self.left.get_constant_string_spec(mode)
if s1 is None:
return None
- s2 = self.right.get_constant_string()
+ s2 = self.right.get_constant_string_spec(mode)
if s2 is None:
return None
return s1 + s2
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
offsetbox = self.left.string_copy_parts(newoperations, targetbox,
- offsetbox)
+ offsetbox, mode)
offsetbox = self.right.string_copy_parts(newoperations, targetbox,
- offsetbox)
+ offsetbox, mode)
return offsetbox
def get_args_for_fail(self, modifier):
@@ -172,7 +227,7 @@
boxes.append(self.box)
def _make_virtual(self, modifier):
- return modifier.make_vstrconcat()
+ return modifier.make_vstrconcat(self.mode is mode_unicode)
class VStringSliceValue(VAbstractStringValue):
@@ -184,12 +239,13 @@
self.vstart = vstart
self.vlength = vlength
- def getstrlen(self, _):
+ def getstrlen(self, _, mode):
return self.vlength.force_box()
- def get_constant_string(self):
+ @specialize.arg(1)
+ def get_constant_string_spec(self, mode):
if self.vstart.is_constant() and self.vlength.is_constant():
- s1 = self.vstr.get_constant_string()
+ s1 = self.vstr.get_constant_string_spec(mode)
if s1 is None:
return None
start = self.vstart.box.getint()
@@ -199,12 +255,12 @@
return s1[start : start + length]
return None
- def string_copy_parts(self, newoperations, targetbox, offsetbox):
- lengthbox = self.getstrlen(newoperations)
+ def string_copy_parts(self, newoperations, targetbox, offsetbox, mode):
+ lengthbox = self.getstrlen(newoperations, mode)
return copy_str_content(newoperations,
self.vstr.force_box(), targetbox,
self.vstart.force_box(), offsetbox,
- lengthbox)
+ lengthbox, mode)
def get_args_for_fail(self, modifier):
if self.box is None and not modifier.already_seen_virtual(self.keybox):
@@ -229,11 +285,11 @@
boxes.append(self.box)
def _make_virtual(self, modifier):
- return modifier.make_vstrslice()
+ return modifier.make_vstrslice(self.mode is mode_unicode)
def copy_str_content(newoperations, srcbox, targetbox,
- srcoffsetbox, offsetbox, lengthbox):
+ srcoffsetbox, offsetbox, lengthbox, mode):
if isinstance(srcbox, ConstPtr) and isinstance(srcoffsetbox, Const):
M = 5
else:
@@ -242,17 +298,18 @@
# up to M characters are done "inline", i.e. with STRGETITEM/STRSETITEM
# instead of just a COPYSTRCONTENT.
for i in range(lengthbox.value):
- charbox = _strgetitem(newoperations, srcbox, srcoffsetbox)
+ charbox = _strgetitem(newoperations, srcbox, srcoffsetbox, mode)
srcoffsetbox = _int_add(newoperations, srcoffsetbox, CONST_1)
- newoperations.append(ResOperation(rop.STRSETITEM, [targetbox,
- offsetbox,
- charbox], None))
+ newoperations.append(ResOperation(mode.STRSETITEM, [targetbox,
+ offsetbox,
+ charbox],
+ None))
offsetbox = _int_add(newoperations, offsetbox, CONST_1)
else:
nextoffsetbox = _int_add(newoperations, offsetbox, lengthbox)
- op = ResOperation(rop.COPYSTRCONTENT, [srcbox, targetbox,
- srcoffsetbox, offsetbox,
- lengthbox], None)
+ op = ResOperation(mode.COPYSTRCONTENT, [srcbox, targetbox,
+ srcoffsetbox, offsetbox,
+ lengthbox], None)
newoperations.append(op)
offsetbox = nextoffsetbox
return offsetbox
@@ -279,12 +336,16 @@
newoperations.append(ResOperation(rop.INT_SUB, [box1, box2], resbox))
return resbox
-def _strgetitem(newoperations, strbox, indexbox):
+def _strgetitem(newoperations, strbox, indexbox, mode):
if isinstance(strbox, ConstPtr) and isinstance(indexbox, ConstInt):
- s = strbox.getref(lltype.Ptr(rstr.STR))
- return ConstInt(ord(s.chars[indexbox.getint()]))
+ if mode is mode_string:
+ s = strbox.getref(lltype.Ptr(rstr.STR))
+ return ConstInt(ord(s.chars[indexbox.getint()]))
+ else:
+ s = strbox.getref(lltype.Ptr(rstr.UNICODE))
+ return ConstInt(ord(s.chars[indexbox.getint()]))
resbox = BoxInt()
- newoperations.append(ResOperation(rop.STRGETITEM, [strbox, indexbox],
+ newoperations.append(ResOperation(mode.STRGETITEM, [strbox, indexbox],
resbox))
return resbox
@@ -292,62 +353,34 @@
class OptString(optimizer.Optimization):
"Handling of strings and unicodes."
- def make_vstring_plain(self, box, source_op=None):
- vvalue = VStringPlainValue(self.optimizer, box, source_op)
+ def make_vstring_plain(self, box, source_op, mode):
+ vvalue = VStringPlainValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
return vvalue
- def make_vstring_concat(self, box, source_op=None):
- vvalue = VStringConcatValue(self.optimizer, box, source_op)
+ def make_vstring_concat(self, box, source_op, mode):
+ vvalue = VStringConcatValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
return vvalue
- def make_vstring_slice(self, box, source_op=None):
- vvalue = VStringSliceValue(self.optimizer, box, source_op)
+ def make_vstring_slice(self, box, source_op, mode):
+ vvalue = VStringSliceValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
return vvalue
- def optimize_CALL(self, op):
- # dispatch based on 'oopspecindex' to a method that handles
- # specifically the given oopspec call. For non-oopspec calls,
- # oopspecindex is just zero.
- effectinfo = op.getdescr().get_extra_info()
- if effectinfo is not None:
- oopspecindex = effectinfo.oopspecindex
- for value, meth in opt_call_oopspec_ops:
- if oopspecindex == value:
- if meth(self, op):
- return
- self.emit_operation(op)
-
- def opt_call_oopspec_ARRAYCOPY(self, op):
- source_value = self.getvalue(op.getarg(1))
- dest_value = self.getvalue(op.getarg(2))
- source_start_box = self.get_constant_box(op.getarg(3))
- dest_start_box = self.get_constant_box(op.getarg(4))
- length = self.get_constant_box(op.getarg(5))
- if (source_value.is_virtual() and source_start_box and dest_start_box
- and length and dest_value.is_virtual()):
- # XXX optimize the case where dest value is not virtual,
- # but we still can avoid a mess
- source_start = source_start_box.getint()
- dest_start = dest_start_box.getint()
- for index in range(length.getint()):
- val = source_value.getitem(index + source_start)
- dest_value.setitem(index + dest_start, val)
- return True
- if length and length.getint() == 0:
- return True # 0-length arraycopy
- return False
-
def optimize_NEWSTR(self, op):
+ self._optimize_NEWSTR(op, mode_string)
+ def optimize_NEWUNICODE(self, op):
+ self._optimize_NEWSTR(op, mode_unicode)
+
+ def _optimize_NEWSTR(self, op, mode):
length_box = self.get_constant_box(op.getarg(0))
if length_box:
# if the original 'op' did not have a ConstInt as argument,
# build a new one with the ConstInt argument
if not isinstance(op.getarg(0), ConstInt):
- op = ResOperation(rop.NEWSTR, [length_box], op.result)
- vvalue = self.make_vstring_plain(op.result, op)
+ op = ResOperation(mode.NEWSTR, [length_box], op.result)
+ vvalue = self.make_vstring_plain(op.result, op, mode)
vvalue.setup(length_box.getint())
else:
self.getvalue(op.result).ensure_nonnull()
@@ -363,13 +396,20 @@
value.ensure_nonnull()
self.emit_operation(op)
+ optimize_UNICODESETITEM = optimize_STRSETITEM
+
def optimize_STRGETITEM(self, op):
+ self._optimize_STRGETITEM(op, mode_string)
+ def optimize_UNICODEGETITEM(self, op):
+ self._optimize_STRGETITEM(op, mode_unicode)
+
+ def _optimize_STRGETITEM(self, op, mode):
value = self.getvalue(op.getarg(0))
vindex = self.getvalue(op.getarg(1))
- vresult = self.strgetitem(value, vindex)
+ vresult = self.strgetitem(value, vindex, mode)
self.make_equal_to(op.result, vresult)
- def strgetitem(self, value, vindex):
+ def strgetitem(self, value, vindex, mode):
value.ensure_nonnull()
#
if value.is_virtual() and isinstance(value, VStringSliceValue):
@@ -384,28 +424,71 @@
return value.getitem(vindex.box.getint())
#
resbox = _strgetitem(self.optimizer.newoperations,
- value.force_box(),vindex.force_box())
+ value.force_box(),vindex.force_box(), mode)
return self.getvalue(resbox)
def optimize_STRLEN(self, op):
+ self._optimize_STRLEN(op, mode_string)
+ def optimize_UNICODELEN(self, op):
+ self._optimize_STRLEN(op, mode_unicode)
+
+ def _optimize_STRLEN(self, op, mode):
value = self.getvalue(op.getarg(0))
- lengthbox = value.getstrlen(self.optimizer.newoperations)
+ lengthbox = value.getstrlen(self.optimizer.newoperations, mode)
self.make_equal_to(op.result, self.getvalue(lengthbox))
- def opt_call_oopspec_STR_CONCAT(self, op):
+ def optimize_CALL(self, op):
+ # dispatch based on 'oopspecindex' to a method that handles
+ # specifically the given oopspec call. For non-oopspec calls,
+ # oopspecindex is just zero.
+ effectinfo = op.getdescr().get_extra_info()
+ if effectinfo is not None:
+ oopspecindex = effectinfo.oopspecindex
+ for value, meth in opt_call_oopspec_ops:
+ if oopspecindex == value: # a match with the OS_STR_xxx
+ if meth(self, op, mode_string):
+ return
+ break
+ if oopspecindex == value + EffectInfo._OS_offset_uni:
+ # a match with the OS_UNI_xxx
+ if meth(self, op, mode_unicode):
+ return
+ break
+ if oopspecindex == EffectInfo.OS_STR2UNICODE:
+ if self.opt_call_str_STR2UNICODE(op):
+ return
+ self.emit_operation(op)
+
+ def opt_call_str_STR2UNICODE(self, op):
+ # Constant-fold unicode("constant string").
+ # More generally, supporting non-constant but virtual cases is
+ # not obvious, because of the exception UnicodeDecodeError that
+ # can be raised by ll_str2unicode()
+ varg = self.getvalue(op.getarg(1))
+ s = varg.get_constant_string_spec(mode_string)
+ if s is None:
+ return False
+ try:
+ u = unicode(s)
+ except UnicodeDecodeError:
+ return False
+ self.make_constant(op.result, get_const_ptr_for_unicode(u))
+ return True
+
+ def opt_call_stroruni_STR_CONCAT(self, op, mode):
vleft = self.getvalue(op.getarg(1))
vright = self.getvalue(op.getarg(2))
vleft.ensure_nonnull()
vright.ensure_nonnull()
newoperations = self.optimizer.newoperations
- len1box = vleft.getstrlen(newoperations)
- len2box = vright.getstrlen(newoperations)
+ len1box = vleft.getstrlen(newoperations, mode)
+ len2box = vright.getstrlen(newoperations, mode)
lengthbox = _int_add(newoperations, len1box, len2box)
- value = self.make_vstring_concat(op.result, op)
+ value = self.make_vstring_concat(op.result, op, mode)
value.setup(vleft, vright, lengthbox)
return True
- def opt_call_oopspec_STR_SLICE(self, op):
+ def opt_call_stroruni_STR_SLICE(self, op, mode):
newoperations = self.optimizer.newoperations
vstr = self.getvalue(op.getarg(1))
vstart = self.getvalue(op.getarg(2))
@@ -414,7 +497,7 @@
if (isinstance(vstr, VStringPlainValue) and vstart.is_constant()
and vstop.is_constant()):
# slicing with constant bounds of a VStringPlainValue
- value = self.make_vstring_plain(op.result, op)
+ value = self.make_vstring_plain(op.result, op, mode)
value.setup_slice(vstr._chars, vstart.box.getint(),
vstop.box.getint())
return True
@@ -432,16 +515,16 @@
vstart.force_box())
vstart = self.getvalue(startbox)
#
- value = self.make_vstring_slice(op.result, op)
+ value = self.make_vstring_slice(op.result, op, mode)
value.setup(vstr, vstart, self.getvalue(lengthbox))
return True
- def opt_call_oopspec_STR_EQUAL(self, op):
+ def opt_call_stroruni_STR_EQUAL(self, op, mode):
v1 = self.getvalue(op.getarg(1))
v2 = self.getvalue(op.getarg(2))
#
- l1box = v1.getstrlen(None)
- l2box = v2.getstrlen(None)
+ l1box = v1.getstrlen(None, mode)
+ l2box = v2.getstrlen(None, mode)
if (l1box is not None and l2box is not None and
isinstance(l1box, ConstInt) and
isinstance(l2box, ConstInt) and
@@ -450,13 +533,13 @@
self.make_constant(op.result, CONST_0)
return True
#
- if self.handle_str_equal_level1(v1, v2, op.result):
+ if self.handle_str_equal_level1(v1, v2, op.result, mode):
return True
- if self.handle_str_equal_level1(v2, v1, op.result):
+ if self.handle_str_equal_level1(v2, v1, op.result, mode):
return True
- if self.handle_str_equal_level2(v1, v2, op.result):
+ if self.handle_str_equal_level2(v1, v2, op.result, mode):
return True
- if self.handle_str_equal_level2(v2, v1, op.result):
+ if self.handle_str_equal_level2(v2, v1, op.result, mode):
return True
#
if v1.is_nonnull() and v2.is_nonnull():
@@ -468,37 +551,37 @@
else:
do = EffectInfo.OS_STREQ_NONNULL
self.generate_modified_call(do, [v1.force_box(),
- v2.force_box()], op.result)
+ v2.force_box()], op.result, mode)
return True
return False
- def handle_str_equal_level1(self, v1, v2, resultbox):
- l2box = v2.getstrlen(None)
+ def handle_str_equal_level1(self, v1, v2, resultbox, mode):
+ l2box = v2.getstrlen(None, mode)
if isinstance(l2box, ConstInt):
if l2box.value == 0:
- lengthbox = v1.getstrlen(self.optimizer.newoperations)
+ lengthbox = v1.getstrlen(self.optimizer.newoperations, mode)
seo = self.optimizer.send_extra_operation
seo(ResOperation(rop.INT_EQ, [lengthbox, CONST_0], resultbox))
return True
if l2box.value == 1:
- l1box = v1.getstrlen(None)
+ l1box = v1.getstrlen(None, mode)
if isinstance(l1box, ConstInt) and l1box.value == 1:
# comparing two single chars
- vchar1 = self.strgetitem(v1, optimizer.CVAL_ZERO)
- vchar2 = self.strgetitem(v2, optimizer.CVAL_ZERO)
+ vchar1 = self.strgetitem(v1, optimizer.CVAL_ZERO, mode)
+ vchar2 = self.strgetitem(v2, optimizer.CVAL_ZERO, mode)
seo = self.optimizer.send_extra_operation
seo(ResOperation(rop.INT_EQ, [vchar1.force_box(),
vchar2.force_box()],
resultbox))
return True
if isinstance(v1, VStringSliceValue):
- vchar = self.strgetitem(v2, optimizer.CVAL_ZERO)
+ vchar = self.strgetitem(v2, optimizer.CVAL_ZERO, mode)
do = EffectInfo.OS_STREQ_SLICE_CHAR
self.generate_modified_call(do, [v1.vstr.force_box(),
v1.vstart.force_box(),
v1.vlength.force_box(),
vchar.force_box()],
- resultbox)
+ resultbox, mode)
return True
#
if v2.is_null():
@@ -516,17 +599,18 @@
#
return False
- def handle_str_equal_level2(self, v1, v2, resultbox):
- l2box = v2.getstrlen(None)
+ def handle_str_equal_level2(self, v1, v2, resultbox, mode):
+ l2box = v2.getstrlen(None, mode)
if isinstance(l2box, ConstInt):
if l2box.value == 1:
- vchar = self.strgetitem(v2, optimizer.CVAL_ZERO)
+ vchar = self.strgetitem(v2, optimizer.CVAL_ZERO, mode)
if v1.is_nonnull():
do = EffectInfo.OS_STREQ_NONNULL_CHAR
else:
do = EffectInfo.OS_STREQ_CHECKNULL_CHAR
self.generate_modified_call(do, [v1.force_box(),
- vchar.force_box()], resultbox)
+ vchar.force_box()], resultbox,
+ mode)
return True
#
if v1.is_virtual() and isinstance(v1, VStringSliceValue):
@@ -537,11 +621,12 @@
self.generate_modified_call(do, [v1.vstr.force_box(),
v1.vstart.force_box(),
v1.vlength.force_box(),
- v2.force_box()], resultbox)
+ v2.force_box()], resultbox, mode)
return True
return False
- def generate_modified_call(self, oopspecindex, args, result):
+ def generate_modified_call(self, oopspecindex, args, result, mode):
+ oopspecindex += mode.OS_offset
calldescr, func = callinfo_for_oopspec(oopspecindex)
op = ResOperation(rop.CALL, [ConstInt(func)] + args, result,
descr=calldescr)
@@ -559,7 +644,7 @@
optimize_ops = _findall(OptString, 'optimize_')
def _findall_call_oopspec():
- prefix = 'opt_call_oopspec_'
+ prefix = 'opt_call_stroruni_'
result = []
for name in dir(OptString):
if name.startswith(prefix):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Sat Oct 30 16:41:17 2010
@@ -76,6 +76,8 @@
assert self.source_op is not None
# ^^^ This case should not occur any more (see test_bug_3).
#
+ if not we_are_translated():
+ self.source_op.name = 'FORCE ' + self.source_op.name
newoperations = self.optimizer.newoperations
newoperations.append(self.source_op)
self.box = box = self.source_op.result
@@ -148,6 +150,11 @@
fielddescrs = self._get_field_descr_list()
return modifier.make_virtual(self.known_class, fielddescrs)
+ def __repr__(self):
+ cls_name = self.known_class.value.adr.ptr._obj._TYPE._name
+ field_names = [field.name for field in self._fields]
+ return "" % (cls_name, field_names)
+
class VStructValue(AbstractVirtualStructValue):
def __init__(self, optimizer, structdescr, keybox, source_op=None):
@@ -179,6 +186,8 @@
def _really_force(self):
assert self.source_op is not None
+ if not we_are_translated():
+ self.source_op.name = 'FORCE ' + self.source_op.name
newoperations = self.optimizer.newoperations
newoperations.append(self.source_op)
self.box = box = self.source_op.result
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py Sat Oct 30 16:41:17 2010
@@ -1,4 +1,4 @@
-import py, os
+import py, os, sys
from pypy.rpython.lltypesystem import lltype, llmemory, rclass
from pypy.rlib.objectmodel import we_are_translated
from pypy.rlib.unroll import unrolling_iterable
@@ -498,6 +498,22 @@
opimpl_getfield_gc_r_pure = _opimpl_getfield_gc_pure_any
opimpl_getfield_gc_f_pure = _opimpl_getfield_gc_pure_any
+ @arguments("orgpc", "box", "descr")
+ def _opimpl_getfield_gc_greenfield_any(self, pc, box, fielddescr):
+ ginfo = self.metainterp.jitdriver_sd.greenfield_info
+ if (ginfo is not None and fielddescr in ginfo.green_field_descrs
+ and not self._nonstandard_virtualizable(pc, box)):
+ # fetch the result, but consider it as a Const box and don't
+ # record any operation
+ resbox = executor.execute(self.metainterp.cpu, self.metainterp,
+ rop.GETFIELD_GC_PURE, fielddescr, box)
+ return resbox.constbox()
+ # fall-back
+ return self.execute_with_descr(rop.GETFIELD_GC_PURE, fielddescr, box)
+ opimpl_getfield_gc_i_greenfield = _opimpl_getfield_gc_greenfield_any
+ opimpl_getfield_gc_r_greenfield = _opimpl_getfield_gc_greenfield_any
+ opimpl_getfield_gc_f_greenfield = _opimpl_getfield_gc_greenfield_any
+
@arguments("box", "descr", "box")
def _opimpl_setfield_gc_any(self, box, fielddescr, valuebox):
self.execute_with_descr(rop.SETFIELD_GC, fielddescr, box, valuebox)
@@ -529,7 +545,8 @@
def _nonstandard_virtualizable(self, pc, box):
# returns True if 'box' is actually not the "standard" virtualizable
# that is stored in metainterp.virtualizable_boxes[-1]
- if self.metainterp.jitdriver_sd.virtualizable_info is None:
+ if (self.metainterp.jitdriver_sd.virtualizable_info is None and
+ self.metainterp.jitdriver_sd.greenfield_info is None):
return True # can occur in case of multiple JITs
standard_box = self.metainterp.virtualizable_boxes[-1]
if standard_box is box:
@@ -799,12 +816,16 @@
@arguments("orgpc", "int", "boxes3", "boxes3")
def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes, redboxes):
+ any_operation = len(self.metainterp.history.operations) > 0
jitdriver_sd = self.metainterp.staticdata.jitdrivers_sd[jdindex]
self.verify_green_args(jitdriver_sd, greenboxes)
# xxx we may disable the following line in some context later
self.debug_merge_point(jitdriver_sd, greenboxes)
if self.metainterp.seen_loop_header_for_jdindex < 0:
- return
+ if not jitdriver_sd.no_loop_header or not any_operation:
+ return
+ # automatically add a loop_header if there is none
+ self.metainterp.seen_loop_header_for_jdindex = jdindex
#
assert self.metainterp.seen_loop_header_for_jdindex == jdindex, (
"found a loop_header for a JitDriver that does not match "
@@ -893,6 +914,40 @@
msg = box.getref(lltype.Ptr(rstr.STR))
lloperation.llop.debug_fatalerror(msg)
+ @arguments("box", "box", "box", "box", "box")
+ def opimpl_jit_debug(self, stringbox, arg1box, arg2box, arg3box, arg4box):
+ from pypy.rpython.lltypesystem import rstr
+ from pypy.rpython.annlowlevel import hlstr
+ msg = stringbox.getref(lltype.Ptr(rstr.STR))
+ debug_print('jit_debug:', hlstr(msg),
+ arg1box.getint(), arg2box.getint(),
+ arg3box.getint(), arg4box.getint())
+ args = [stringbox, arg1box, arg2box, arg3box, arg4box]
+ i = 4
+ while i > 0 and args[i].getint() == -sys.maxint-1:
+ i -= 1
+ assert i >= 0
+ op = self.metainterp.history.record(rop.JIT_DEBUG, args[:i+1], None)
+ self.metainterp.attach_debug_info(op)
+
+ @arguments("box")
+ def _opimpl_assert_green(self, box):
+ if not isinstance(box, Const):
+ msg = "assert_green failed at %s:%d" % (
+ self.jitcode.name,
+ self.pc)
+ if we_are_translated():
+ from pypy.rpython.annlowlevel import llstr
+ from pypy.rpython.lltypesystem import lloperation
+ lloperation.llop.debug_fatalerror(lltype.Void, llstr(msg))
+ else:
+ from pypy.rlib.jit import AssertGreenFailed
+ raise AssertGreenFailed(msg)
+
+ opimpl_int_assert_green = _opimpl_assert_green
+ opimpl_ref_assert_green = _opimpl_assert_green
+ opimpl_float_assert_green = _opimpl_assert_green
+
@arguments("box")
def opimpl_virtual_ref(self, box):
# Details on the content of metainterp.virtualref_boxes:
@@ -998,7 +1053,8 @@
guard_op = metainterp.history.record(opnum, moreargs, None,
descr=resumedescr)
virtualizable_boxes = None
- if metainterp.jitdriver_sd.virtualizable_info is not None:
+ if (metainterp.jitdriver_sd.virtualizable_info is not None or
+ metainterp.jitdriver_sd.greenfield_info is not None):
virtualizable_boxes = metainterp.virtualizable_boxes
saved_pc = self.pc
if resumepc >= 0:
@@ -1646,6 +1702,7 @@
duplicates)
live_arg_boxes += self.virtualizable_boxes
live_arg_boxes.pop()
+ #
assert len(self.virtualref_boxes) == 0, "missing virtual_ref_finish()?"
# Called whenever we reach the 'loop_header' hint.
# First, attempt to make a bridge:
@@ -1832,6 +1889,7 @@
f.setup_call(original_boxes)
assert self.in_recursion == 0
self.virtualref_boxes = []
+ self.initialize_withgreenfields(original_boxes)
self.initialize_virtualizable(original_boxes)
def initialize_state_from_guard_failure(self, resumedescr):
@@ -1856,6 +1914,14 @@
self.virtualizable_boxes.append(virtualizable_box)
self.initialize_virtualizable_enter()
+ def initialize_withgreenfields(self, original_boxes):
+ ginfo = self.jitdriver_sd.greenfield_info
+ if ginfo is not None:
+ assert self.jitdriver_sd.virtualizable_info is None
+ index = (self.jitdriver_sd.num_green_args +
+ ginfo.red_index)
+ self.virtualizable_boxes = [original_boxes[index]]
+
def initialize_virtualizable_enter(self):
vinfo = self.jitdriver_sd.virtualizable_info
virtualizable_box = self.virtualizable_boxes[-1]
@@ -1949,8 +2015,10 @@
def rebuild_state_after_failure(self, resumedescr):
vinfo = self.jitdriver_sd.virtualizable_info
+ ginfo = self.jitdriver_sd.greenfield_info
self.framestack = []
- boxlists = resume.rebuild_from_resumedata(self, resumedescr, vinfo)
+ boxlists = resume.rebuild_from_resumedata(self, resumedescr, vinfo,
+ ginfo)
inputargs_and_holes, virtualizable_boxes, virtualref_boxes = boxlists
#
# virtual refs: make the vrefs point to the freshly allocated virtuals
@@ -1975,6 +2043,12 @@
assert not virtualizable.vable_token
# fill the virtualizable with the local boxes
self.synchronize_virtualizable()
+ #
+ elif self.jitdriver_sd.greenfield_info:
+ self.virtualizable_boxes = virtualizable_boxes
+ else:
+ assert not virtualizable_boxes
+ #
return inputargs_and_holes
def check_synchronized_virtualizable(self):
@@ -2048,7 +2122,8 @@
for i in range(len(boxes)):
if boxes[i] is oldbox:
boxes[i] = newbox
- if self.jitdriver_sd.virtualizable_info is not None:
+ if (self.jitdriver_sd.virtualizable_info is not None or
+ self.jitdriver_sd.greenfield_info is not None):
boxes = self.virtualizable_boxes
for i in range(len(boxes)):
if boxes[i] is oldbox:
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resoperation.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resoperation.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resoperation.py Sat Oct 30 16:41:17 2010
@@ -93,7 +93,7 @@
def __repr__(self):
return self.repr()
- def repr(self):
+ def repr(self, graytext=False):
# RPython-friendly version
if self.result is not None:
sres = '%s = ' % (self.result,)
@@ -101,6 +101,8 @@
sres = ''
if self.name:
prefix = "%s:%s " % (self.name, self.pc)
+ if graytext:
+ prefix = "\f%s\f" % prefix
else:
prefix = ""
args = self.getarglist()
@@ -455,15 +457,16 @@
'UNICODESETITEM/3',
'NEWUNICODE/1',
#'RUNTIMENEW/1', # ootype operation
- 'COND_CALL_GC_WB/1d', # [objptr] (for the write barrier)
+ 'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier)
'DEBUG_MERGE_POINT/1', # debugging only
+ 'JIT_DEBUG/*', # debugging only
'VIRTUAL_REF_FINISH/2', # removed before it's passed to the backend
'COPYSTRCONTENT/5', # src, dst, srcstart, dststart, length
'COPYUNICODECONTENT/5',
'_CANRAISE_FIRST', # ----- start of can_raise operations -----
'CALL/*d',
- 'CALL_ASSEMBLER/*d',
+ 'CALL_ASSEMBLER/*d', # call already compiled assembler
'CALL_MAY_FORCE/*d',
'CALL_LOOPINVARIANT/*d',
#'OOSEND', # ootype operation
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py Sat Oct 30 16:41:17 2010
@@ -255,13 +255,19 @@
def make_varray(self, arraydescr):
return VArrayInfo(arraydescr)
- def make_vstrplain(self):
+ def make_vstrplain(self, is_unicode=False):
+ if is_unicode:
+ return VUniPlainInfo()
return VStrPlainInfo()
- def make_vstrconcat(self):
+ def make_vstrconcat(self, is_unicode=False):
+ if is_unicode:
+ return VUniConcatInfo()
return VStrConcatInfo()
- def make_vstrslice(self):
+ def make_vstrslice(self, is_unicode=False):
+ if is_unicode:
+ return VUniSliceInfo()
return VStrSliceInfo()
def register_virtual_fields(self, virtualbox, fieldboxes):
@@ -550,6 +556,60 @@
for i in self.fieldnums:
debug_print("\t\t", str(untag(i)))
+
+class VUniPlainInfo(AbstractVirtualInfo):
+ """Stands for the unicode string made out of the characters of all
+ fieldnums."""
+
+ @specialize.argtype(1)
+ def allocate(self, decoder, index):
+ length = len(self.fieldnums)
+ string = decoder.allocate_unicode(length)
+ decoder.virtuals_cache[index] = string
+ for i in range(length):
+ decoder.unicode_setitem(string, i, self.fieldnums[i])
+ return string
+
+ def debug_prints(self):
+ debug_print("\tvuniplaininfo length", len(self.fieldnums))
+
+
+class VUniConcatInfo(AbstractVirtualInfo):
+ """Stands for the unicode string made out of the concatenation of two
+ other unicode strings."""
+
+ @specialize.argtype(1)
+ def allocate(self, decoder, index):
+ # xxx for blackhole resuming, this will build all intermediate
+ # strings and throw them away immediately, which is a bit sub-
+ # efficient. Not sure we care.
+ left, right = self.fieldnums
+ string = decoder.concat_unicodes(left, right)
+ decoder.virtuals_cache[index] = string
+ return string
+
+ def debug_prints(self):
+ debug_print("\tvuniconcatinfo")
+ for i in self.fieldnums:
+ debug_print("\t\t", str(untag(i)))
+
+
+class VUniSliceInfo(AbstractVirtualInfo):
+ """Stands for the unicode string made out of slicing another
+ unicode string."""
+
+ @specialize.argtype(1)
+ def allocate(self, decoder, index):
+ largerstr, start, length = self.fieldnums
+ string = decoder.slice_unicode(largerstr, start, length)
+ decoder.virtuals_cache[index] = string
+ return string
+
+ def debug_prints(self):
+ debug_print("\tvunisliceinfo")
+ for i in self.fieldnums:
+ debug_print("\t\t", str(untag(i)))
+
# ____________________________________________________________
class AbstractResumeDataReader(object):
@@ -629,9 +689,11 @@
# ---------- when resuming for pyjitpl.py, make boxes ----------
-def rebuild_from_resumedata(metainterp, storage, virtualizable_info):
+def rebuild_from_resumedata(metainterp, storage, virtualizable_info,
+ greenfield_info):
resumereader = ResumeDataBoxReader(storage, metainterp)
- boxes = resumereader.consume_vref_and_vable_boxes(virtualizable_info)
+ boxes = resumereader.consume_vref_and_vable_boxes(virtualizable_info,
+ greenfield_info)
virtualizable_boxes, virtualref_boxes = boxes
frameinfo = storage.rd_frame_info_list
while True:
@@ -676,15 +738,18 @@
assert (end & 1) == 0
return [self.decode_ref(nums[i]) for i in range(end)]
- def consume_vref_and_vable_boxes(self, vinfo):
+ def consume_vref_and_vable_boxes(self, vinfo, ginfo):
nums = self.cur_numb.nums
self.cur_numb = self.cur_numb.prev
- if vinfo is None:
- virtualizable_boxes = None
- end = len(nums)
- else:
+ if vinfo is not None:
virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, nums)
end = len(nums) - len(virtualizable_boxes)
+ elif ginfo is not None:
+ virtualizable_boxes = [self.decode_ref(nums[-1])]
+ end = len(nums) - 1
+ else:
+ virtualizable_boxes = None
+ end = len(nums)
virtualref_boxes = self.consume_virtualref_boxes(nums, end)
return virtualizable_boxes, virtualref_boxes
@@ -725,6 +790,32 @@
return self.metainterp.execute_and_record_varargs(
rop.CALL, [ConstInt(func), strbox, startbox, stopbox], calldescr)
+ def allocate_unicode(self, length):
+ return self.metainterp.execute_and_record(rop.NEWUNICODE,
+ None, ConstInt(length))
+
+ def unicode_setitem(self, strbox, index, charnum):
+ charbox = self.decode_box(charnum, INT)
+ self.metainterp.execute_and_record(rop.UNICODESETITEM, None,
+ strbox, ConstInt(index), charbox)
+
+ def concat_unicodes(self, str1num, str2num):
+ calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT)
+ str1box = self.decode_box(str1num, REF)
+ str2box = self.decode_box(str2num, REF)
+ return self.metainterp.execute_and_record_varargs(
+ rop.CALL, [ConstInt(func), str1box, str2box], calldescr)
+
+ def slice_unicode(self, strnum, startnum, lengthnum):
+ calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE)
+ strbox = self.decode_box(strnum, REF)
+ startbox = self.decode_box(startnum, INT)
+ lengthbox = self.decode_box(lengthnum, INT)
+ stopbox = self.metainterp.execute_and_record(rop.INT_ADD, None,
+ startbox, lengthbox)
+ return self.metainterp.execute_and_record_varargs(
+ rop.CALL, [ConstInt(func), strbox, startbox, stopbox], calldescr)
+
def setfield(self, descr, structbox, fieldnum):
if descr.is_pointer_field():
kind = REF
@@ -815,8 +906,9 @@
resumereader = ResumeDataDirectReader(blackholeinterpbuilder.cpu, storage,
all_virtuals)
vinfo = jitdriver_sd.virtualizable_info
+ ginfo = jitdriver_sd.greenfield_info
vrefinfo = blackholeinterpbuilder.metainterp_sd.virtualref_info
- resumereader.consume_vref_and_vable(vrefinfo, vinfo)
+ resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo)
#
# First get a chain of blackhole interpreters whose length is given
# by the depth of rd_frame_info_list. The first one we get must be
@@ -846,11 +938,11 @@
resumereader.done()
return firstbh
-def force_from_resumedata(metainterp_sd, storage, vinfo=None):
+def force_from_resumedata(metainterp_sd, storage, vinfo, ginfo):
resumereader = ResumeDataDirectReader(metainterp_sd.cpu, storage)
resumereader.handling_async_forcing()
vrefinfo = metainterp_sd.virtualref_info
- resumereader.consume_vref_and_vable(vrefinfo, vinfo)
+ resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo)
return resumereader.force_all_virtuals()
class ResumeDataDirectReader(AbstractResumeDataReader):
@@ -925,11 +1017,12 @@
return specialize_value(TYPE, x)
load_value_of_type._annspecialcase_ = 'specialize:arg(1)'
- def consume_vref_and_vable(self, vrefinfo, vinfo):
+ def consume_vref_and_vable(self, vrefinfo, vinfo, ginfo):
nums = self.cur_numb.nums
self.cur_numb = self.cur_numb.prev
if self.resume_after_guard_not_forced != 2:
end_vref = self.consume_vable_info(vinfo, nums)
+ if ginfo is not None: end_vref -= 1
self.consume_virtualref_info(vrefinfo, nums, end_vref)
def allocate_with_vtable(self, known_class):
@@ -967,6 +1060,31 @@
result = funcptr(str, start, start + length)
return lltype.cast_opaque_ptr(llmemory.GCREF, result)
+ def allocate_unicode(self, length):
+ return self.cpu.bh_newunicode(length)
+
+ def unicode_setitem(self, str, index, charnum):
+ char = self.decode_int(charnum)
+ self.cpu.bh_unicodesetitem(str, index, char)
+
+ def concat_unicodes(self, str1num, str2num):
+ str1 = self.decode_ref(str1num)
+ str2 = self.decode_ref(str2num)
+ str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str1)
+ str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str2)
+ funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT)
+ result = funcptr(str1, str2)
+ return lltype.cast_opaque_ptr(llmemory.GCREF, result)
+
+ def slice_unicode(self, strnum, startnum, lengthnum):
+ str = self.decode_ref(strnum)
+ start = self.decode_int(startnum)
+ length = self.decode_int(lengthnum)
+ str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str)
+ funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE)
+ result = funcptr(str, start, start + length)
+ return lltype.cast_opaque_ptr(llmemory.GCREF, result)
+
def setfield(self, descr, struct, fieldnum):
if descr.is_pointer_field():
newvalue = self.decode_ref(fieldnum)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/simple_optimize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/simple_optimize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/simple_optimize.py Sat Oct 30 16:41:17 2010
@@ -9,11 +9,13 @@
def transform(op):
from pypy.jit.metainterp.history import AbstractDescr
- # Rename CALL_PURE to CALL.
+ # Rename CALL_PURE and CALL_LOOPINVARIANT to CALL.
# Simplify the VIRTUAL_REF_* so that they don't show up in the backend.
if op.getopnum() == rop.CALL_PURE:
op = ResOperation(rop.CALL, op.getarglist()[1:], op.result,
op.getdescr())
+ elif op.getopnum() == rop.CALL_LOOPINVARIANT:
+ op = op.copy_and_change(rop.CALL)
elif op.getopnum() == rop.VIRTUAL_REF:
op = ResOperation(rop.SAME_AS, [op.getarg(0)], op.result)
elif op.getopnum() == rop.VIRTUAL_REF_FINISH:
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py Sat Oct 30 16:41:17 2010
@@ -2,6 +2,7 @@
import sys
from pypy.rlib.jit import JitDriver, we_are_jitted, hint, dont_look_inside
from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_SIMPLE, loop_invariant
+from pypy.rlib.jit import jit_debug, assert_green, AssertGreenFailed
from pypy.jit.metainterp.warmspot import ll_meta_interp, get_stats
from pypy.jit.backend.llgraph import runner
from pypy.jit.metainterp import pyjitpl, history
@@ -44,6 +45,7 @@
num_green_args = 0
portal_graph = graphs[0]
virtualizable_info = None
+ greenfield_info = None
result_type = result_kind
portal_runner_ptr = "???"
@@ -1775,6 +1777,33 @@
res = self.interp_operations(f, [10, 3.5])
assert res == 3.5
+ def test_jit_debug(self):
+ myjitdriver = JitDriver(greens = [], reds = ['x'])
+ class A:
+ pass
+ def f(x):
+ while x > 0:
+ myjitdriver.can_enter_jit(x=x)
+ myjitdriver.jit_merge_point(x=x)
+ jit_debug("hi there:", x)
+ jit_debug("foobar")
+ x -= 1
+ return x
+ res = self.meta_interp(f, [8])
+ assert res == 0
+ self.check_loops(jit_debug=2)
+
+ def test_assert_green(self):
+ def f(x, promote):
+ if promote:
+ x = hint(x, promote=True)
+ assert_green(x)
+ return x
+ res = self.interp_operations(f, [8, 1])
+ assert res == 8
+ py.test.raises(AssertGreenFailed, self.interp_operations, f, [8, 0])
+
+
class TestOOtype(BasicTests, OOJitMixin):
def test_oohash(self):
@@ -1882,7 +1911,7 @@
c = bool(p1)
d = not bool(p2)
return 1000*a + 100*b + 10*c + d
- prebuilt = [lltype.malloc(TP, flavor='raw')] * 2
+ prebuilt = [lltype.malloc(TP, flavor='raw', immortal=True)] * 2
expected = f(0, 1)
assert self.interp_operations(f, [0, 1]) == expected
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py Sat Oct 30 16:41:17 2010
@@ -6,8 +6,8 @@
from pypy.jit.metainterp.compile import ResumeGuardCountersInt
from pypy.jit.metainterp.compile import compile_tmp_callback
from pypy.jit.metainterp import optimize_nopspec, jitprof, typesystem, compile
-from pypy.jit.metainterp.test.oparser import parse
from pypy.jit.metainterp.test.test_optimizefindnode import LLtypeMixin
+from pypy.jit.tool.oparser import parse
def test_insert_loop_token():
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_jitdriver.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_jitdriver.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_jitdriver.py Sat Oct 30 16:41:17 2010
@@ -72,6 +72,33 @@
# we expect no loop at all for 'loop1': it should always be inlined
self.check_tree_loop_count(2) # 1 x loop, 1 x enter bridge
+ def test_inactive_jitdriver(self):
+ myjitdriver1 = JitDriver(greens=[], reds=['n', 'm'],
+ get_printable_location = getloc1)
+ myjitdriver2 = JitDriver(greens=['g'], reds=['r'],
+ get_printable_location = getloc2)
+ #
+ myjitdriver1.active = False # <===
+ #
+ def loop1(n, m):
+ while n > 0:
+ myjitdriver1.can_enter_jit(n=n, m=m)
+ myjitdriver1.jit_merge_point(n=n, m=m)
+ n -= m
+ return n
+ #
+ def loop2(g, r):
+ while r > 0:
+ myjitdriver2.can_enter_jit(g=g, r=r)
+ myjitdriver2.jit_merge_point(g=g, r=r)
+ r += loop1(r, g) + (-1)
+ return r
+ #
+ res = self.meta_interp(loop2, [4, 40], repeat=7, inline=True)
+ assert res == loop2(4, 40)
+ # we expect no int_sub, but a residual call
+ self.check_loops(int_sub=0, call=1)
+
class TestLLtype(MultipleJitDriversTests, LLJitMixin):
pass
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_logger.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_logger.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_logger.py Sat Oct 30 16:41:17 2010
@@ -1,6 +1,6 @@
import sys
from pypy.rlib import debug
-from pypy.jit.metainterp.test.oparser import pure_parse
+from pypy.jit.tool.oparser import pure_parse
from pypy.jit.metainterp import logger
from pypy.jit.metainterp.typesystem import llhelper
from StringIO import StringIO
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_oparser.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_oparser.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_oparser.py Sat Oct 30 16:41:17 2010
@@ -1,7 +1,7 @@
from pypy.rpython.lltypesystem import lltype, llmemory
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.jit.metainterp.resoperation import rop
from pypy.jit.metainterp.history import AbstractDescr, BoxInt, LoopToken,\
BoxFloat
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py Sat Oct 30 16:41:17 2010
@@ -131,14 +131,21 @@
# ____________________________________________________________
def equaloplists(oplist1, oplist2, strict_fail_args=True, remap={}):
- print '-'*20, 'Comparing lists', '-'*20
+ # try to use the full width of the terminal to display the list
+ # unfortunately, does not work with the default capture method of py.test
+ # (which is fd), you you need to use either -s or --capture=sys, else you
+ # get the standard 80 columns width
+ totwidth = py.io.get_terminal_width()
+ width = totwidth / 2 - 1
+ print ' Comparing lists '.center(totwidth, '-')
+ print '%s| %s' % ('optimized'.center(width), 'expected'.center(width))
for op1, op2 in zip(oplist1, oplist2):
txt1 = str(op1)
txt2 = str(op2)
while txt1 or txt2:
- print '%-39s| %s' % (txt1[:39], txt2[:39])
- txt1 = txt1[39:]
- txt2 = txt2[39:]
+ print '%s| %s' % (txt1[:width].ljust(width), txt2[:width])
+ txt1 = txt1[width:]
+ txt2 = txt2[width:]
assert op1.getopnum() == op2.getopnum()
assert op1.numargs() == op2.numargs()
for i in range(op1.numargs()):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizefindnode.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizefindnode.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizefindnode.py Sat Oct 30 16:41:17 2010
@@ -18,7 +18,7 @@
from pypy.jit.metainterp.specnode import ConstantSpecNode
from pypy.jit.codewriter.effectinfo import EffectInfo
from pypy.jit.codewriter.heaptracker import register_known_gctype, adr2int
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
def test_sort_descrs():
class PseudoDescr(AbstractDescr):
@@ -117,33 +117,32 @@
EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE))
arraycopydescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
EffectInfo([], [], [], oopspecindex=EffectInfo.OS_ARRAYCOPY))
- strconcatdescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_CONCAT))
- slicedescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_SLICE))
- strequaldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_EQUAL))
- streq_slice_checknull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_CHECKNULL))
- streq_slice_nonnull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_NONNULL))
- streq_slice_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_CHAR))
- streq_nonnull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_NONNULL))
- streq_nonnull_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_NONNULL_CHAR))
- streq_checknull_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_CHECKNULL_CHAR))
- streq_lengthok_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_LENGTHOK))
+
+ for _name, _os in [
+ ('strconcatdescr', 'OS_STR_CONCAT'),
+ ('strslicedescr', 'OS_STR_SLICE'),
+ ('strequaldescr', 'OS_STR_EQUAL'),
+ ('streq_slice_checknull_descr', 'OS_STREQ_SLICE_CHECKNULL'),
+ ('streq_slice_nonnull_descr', 'OS_STREQ_SLICE_NONNULL'),
+ ('streq_slice_char_descr', 'OS_STREQ_SLICE_CHAR'),
+ ('streq_nonnull_descr', 'OS_STREQ_NONNULL'),
+ ('streq_nonnull_char_descr', 'OS_STREQ_NONNULL_CHAR'),
+ ('streq_checknull_char_descr', 'OS_STREQ_CHECKNULL_CHAR'),
+ ('streq_lengthok_descr', 'OS_STREQ_LENGTHOK'),
+ ]:
+ _oopspecindex = getattr(EffectInfo, _os)
+ locals()[_name] = \
+ cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=_oopspecindex))
+ #
+ _oopspecindex = getattr(EffectInfo, _os.replace('STR', 'UNI'))
+ locals()[_name.replace('str', 'unicode')] = \
+ cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=_oopspecindex))
+
+ s2u_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR2UNICODE))
+ #
class LoopToken(AbstractDescr):
pass
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 16:41:17 2010
@@ -12,7 +12,7 @@
from pypy.jit.metainterp.jitprof import EmptyProfiler
from pypy.jit.metainterp import executor, compile, resume, history
from pypy.jit.metainterp.resoperation import rop, opname, ResOperation
-from pypy.jit.metainterp.test.oparser import pure_parse
+from pypy.jit.tool.oparser import pure_parse
from pypy.jit.metainterp.test.test_optimizebasic import equaloplists
class Fake(object):
@@ -28,7 +28,6 @@
self.globaldata = Fake()
# ____________________________________________________________
-
class Storage(compile.ResumeGuardDescr):
"for tests."
def __init__(self, metainterp_sd=None, original_greenkey=None):
@@ -93,6 +92,10 @@
expected = self.parse(expected_preamble)
self.assert_equal(loop.preamble, expected)
+ return loop
+
+class OptimizeOptTest(BaseTestOptimizeOpt):
+
def test_simple(self):
ops = """
[]
@@ -2609,7 +2612,7 @@
''', rop.GUARD_TRUE)
-class TestLLtype(BaseTestOptimizeOpt, LLtypeMixin):
+class TestLLtype(OptimizeOptTest, LLtypeMixin):
def test_residual_call_does_not_invalidate_caches(self):
ops = """
@@ -3455,7 +3458,7 @@
i0 = strlen(p0)
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not', expected)
def test_addsub_const(self):
ops = """
@@ -3900,6 +3903,15 @@
"""
self.optimize_loop(ops, 'Not, Not', expected)
+ # ----------
+ def optimize_strunicode_loop(self, ops, spectext, optops):
+ # check with the arguments passed in
+ self.optimize_loop(ops, spectext, optops)
+ # check with replacing 'str' with 'unicode' everywhere
+ self.optimize_loop(ops.replace('str','unicode').replace('s"', 'u"'),
+ spectext,
+ optops.replace('str','unicode').replace('s"', 'u"'))
+
def test_newstr_1(self):
ops = """
[i0]
@@ -3912,7 +3924,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not', expected)
def test_newstr_2(self):
ops = """
@@ -3928,7 +3940,7 @@
[i0, i1]
jump(i1, i0)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_concat_1(self):
ops = """
@@ -3949,7 +3961,7 @@
copystrcontent(p2, p3, 0, i4, i5)
jump(p2, p3)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_concat_vstr2_str(self):
ops = """
@@ -3972,7 +3984,7 @@
copystrcontent(p2, p3, 0, 2, i4)
jump(i1, i0, p3)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_concat_str_vstr2(self):
ops = """
@@ -3996,7 +4008,7 @@
i6 = int_add(i5, 1) # will be killed by the backend
jump(i1, i0, p3)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_concat_str_str_str(self):
ops = """
@@ -4023,12 +4035,12 @@
copystrcontent(p3, p5, 0, i12b, i3b)
jump(p2, p3, p5)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_concat_str_cstr1(self):
ops = """
[p2]
- p3 = call(0, p2, "x", descr=strconcatdescr)
+ p3 = call(0, p2, s"x", descr=strconcatdescr)
jump(p3)
"""
expected = """
@@ -4042,28 +4054,28 @@
i5 = int_add(i4, 1) # will be killed by the backend
jump(p3)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not', expected)
def test_str_concat_consts(self):
ops = """
[]
- p1 = same_as("ab")
- p2 = same_as("cde")
+ p1 = same_as(s"ab")
+ p2 = same_as(s"cde")
p3 = call(0, p1, p2, descr=strconcatdescr)
escape(p3)
jump()
"""
expected = """
[]
- escape("abcde")
+ escape(s"abcde")
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_strunicode_loop(ops, '', expected)
def test_str_slice_1(self):
ops = """
[p1, i1, i2]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
jump(p2, i1, i2)
"""
expected = """
@@ -4073,12 +4085,12 @@
copystrcontent(p1, p2, i1, 0, i3)
jump(p2, i1, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
def test_str_slice_2(self):
ops = """
[p1, i2]
- p2 = call(0, p1, 0, i2, descr=slicedescr)
+ p2 = call(0, p1, 0, i2, descr=strslicedescr)
jump(p2, i2)
"""
expected = """
@@ -4087,13 +4099,13 @@
copystrcontent(p1, p2, 0, 0, i2)
jump(p2, i2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_slice_3(self):
ops = """
[p1, i1, i2, i3, i4]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
- p3 = call(0, p2, i3, i4, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
+ p3 = call(0, p2, i3, i4, descr=strslicedescr)
jump(p3, i1, i2, i3, i4)
"""
expected = """
@@ -4105,12 +4117,12 @@
copystrcontent(p1, p3, i6, 0, i5)
jump(p3, i1, i2, i3, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not, Not', expected)
def test_str_slice_getitem1(self):
ops = """
[p1, i1, i2, i3]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
i4 = strgetitem(p2, i3)
escape(i4)
jump(p1, i1, i2, i3)
@@ -4123,7 +4135,7 @@
escape(i4)
jump(p1, i1, i2, i3)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not', expected)
def test_str_slice_plain(self):
ops = """
@@ -4131,7 +4143,7 @@
p1 = newstr(2)
strsetitem(p1, 0, i3)
strsetitem(p1, 1, i4)
- p2 = call(0, p1, 1, 2, descr=slicedescr)
+ p2 = call(0, p1, 1, 2, descr=strslicedescr)
i5 = strgetitem(p2, 0)
escape(i5)
jump(i3, i4)
@@ -4141,12 +4153,12 @@
escape(i4)
jump(i3, i4)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not', expected)
def test_str_slice_concat(self):
ops = """
[p1, i1, i2, p2]
- p3 = call(0, p1, i1, i2, descr=slicedescr)
+ p3 = call(0, p1, i1, i2, descr=strslicedescr)
p4 = call(0, p3, p2, descr=strconcatdescr)
jump(p4, i1, i2, p2)
"""
@@ -4162,10 +4174,10 @@
copystrcontent(p2, p4, 0, i3, i4b)
jump(p4, i1, i2, p2)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not', expected)
# ----------
- def optimize_loop_extradescrs(self, ops, spectext, optops):
+ def optimize_strunicode_loop_extradescrs(self, ops, spectext, optops):
from pypy.jit.metainterp.optimizeopt import string
def my_callinfo_for_oopspec(oopspecindex):
calldescrtype = type(LLtypeMixin.strequaldescr)
@@ -4180,7 +4192,7 @@
saved = string.callinfo_for_oopspec
try:
string.callinfo_for_oopspec = my_callinfo_for_oopspec
- self.optimize_loop(ops, spectext, optops)
+ self.optimize_strunicode_loop(ops, spectext, optops)
finally:
string.callinfo_for_oopspec = saved
@@ -4191,7 +4203,7 @@
escape(i0)
jump(p1, p2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', ops)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', ops)
def test_str_equal_noop2(self):
ops = """
@@ -4216,12 +4228,13 @@
escape(i0)
jump(p1, p2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not',
+ expected)
def test_str_equal_slice1(self):
ops = """
[p1, i1, i2, p3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p4, p3, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4233,12 +4246,13 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_slice2(self):
ops = """
[p1, i1, i2, p3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p3, p4, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4250,13 +4264,14 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_slice3(self):
ops = """
[p1, i1, i2, p3]
guard_nonnull(p3) []
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p3, p4, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4269,13 +4284,14 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_slice4(self):
ops = """
[p1, i1, i2]
- p3 = call(0, p1, i1, i2, descr=slicedescr)
- i0 = call(0, p3, "x", descr=strequaldescr)
+ p3 = call(0, p1, i1, i2, descr=strslicedescr)
+ i0 = call(0, p3, s"x", descr=strequaldescr)
escape(i0)
jump(p1, i1, i2)
"""
@@ -4286,12 +4302,13 @@
escape(i0)
jump(p1, i1, i2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not',
+ expected)
def test_str_equal_slice5(self):
ops = """
[p1, i1, i2, i3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
p5 = newstr(1)
strsetitem(p5, 0, i3)
i0 = call(0, p5, p4, descr=strequaldescr)
@@ -4305,7 +4322,8 @@
escape(i0)
jump(p1, i1, i2, i3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ expected)
def test_str_equal_none1(self):
ops = """
@@ -4320,7 +4338,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_none2(self):
ops = """
@@ -4335,30 +4353,30 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull1(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "hello world", descr=strequaldescr)
+ i0 = call(0, p1, s"hello world", descr=strequaldescr)
escape(i0)
jump(p1)
"""
expected = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "hello world", descr=streq_nonnull_descr)
+ i0 = call(0, p1, s"hello world", descr=streq_nonnull_descr)
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull2(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "", descr=strequaldescr)
+ i0 = call(0, p1, s"", descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4370,13 +4388,13 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull3(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "x", descr=strequaldescr)
+ i0 = call(0, p1, s"x", descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4387,13 +4405,13 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_nonnull4(self):
ops = """
[p1, p2]
p4 = call(0, p1, p2, descr=strconcatdescr)
- i0 = call(0, "hello world", p4, descr=strequaldescr)
+ i0 = call(0, s"hello world", p4, descr=strequaldescr)
escape(i0)
jump(p1, p2)
"""
@@ -4408,17 +4426,17 @@
i5 = strlen(p2)
i6 = int_add(i4, i5) # will be killed by the backend
copystrcontent(p2, p4, 0, i4, i5)
- i0 = call(0, "hello world", p4, descr=streq_nonnull_descr)
+ i0 = call(0, s"hello world", p4, descr=streq_nonnull_descr)
escape(i0)
jump(p1, p2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', expected)
def test_str_equal_chars0(self):
ops = """
[i1]
p1 = newstr(0)
- i0 = call(0, p1, "", descr=strequaldescr)
+ i0 = call(0, p1, s"", descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4427,14 +4445,14 @@
escape(1)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_chars1(self):
ops = """
[i1]
p1 = newstr(1)
strsetitem(p1, 0, i1)
- i0 = call(0, p1, "x", descr=strequaldescr)
+ i0 = call(0, p1, s"x", descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4444,7 +4462,7 @@
escape(i0)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_chars2(self):
ops = """
@@ -4452,7 +4470,7 @@
p1 = newstr(2)
strsetitem(p1, 0, i1)
strsetitem(p1, 1, i2)
- i0 = call(0, p1, "xy", descr=strequaldescr)
+ i0 = call(0, p1, s"xy", descr=strequaldescr)
escape(i0)
jump(i1, i2)
"""
@@ -4461,16 +4479,16 @@
p1 = newstr(2)
strsetitem(p1, 0, i1)
strsetitem(p1, 1, i2)
- i0 = call(0, p1, "xy", descr=streq_lengthok_descr)
+ i0 = call(0, p1, s"xy", descr=streq_lengthok_descr)
escape(i0)
jump(i1, i2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', expected)
def test_str_equal_chars3(self):
ops = """
[p1]
- i0 = call(0, "x", p1, descr=strequaldescr)
+ i0 = call(0, s"x", p1, descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4480,14 +4498,14 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
def test_str_equal_lengthmismatch1(self):
ops = """
[i1]
p1 = newstr(1)
strsetitem(p1, 0, i1)
- i0 = call(0, "xy", p1, descr=strequaldescr)
+ i0 = call(0, s"xy", p1, descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4496,13 +4514,36 @@
escape(0)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
- # XXX unicode operations
- # XXX str2unicode
+ def test_str2unicode_constant(self):
+ ops = """
+ []
+ p0 = call(0, "xy", descr=s2u_descr) # string -> unicode
+ escape(p0)
+ jump()
+ """
+ expected = """
+ []
+ escape(u"xy")
+ jump()
+ """
+ self.optimize_strunicode_loop_extradescrs(ops, '', expected)
+
+ def test_str2unicode_nonconstant(self):
+ ops = """
+ [p0]
+ p1 = call(0, p0, descr=s2u_descr) # string -> unicode
+ escape(p1)
+ jump(p1)
+ """
+ self.optimize_strunicode_loop_extradescrs(ops, 'Not', ops)
+ # more generally, supporting non-constant but virtual cases is
+ # not obvious, because of the exception UnicodeDecodeError that
+ # can be raised by ll_str2unicode()
-##class TestOOtype(BaseTestOptimizeOpt, OOtypeMixin):
+##class TestOOtype(OptimizeOptTest, OOtypeMixin):
## def test_instanceof(self):
## ops = """
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py Sat Oct 30 16:41:17 2010
@@ -240,6 +240,17 @@
return FakeBuiltObject(strconcat=[left, right])
def slice_string(self, str, start, length):
return FakeBuiltObject(strslice=[str, start, length])
+ def allocate_unicode(self, length):
+ return FakeBuiltObject(unistring=[None]*length)
+ def unicode_setitem(self, unistring, i, fieldnum):
+ value, tag = untag(fieldnum)
+ assert tag == TAGINT
+ assert 0 <= i < len(unistring.unistring)
+ unistring.unistring[i] = value
+ def concat_unicodes(self, left, right):
+ return FakeBuiltObject(uniconcat=[left, right])
+ def slice_unicode(self, str, start, length):
+ return FakeBuiltObject(unislice=[str, start, length])
class FakeBuiltObject(object):
def __init__(self, **kwds):
@@ -304,6 +315,30 @@
assert reader.force_all_virtuals() == [
FakeBuiltObject(strslice=info.fieldnums)]
+def test_vuniplaininfo():
+ info = VUniPlainInfo()
+ info.fieldnums = [tag(60, TAGINT)]
+ reader = FakeResumeDataReader()
+ reader._prepare_virtuals([info])
+ assert reader.force_all_virtuals() == [
+ FakeBuiltObject(unistring=[60])]
+
+def test_vuniconcatinfo():
+ info = VUniConcatInfo()
+ info.fieldnums = [tag(10, TAGBOX), tag(20, TAGBOX)]
+ reader = FakeResumeDataReader()
+ reader._prepare_virtuals([info])
+ assert reader.force_all_virtuals() == [
+ FakeBuiltObject(uniconcat=info.fieldnums)]
+
+def test_vunisliceinfo():
+ info = VUniSliceInfo()
+ info.fieldnums = [tag(10, TAGBOX), tag(20, TAGBOX), tag(30, TAGBOX)]
+ reader = FakeResumeDataReader()
+ reader._prepare_virtuals([info])
+ assert reader.force_all_virtuals() == [
+ FakeBuiltObject(unislice=info.fieldnums)]
+
# ____________________________________________________________
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_string.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_string.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_string.py Sat Oct 30 16:41:17 2010
@@ -6,14 +6,17 @@
class StringTests:
+ _str, _chr = str, chr
+
def test_eq_residual(self):
+ _str = self._str
jitdriver = JitDriver(greens = [], reds = ['n', 'i', 's'])
- global_s = "hello"
+ global_s = _str("hello")
def f(n, b, s):
if b:
- s += "ello"
+ s += _str("ello")
else:
- s += "allo"
+ s += _str("allo")
i = 0
while n > 0:
jitdriver.can_enter_jit(s=s, n=n, i=i)
@@ -21,18 +24,19 @@
n -= 1 + (s == global_s)
i += 1
return i
- res = self.meta_interp(f, [10, True, 'h'], listops=True)
+ res = self.meta_interp(f, [10, True, _str('h')], listops=True)
assert res == 5
self.check_loops(**{self.CALL: 1, self.CALL_PURE: 0})
def test_eq_folded(self):
+ _str = self._str
jitdriver = JitDriver(greens = ['s'], reds = ['n', 'i'])
- global_s = "hello"
+ global_s = _str("hello")
def f(n, b, s):
if b:
- s += "ello"
+ s += _str("ello")
else:
- s += "allo"
+ s += _str("allo")
i = 0
while n > 0:
jitdriver.can_enter_jit(s=s, n=n, i=i)
@@ -40,31 +44,18 @@
n -= 1 + (s == global_s)
i += 1
return i
- res = self.meta_interp(f, [10, True, 'h'], listops=True)
+ res = self.meta_interp(f, [10, True, _str('h')], listops=True)
assert res == 5
self.check_loops(**{self.CALL: 0, self.CALL_PURE: 0})
def test_newstr(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['n', 'm'])
def f(n, m):
while True:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- bytecode = 'adlfkj' + chr(n)
- res = bytecode[n]
- m -= 1
- if m < 0:
- return ord(res)
- res = self.meta_interp(f, [6, 10])
- assert res == 6
-
- def test_newunicode(self):
- jitdriver = JitDriver(greens = [], reds = ['n', 'm'])
- def f(n, m):
- while True:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- bytecode = u'adlfkj' + unichr(n)
+ bytecode = _str('adlfkj') + _chr(n)
res = bytecode[n]
m -= 1
if m < 0:
@@ -73,95 +64,96 @@
assert res == 6
def test_char2string_pure(self):
- for dochr in [chr, ]: #unichr]:
- jitdriver = JitDriver(greens = [], reds = ['n'])
- @dont_look_inside
- def escape(x):
- pass
- def f(n):
- while n > 0:
- jitdriver.can_enter_jit(n=n)
- jitdriver.jit_merge_point(n=n)
- s = dochr(n)
- if not we_are_jitted():
- s += s # forces to be a string
- if n > 100:
- escape(s)
- n -= 1
- return 42
- self.meta_interp(f, [6])
- self.check_loops(newstr=0, strsetitem=0, strlen=0,
- newunicode=0, unicodesetitem=0, unicodelen=0)
+ _str, _chr = self._str, self._chr
+ jitdriver = JitDriver(greens = [], reds = ['n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ def f(n):
+ while n > 0:
+ jitdriver.can_enter_jit(n=n)
+ jitdriver.jit_merge_point(n=n)
+ s = _chr(n)
+ if not we_are_jitted():
+ s += s # forces to be a string
+ if n > 100:
+ escape(s)
+ n -= 1
+ return 42
+ self.meta_interp(f, [6])
+ self.check_loops(newstr=0, strsetitem=0, strlen=0,
+ newunicode=0, unicodesetitem=0, unicodelen=0)
def test_char2string_escape(self):
- for dochr in [chr, ]: #unichr]:
- jitdriver = JitDriver(greens = [], reds = ['n', 'total'])
- @dont_look_inside
- def escape(x):
- return ord(x[0])
- def f(n):
- total = 0
- while n > 0:
- jitdriver.can_enter_jit(n=n, total=total)
- jitdriver.jit_merge_point(n=n, total=total)
- s = dochr(n)
- if not we_are_jitted():
- s += s # forces to be a string
- total += escape(s)
- n -= 1
- return total
- res = self.meta_interp(f, [6])
- assert res == 21
+ _str, _chr = self._str, self._chr
+ jitdriver = JitDriver(greens = [], reds = ['n', 'total'])
+ @dont_look_inside
+ def escape(x):
+ return ord(x[0])
+ def f(n):
+ total = 0
+ while n > 0:
+ jitdriver.can_enter_jit(n=n, total=total)
+ jitdriver.jit_merge_point(n=n, total=total)
+ s = _chr(n)
+ if not we_are_jitted():
+ s += s # forces to be a string
+ total += escape(s)
+ n -= 1
+ return total
+ res = self.meta_interp(f, [6])
+ assert res == 21
def test_char2string2char(self):
- for dochr in [chr, ]: #unichr]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'total'])
- def f(m):
- total = 0
- while m > 0:
- jitdriver.can_enter_jit(m=m, total=total)
- jitdriver.jit_merge_point(m=m, total=total)
- string = dochr(m)
- if m > 100:
- string += string # forces to be a string
- # read back the character
- c = string[0]
- total += ord(c)
- m -= 1
- return total
- res = self.meta_interp(f, [6])
- assert res == 21
- self.check_loops(newstr=0, strgetitem=0, strsetitem=0, strlen=0,
- newunicode=0, unicodegetitem=0, unicodesetitem=0,
- unicodelen=0)
+ _str, _chr = self._str, self._chr
+ jitdriver = JitDriver(greens = [], reds = ['m', 'total'])
+ def f(m):
+ total = 0
+ while m > 0:
+ jitdriver.can_enter_jit(m=m, total=total)
+ jitdriver.jit_merge_point(m=m, total=total)
+ string = _chr(m)
+ if m > 100:
+ string += string # forces to be a string
+ # read back the character
+ c = string[0]
+ total += ord(c)
+ m -= 1
+ return total
+ res = self.meta_interp(f, [6])
+ assert res == 21
+ self.check_loops(newstr=0, strgetitem=0, strsetitem=0, strlen=0,
+ newunicode=0, unicodegetitem=0, unicodesetitem=0,
+ unicodelen=0)
def test_strconcat_pure(self):
- for somestr in ["abc", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- mylist = [somestr+str(i) for i in range(10)]
- def f(n, m):
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + mylist[m]
- if m > 100:
- escape(s)
- m -= 1
- return 42
- self.meta_interp(f, [6, 7])
- self.check_loops(newstr=0, strsetitem=0,
- newunicode=0, unicodesetitem=0,
- call=0, call_pure=0)
+ _str = self._str
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ mylist = [_str("abc") + _str(i) for i in range(10)]
+ def f(n, m):
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = mylist[n] + mylist[m]
+ if m > 100:
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 7])
+ self.check_loops(newstr=0, strsetitem=0,
+ newunicode=0, unicodesetitem=0,
+ call=0, call_pure=0)
def test_strconcat_escape_str_str(self):
+ _str = self._str
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
@@ -171,46 +163,64 @@
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=0, copystrcontent=2,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=0, copystrcontent=2,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=0,
+ copyunicodecontent=2,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_str_char(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + chr(m)
+ s = mylist[n] + _chr(m)
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=1,
+ copyunicodecontent=1,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_char_str(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = chr(n) + mylist[m]
+ s = _chr(n) + mylist[m]
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=1, copystrcontent=1,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=1,
+ copyunicodecontent=1,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_char_char(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
@@ -219,91 +229,132 @@
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = chr(n) + chr(m)
+ s = _chr(n) + _chr(m)
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=2, copystrcontent=0,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=2, copystrcontent=0,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=2,
+ copyunicodecontent=0,
+ call=1, call_pure=0) # escape
def test_strconcat_escape_str_char_str(self):
+ _str, _chr = self._str, self._chr
jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
@dont_look_inside
def escape(x):
pass
- mylist = ["somestr"+str(i) for i in range(10)]
+ mylist = [_str("somestr") + _str(i) for i in range(10)]
def f(n, m):
while m >= 0:
jitdriver.can_enter_jit(m=m, n=n)
jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + chr(n) + mylist[m]
+ s = mylist[n] + _chr(n) + mylist[m]
escape(s)
m -= 1
return 42
self.meta_interp(f, [6, 7])
- self.check_loops(newstr=1, strsetitem=1, copystrcontent=2,
- call=1, call_pure=0) # escape
+ if _str is str:
+ self.check_loops(newstr=1, strsetitem=1, copystrcontent=2,
+ call=1, call_pure=0) # escape
+ else:
+ self.check_loops(newunicode=1, unicodesetitem=1,
+ copyunicodecontent=2,
+ call=1, call_pure=0) # escape
def test_strconcat_guard_fail(self):
- for somestr in ["abc", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- mylist = [somestr+str(i) for i in range(12)]
- def f(n, m):
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = mylist[n] + mylist[m]
- if m & 1:
- escape(s)
- m -= 1
- return 42
- self.meta_interp(f, [6, 10])
+ _str = self._str
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ mylist = [_str("abc") + _str(i) for i in range(12)]
+ def f(n, m):
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = mylist[n] + mylist[m]
+ if m & 1:
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 10])
def test_strslice(self):
- for somestr in ["abc", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- def f(n, m):
- assert n >= 0
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = "foobarbazetc"[m:n]
- if m <= 5:
- escape(s)
- m -= 1
- return 42
- self.meta_interp(f, [10, 10])
+ _str = self._str
+ longstring = _str("foobarbazetc")
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ def f(n, m):
+ assert n >= 0
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = longstring[m:n]
+ if m <= 5:
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [10, 10])
def test_streq_char(self):
- for somestr in ["?abcdefg", ]: #u"def"]:
- jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
- @dont_look_inside
- def escape(x):
- pass
- def f(n, m):
- assert n >= 0
- while m >= 0:
- jitdriver.can_enter_jit(m=m, n=n)
- jitdriver.jit_merge_point(m=m, n=n)
- s = somestr[:m]
- escape(s == "?")
- m -= 1
- return 42
- self.meta_interp(f, [6, 7])
- self.check_loops(newstr=0, newunicode=0)
-
-
-class TestOOtype(StringTests, OOJitMixin):
- CALL = "oosend"
- CALL_PURE = "oosend_pure"
+ _str = self._str
+ longstring = _str("?abcdefg")
+ somechar = _str("?")
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ @dont_look_inside
+ def escape(x):
+ pass
+ def f(n, m):
+ assert n >= 0
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ s = longstring[:m]
+ escape(s == somechar)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 7])
+ self.check_loops(newstr=0, newunicode=0)
+
+
+#class TestOOtype(StringTests, OOJitMixin):
+# CALL = "oosend"
+# CALL_PURE = "oosend_pure"
class TestLLtype(StringTests, LLJitMixin):
CALL = "call"
CALL_PURE = "call_pure"
+
+class TestLLtypeUnicode(TestLLtype):
+ _str, _chr = unicode, unichr
+
+ def test_str2unicode(self):
+ _str = self._str
+ jitdriver = JitDriver(greens = [], reds = ['m', 'n'])
+ class Foo:
+ pass
+ @dont_look_inside
+ def escape(x):
+ assert x == _str("6y")
+ def f(n, m):
+ while m >= 0:
+ jitdriver.can_enter_jit(m=m, n=n)
+ jitdriver.jit_merge_point(m=m, n=n)
+ foo = Foo()
+ foo.y = chr(m)
+ foo.y = "y"
+ s = _str(str(n)) + _str(foo.y)
+ escape(s)
+ m -= 1
+ return 42
+ self.meta_interp(f, [6, 7])
+ self.check_loops(call=3, # str(), _str(), escape()
+ newunicode=1, unicodegetitem=0,
+ unicodesetitem=1, copyunicodecontent=1)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualref.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualref.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualref.py Sat Oct 30 16:41:17 2010
@@ -93,7 +93,7 @@
lst = []
vrefinfo.continue_tracing = lambda vref, virtual: \
lst.append((vref, virtual))
- resumereader.consume_vref_and_vable(vrefinfo, None)
+ resumereader.consume_vref_and_vable(vrefinfo, None, None)
del vrefinfo.continue_tracing
assert len(lst) == 1
lltype.cast_opaque_ptr(lltype.Ptr(JIT_VIRTUAL_REF),
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_warmspot.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_warmspot.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_warmspot.py Sat Oct 30 16:41:17 2010
@@ -296,6 +296,69 @@
assert res == 1
self.check_loops(int_add=1) # I get 13 without the loop_header()
+ def test_omit_can_enter_jit(self):
+ # Simple test comparing the effects of always giving a can_enter_jit(),
+ # or not giving any. Mostly equivalent, except that if given, it is
+ # ignored the first time, and so it ends up taking one extra loop to
+ # start JITting.
+ mydriver = JitDriver(greens=[], reds=['m'])
+ #
+ for i2 in range(10):
+ def f2(m):
+ while m > 0:
+ mydriver.jit_merge_point(m=m)
+ m -= 1
+ self.meta_interp(f2, [i2])
+ try:
+ self.check_tree_loop_count(1)
+ break
+ except AssertionError:
+ print "f2: no loop generated for i2==%d" % i2
+ else:
+ raise # re-raise the AssertionError: check_loop_count never 1
+ #
+ for i1 in range(10):
+ def f1(m):
+ while m > 0:
+ mydriver.can_enter_jit(m=m)
+ mydriver.jit_merge_point(m=m)
+ m -= 1
+ self.meta_interp(f1, [i1])
+ try:
+ self.check_tree_loop_count(1)
+ break
+ except AssertionError:
+ print "f1: no loop generated for i1==%d" % i1
+ else:
+ raise # re-raise the AssertionError: check_loop_count never 1
+ #
+ assert i1 - 1 == i2
+
+ def test_no_loop_at_all(self):
+ mydriver = JitDriver(greens=[], reds=['m'])
+ def f2(m):
+ mydriver.jit_merge_point(m=m)
+ return m - 1
+ def f1(m):
+ while m > 0:
+ m = f2(m)
+ self.meta_interp(f1, [8])
+ # it should generate one "loop" only, which ends in a FINISH
+ # corresponding to the return from f2.
+ self.check_tree_loop_count(1)
+ self.check_loop_count(0)
+
+ def test_simple_loop(self):
+ mydriver = JitDriver(greens=[], reds=['m'])
+ def f1(m):
+ while m > 0:
+ mydriver.jit_merge_point(m=m)
+ m = m - 1
+ self.meta_interp(f1, [8])
+ self.check_loop_count(1)
+ self.check_loops({'int_sub': 1, 'int_gt': 1, 'guard_true': 1,
+ 'jump': 1})
+
class TestLLWarmspot(WarmspotTests, LLJitMixin):
CPUClass = runner.LLtypeCPU
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/virtualref.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/virtualref.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/virtualref.py Sat Oct 30 16:41:17 2010
@@ -16,7 +16,8 @@
('virtualref_index', lltype.Signed),
('forced', rclass.OBJECTPTR))
self.jit_virtual_ref_vtable = lltype.malloc(rclass.OBJECT_VTABLE,
- zero=True, flavor='raw')
+ zero=True, flavor='raw',
+ immortal=True)
self.jit_virtual_ref_vtable.name = rclass.alloc_array_name(
'jit_virtual_ref')
# build some constants
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmspot.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmspot.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmspot.py Sat Oct 30 16:41:17 2010
@@ -67,9 +67,16 @@
def jittify_and_run(interp, graph, args, repeat=1,
backendopt=False, trace_limit=sys.maxint,
debug_level=DEBUG_STEPS, inline=False, **kwds):
+ from pypy.config.config import ConfigError
translator = interp.typer.annotator.translator
- translator.config.translation.gc = "boehm"
- translator.config.translation.list_comprehension_operations = True
+ try:
+ translator.config.translation.gc = "boehm"
+ except ConfigError:
+ pass
+ try:
+ translator.config.translation.list_comprehension_operations = True
+ except ConfigError:
+ pass
warmrunnerdesc = WarmRunnerDesc(translator, backendopt=backendopt, **kwds)
for jd in warmrunnerdesc.jitdrivers_sd:
jd.warmstate.set_param_threshold(3) # for tests
@@ -103,15 +110,16 @@
for i in range(len(block.operations)):
op = block.operations[i]
if (op.opname == 'jit_marker' and
- op.args[0].value == marker_name):
+ op.args[0].value == marker_name and
+ op.args[1].value.active): # the jitdriver
results.append((graph, block, i))
return results
def find_can_enter_jit(graphs):
- results = _find_jit_marker(graphs, 'can_enter_jit')
- if not results:
- raise Exception("no can_enter_jit found!")
- return results
+ return _find_jit_marker(graphs, 'can_enter_jit')
+
+def find_loop_headers(graphs):
+ return _find_jit_marker(graphs, 'loop_header')
def find_jit_merge_points(graphs):
results = _find_jit_marker(graphs, 'jit_merge_point')
@@ -204,9 +212,9 @@
"there are multiple jit_merge_points with the same jitdriver"
def split_graph_and_record_jitdriver(self, graph, block, pos):
- jd = JitDriverStaticData()
- jd._jit_merge_point_pos = (graph, block, pos)
op = block.operations[pos]
+ jd = JitDriverStaticData()
+ jd._jit_merge_point_pos = (graph, op)
args = op.args[2:]
s_binding = self.translator.annotator.binding
jd._portal_args_s = [s_binding(v) for v in args]
@@ -279,10 +287,20 @@
def make_virtualizable_infos(self):
vinfos = {}
for jd in self.jitdrivers_sd:
+ #
+ jd.greenfield_info = None
+ for name in jd.jitdriver.greens:
+ if '.' in name:
+ from pypy.jit.metainterp.greenfield import GreenFieldInfo
+ jd.greenfield_info = GreenFieldInfo(self.cpu, jd)
+ break
+ #
if not jd.jitdriver.virtualizables:
jd.virtualizable_info = None
jd.index_of_virtualizable = -1
continue
+ else:
+ assert jd.greenfield_info is None, "XXX not supported yet"
#
jitdriver = jd.jitdriver
assert len(jitdriver.virtualizables) == 1 # for now
@@ -450,8 +468,7 @@
self.make_args_specification(jd)
def make_args_specification(self, jd):
- graph, block, index = jd._jit_merge_point_pos
- op = block.operations[index]
+ graph, op = jd._jit_merge_point_pos
greens_v, reds_v = support.decode_hp_hint_args(op)
ALLARGS = [v.concretetype for v in (greens_v + reds_v)]
jd._green_args_spec = [v.concretetype for v in greens_v]
@@ -467,26 +484,37 @@
[lltype.Signed, llmemory.GCREF], RESTYPE)
def rewrite_can_enter_jits(self):
- can_enter_jits = find_can_enter_jit(self.translator.graphs)
sublists = {}
for jd in self.jitdrivers_sd:
- sublists[jd.jitdriver] = []
+ sublists[jd.jitdriver] = jd, []
+ jd.no_loop_header = True
+ #
+ loop_headers = find_loop_headers(self.translator.graphs)
+ for graph, block, index in loop_headers:
+ op = block.operations[index]
+ jitdriver = op.args[1].value
+ assert jitdriver in sublists, \
+ "loop_header with no matching jit_merge_point"
+ jd, sublist = sublists[jitdriver]
+ jd.no_loop_header = False
+ #
+ can_enter_jits = find_can_enter_jit(self.translator.graphs)
for graph, block, index in can_enter_jits:
op = block.operations[index]
jitdriver = op.args[1].value
assert jitdriver in sublists, \
"can_enter_jit with no matching jit_merge_point"
+ jd, sublist = sublists[jitdriver]
origportalgraph = jd._jit_merge_point_pos[0]
if graph is not origportalgraph:
- sublists[jitdriver].append((graph, block, index))
+ sublist.append((graph, block, index))
+ jd.no_loop_header = False
else:
pass # a 'can_enter_jit' before the 'jit-merge_point', but
# originally in the same function: we ignore it here
# see e.g. test_jitdriver.test_simple
for jd in self.jitdrivers_sd:
- sublist = sublists[jd.jitdriver]
- assert len(sublist) > 0, \
- "found no can_enter_jit for %r" % (jd.jitdriver,)
+ _, sublist = sublists[jd.jitdriver]
self.rewrite_can_enter_jit(jd, sublist)
def rewrite_can_enter_jit(self, jd, can_enter_jits):
@@ -494,6 +522,19 @@
FUNCPTR = jd._PTR_JIT_ENTER_FUNCTYPE
jit_enter_fnptr = self.helper_func(FUNCPTR, jd._maybe_enter_jit_fn)
+ if len(can_enter_jits) == 0:
+ # see test_warmspot.test_no_loop_at_all
+ operations = jd.portal_graph.startblock.operations
+ op1 = operations[0]
+ assert (op1.opname == 'jit_marker' and
+ op1.args[0].value == 'jit_merge_point')
+ op0 = SpaceOperation(
+ 'jit_marker',
+ [Constant('can_enter_jit', lltype.Void)] + op1.args[1:],
+ None)
+ operations.insert(0, op0)
+ can_enter_jits = [(jd.portal_graph, jd.portal_graph.startblock, 0)]
+
for graph, block, index in can_enter_jits:
if graph is jd._jit_merge_point_pos[0]:
continue
@@ -702,8 +743,14 @@
# ____________________________________________________________
# Now mutate origportalgraph to end with a call to portal_runner_ptr
#
- _, origblock, origindex = jd._jit_merge_point_pos
- op = origblock.operations[origindex]
+ _, op = jd._jit_merge_point_pos
+ for origblock in origportalgraph.iterblocks():
+ if op in origblock.operations:
+ break
+ else:
+ assert False, "lost the operation %r in the graph %r" % (
+ op, origportalgraph)
+ origindex = origblock.operations.index(op)
assert op.opname == 'jit_marker'
assert op.args[0].value == 'jit_merge_point'
greens_v, reds_v = support.decode_hp_hint_args(op)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmstate.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmstate.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmstate.py Sat Oct 30 16:41:17 2010
@@ -1,5 +1,5 @@
import sys
-from pypy.rpython.lltypesystem import lltype, llmemory, rstr
+from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi
from pypy.rpython.ootypesystem import ootype
from pypy.rpython.annlowlevel import hlstr, llstr, cast_base_ptr_to_instance
from pypy.rpython.annlowlevel import cast_object_to_ptr
@@ -24,7 +24,11 @@
"""
INPUT = lltype.typeOf(x)
if INPUT is lltype.Signed:
- return lltype.cast_primitive(TYPE, x) # XXX missing: Ptr(non-gc)
+ if isinstance(TYPE, lltype.Ptr) and TYPE.TO._gckind == 'raw':
+ # non-gc pointer
+ return rffi.cast(TYPE, x)
+ else:
+ return lltype.cast_primitive(TYPE, x)
elif INPUT is lltype.Float:
assert TYPE is lltype.Float
return x
@@ -172,6 +176,9 @@
meth(default_value)
def set_param_threshold(self, threshold):
+ if threshold < 0:
+ self.increment_threshold = 0 # never reach the THRESHOLD_LIMIT
+ return
if threshold < 2:
threshold = 2
self.increment_threshold = (self.THRESHOLD_LIMIT // threshold) + 1
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit.py Sat Oct 30 16:41:17 2010
@@ -40,8 +40,10 @@
config.objspace.usemodules.array = True
config.objspace.usemodules._weakref = False
config.objspace.usemodules._sre = False
+#
+config.objspace.usemodules._ffi = True
+#
set_pypy_opt_level(config, level='jit')
-config.objspace.std.withinlineddict = True
if BACKEND == 'c':
config.objspace.std.multimethods = 'mrd'
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py Sat Oct 30 16:41:17 2010
@@ -16,7 +16,7 @@
interp.heap.malloc_nonmovable = returns_null # XXX
from pypy.jit.backend.llgraph.runner import LLtypeCPU
- LLtypeCPU.supports_floats = False # for now
+ #LLtypeCPU.supports_floats = False # for now
apply_jit(interp, graph, LLtypeCPU)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/spli/interpreter.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tl/spli/interpreter.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tl/spli/interpreter.py Sat Oct 30 16:41:17 2010
@@ -1,12 +1,14 @@
import os
-from pypy.tool import stdlib_opcode as opcode
+from pypy.tool import stdlib_opcode
from pypy.jit.tl.spli import objects, pycode
-from pypy.tool.stdlib_opcode import unrolling_opcode_descs
-from pypy.tool.stdlib_opcode import opcode_method_names
from pypy.rlib.unroll import unrolling_iterable
from pypy.rlib.jit import JitDriver, hint, dont_look_inside
from pypy.rlib.objectmodel import we_are_translated
+opcode_method_names = stdlib_opcode.host_bytecode_spec.method_names
+unrolling_opcode_descs = unrolling_iterable(
+ stdlib_opcode.host_bytecode_spec.ordered_opdescs)
+HAVE_ARGUMENT = stdlib_opcode.host_HAVE_ARGUMENT
compare_ops = [
"cmp_lt", # "<"
@@ -79,7 +81,7 @@
self.stack_depth = hint(self.stack_depth, promote=True)
op = ord(code[instr_index])
instr_index += 1
- if op >= opcode.HAVE_ARGUMENT:
+ if op >= HAVE_ARGUMENT:
low = ord(code[instr_index])
hi = ord(code[instr_index + 1])
oparg = (hi << 8) | low
@@ -183,6 +185,12 @@
next_instr += arg
return next_instr
+ def POP_JUMP_IF_FALSE(self, arg, next_instr, code):
+ w_cond = self.pop()
+ if not w_cond.is_true():
+ next_instr = arg
+ return next_instr
+
def JUMP_FORWARD(self, arg, next_instr, code):
return next_instr + arg
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/tla/test_tla.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tl/tla/test_tla.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tl/tla/test_tla.py Sat Oct 30 16:41:17 2010
@@ -156,7 +156,7 @@
# ____________________________________________________________
from pypy.jit.metainterp.test.test_basic import LLJitMixin
-from pypy.rlib.jit import OPTIMIZER_FULL
+from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_NO_PERFECTSPEC
class TestLLtype(LLJitMixin):
def test_loop(self):
@@ -178,5 +178,5 @@
assert isinstance(w_result, tla.W_IntObject)
return w_result.intvalue
res = self.meta_interp(interp_w, [42], listops=True,
- optimizer=OPTIMIZER_FULL)
+ optimizer=OPTIMIZER_NO_PERFECTSPEC)
assert res == 0
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tool/loopviewer.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tool/loopviewer.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tool/loopviewer.py Sat Oct 30 16:41:17 2010
@@ -9,7 +9,7 @@
import optparse
from pprint import pprint
from pypy.tool import logparser
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.jit.metainterp.history import ConstInt
from pypy.rpython.lltypesystem import llmemory, lltype
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tool/showstats.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tool/showstats.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tool/showstats.py Sat Oct 30 16:41:17 2010
@@ -4,15 +4,15 @@
import autopath
import sys, py
from pypy.tool import logparser
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
from pypy.jit.metainterp.resoperation import rop
from pypy.rpython.lltypesystem import lltype, llmemory
def main(argv):
log = logparser.parse_log_file(argv[0])
- parts = logparser.extract_category(log, "jit-log-opt-")
+ parts = logparser.extract_category(log, "jit-log-noopt-")
for i, oplist in enumerate(parts):
- loop = parse(oplist, no_namespace=True)
+ loop = parse(oplist, no_namespace=True, nonstrict=True)
num_ops = 0
num_dmp = 0
num_guards = 0
@@ -23,7 +23,10 @@
num_ops += 1
if op.is_guard():
num_guards += 1
- print "Loop #%d, length: %d, opcodes: %d, guards: %d, %f" % (i, num_ops, num_dmp, num_guards, num_ops/num_dmp)
+ if num_dmp == 0:
+ print "Loop #%d, length: %d, opcodes: %d, guards: %d" % (i, num_ops, num_dmp, num_guards)
+ else:
+ print "Loop #%d, length: %d, opcodes: %d, guards: %d, %f" % (i, num_ops, num_dmp, num_guards, num_ops/num_dmp)
if __name__ == '__main__':
main(sys.argv[1:])
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tool/traceviewer.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tool/traceviewer.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tool/traceviewer.py Sat Oct 30 16:41:17 2010
@@ -253,10 +253,10 @@
def main(loopfile, use_threshold, view=True):
countname = py.path.local(loopfile + '.count')
if countname.check():
- counts = [re.split('( 20 and use_threshold:
counts.threshold = l[-20]
Modified: pypy/branch/jit-unroll-loops/pypy/module/__builtin__/interp_classobj.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/__builtin__/interp_classobj.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/__builtin__/interp_classobj.py Sat Oct 30 16:41:17 2010
@@ -2,9 +2,11 @@
from pypy.interpreter.error import OperationError, operationerrfmt
from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped, applevel
from pypy.interpreter.gateway import interp2app, ObjSpace
-from pypy.interpreter.typedef import TypeDef, make_weakref_descr
+from pypy.interpreter.typedef import TypeDef
from pypy.interpreter.argument import Arguments
from pypy.interpreter.baseobjspace import Wrappable
+from pypy.interpreter.typedef import GetSetProperty, descr_get_dict
+from pypy.interpreter.typedef import descr_set_dict
from pypy.rlib.rarithmetic import r_uint, intmask
from pypy.rlib.objectmodel import compute_identity_hash
from pypy.rlib.debug import make_sure_not_resized
@@ -57,6 +59,14 @@
self.bases_w = bases
self.w_dict = w_dict
+ def instantiate(self, space):
+ cache = space.fromcache(Cache)
+ if self.lookup(space, '__del__') is not None:
+ w_inst = cache.cls_with_del(space, self)
+ else:
+ w_inst = cache.cls_without_del(space, self)
+ return w_inst
+
def getdict(self):
return self.w_dict
@@ -100,15 +110,15 @@
return False
@jit.unroll_safe
- def lookup(self, space, w_attr):
+ def lookup(self, space, attr):
# returns w_value or interplevel None
- w_result = space.finditem(self.w_dict, w_attr)
+ w_result = space.finditem_str(self.w_dict, attr)
if w_result is not None:
return w_result
for base in self.bases_w:
# XXX fix annotation of bases_w to be a list of W_ClassObjects
assert isinstance(base, W_ClassObject)
- w_result = base.lookup(space, w_attr)
+ w_result = base.lookup(space, attr)
if w_result is not None:
return w_result
return None
@@ -122,7 +132,7 @@
return space.wrap(self.name)
elif name == "__bases__":
return space.newtuple(self.bases_w)
- w_value = self.lookup(space, w_attr)
+ w_value = self.lookup(space, name)
if w_value is None:
raise operationerrfmt(
space.w_AttributeError,
@@ -147,7 +157,7 @@
self.setbases(space, w_value)
return
elif name == "__del__":
- if self.lookup(space, w_attr) is None:
+ if self.lookup(space, name) is None:
msg = ("a __del__ method added to an existing class "
"will not be called")
space.warn(msg, space.w_RuntimeWarning)
@@ -195,13 +205,20 @@
# NOT_RPYTHON
return '' % self.name
+class Cache:
+ def __init__(self, space):
+ from pypy.interpreter.typedef import _usersubclswithfeature
+ # evil
+ self.cls_without_del = _usersubclswithfeature(
+ space.config, W_InstanceObject, "dict", "weakref")
+ self.cls_with_del = _usersubclswithfeature(
+ space.config, self.cls_without_del, "del")
+
+
def class_descr_call(space, w_self, __args__):
self = space.interp_w(W_ClassObject, w_self)
- if self.lookup(space, space.wrap('__del__')) is not None:
- w_inst = W_InstanceObjectWithDel(space, self)
- else:
- w_inst = W_InstanceObject(space, self)
- w_init = w_inst.getattr_from_class(space, space.wrap('__init__'))
+ w_inst = self.instantiate(space)
+ w_init = w_inst.getattr_from_class(space, '__init__')
if w_init is not None:
w_result = space.call_args(w_init, __args__)
if not space.is_w(w_result, space.w_None):
@@ -234,7 +251,7 @@
def make_unary_instance_method(name):
def unaryop(self, space):
- w_meth = self.getattr(space, space.wrap(name), True)
+ w_meth = self.getattr(space, name, True)
return space.call_function(w_meth)
unaryop.func_name = name
return unaryop
@@ -242,7 +259,7 @@
def make_binary_returning_notimplemented_instance_method(name):
def binaryop(self, space, w_other):
try:
- w_meth = self.getattr(space, space.wrap(name), False)
+ w_meth = self.getattr(space, name, False)
except OperationError, e:
if e.match(space, space.w_AttributeError):
return space.w_NotImplemented
@@ -267,7 +284,7 @@
w_a = self
w_b = w_other
if w_a is self:
- w_meth = self.getattr(space, space.wrap(specialname), False)
+ w_meth = self.getattr(space, specialname, False)
if w_meth is None:
return space.w_NotImplemented
return space.call_function(w_meth, w_b)
@@ -278,7 +295,7 @@
def rbinaryop(self, space, w_other):
w_a, w_b = _coerce_helper(space, self, w_other)
if w_a is None or w_a is self:
- w_meth = self.getattr(space, space.wrap(rspecialname), False)
+ w_meth = self.getattr(space, rspecialname, False)
if w_meth is None:
return space.w_NotImplemented
return space.call_function(w_meth, w_other)
@@ -302,46 +319,34 @@
raise OperationError(
space.w_TypeError,
space.wrap("instance() first arg must be class"))
- if space.is_w(w_dict, space.w_None):
- w_dict = None
- elif not space.is_true(space.isinstance(w_dict, space.w_dict)):
- raise OperationError(
- space.w_TypeError,
- space.wrap("instance() second arg must be dictionary or None"))
- return W_InstanceObject(space, w_class, w_dict)
+ w_result = w_class.instantiate(space)
+ if not space.is_w(w_dict, space.w_None):
+ w_result.setdict(space, w_dict)
+ return w_result
class W_InstanceObject(Wrappable):
- def __init__(self, space, w_class, w_dict=None):
- if w_dict is None:
- w_dict = space.newdict(instance=True)
+ def __init__(self, space, w_class):
+ # note that user_setup is overridden by the typedef.py machinery
+ self.user_setup(space, space.gettypeobject(self.typedef))
assert isinstance(w_class, W_ClassObject)
self.w_class = w_class
- self.w_dict = w_dict
- self.space = space
-
- def getdict(self):
- return self.w_dict
- def setdict(self, space, w_dict):
- if (w_dict is None or
- not space.is_true(space.isinstance(w_dict, space.w_dict))):
- raise OperationError(
- space.w_TypeError,
- space.wrap("__dict__ must be a dictionary object"))
- self.w_dict = w_dict
+ def user_setup(self, space, w_subtype):
+ self.space = space
- def setclass(self, space, w_class):
+ def set_oldstyle_class(self, space, w_class):
if w_class is None or not isinstance(w_class, W_ClassObject):
raise OperationError(
space.w_TypeError,
space.wrap("__class__ must be set to a class"))
self.w_class = w_class
- def getattr_from_class(self, space, w_name):
+ def getattr_from_class(self, space, name):
# Look up w_name in the class dict, and call its __get__.
# This method ignores the instance dict and the __getattr__.
# Returns None if not found.
- w_value = self.w_class.lookup(space, w_name)
+ assert isinstance(name, str)
+ w_value = self.w_class.lookup(space, name)
if w_value is None:
return None
w_descr_get = space.lookup(w_value, '__get__')
@@ -349,19 +354,20 @@
return w_value
return space.call_function(w_descr_get, w_value, self, self.w_class)
- def getattr(self, space, w_name, exc=True):
+ def getattr(self, space, name, exc=True):
# Normal getattr rules: look up w_name in the instance dict,
# in the class dict, and then via a call to __getatttr__.
- w_result = space.finditem(self.w_dict, w_name)
+ assert isinstance(name, str)
+ w_result = self.getdictvalue(space, name)
if w_result is not None:
return w_result
- w_result = self.getattr_from_class(space, w_name)
+ w_result = self.getattr_from_class(space, name)
if w_result is not None:
return w_result
- w_meth = self.getattr_from_class(space, space.wrap('__getattr__'))
+ w_meth = self.getattr_from_class(space, '__getattr__')
if w_meth is not None:
try:
- return space.call_function(w_meth, w_name)
+ return space.call_function(w_meth, space.wrap(name))
except OperationError, e:
if not exc and e.match(space, space.w_AttributeError):
return None # eat the AttributeError
@@ -371,7 +377,7 @@
raise operationerrfmt(
space.w_AttributeError,
"%s instance has no attribute '%s'",
- self.w_class.name, space.str_w(w_name))
+ self.w_class.name, name)
else:
return None
@@ -379,24 +385,25 @@
name = space.str_w(w_attr)
if len(name) >= 8 and name[0] == '_':
if name == "__dict__":
- return self.w_dict
+ return self.getdict()
elif name == "__class__":
return self.w_class
- return self.getattr(space, w_attr)
+ return self.getattr(space, name)
def descr_setattr(self, space, w_name, w_value):
name = unwrap_attr(space, w_name)
- w_meth = self.getattr_from_class(space, space.wrap('__setattr__'))
+ w_meth = self.getattr_from_class(space, '__setattr__')
if name and name[0] == "_":
if name == '__dict__':
self.setdict(space, w_value)
return
if name == '__class__':
- self.setclass(space, w_value)
+ self.set_oldstyle_class(space, w_value)
return
if name == '__del__' and w_meth is None:
- if (not isinstance(self, W_InstanceObjectWithDel)
- and space.finditem(self.w_dict, w_name) is None):
+ cache = space.fromcache(Cache)
+ if (not isinstance(self, cache.cls_with_del)
+ and self.getdictvalue(space, '__del__') is None):
msg = ("a __del__ method added to an instance "
"with no __del__ in the class will not be called")
space.warn(msg, space.w_RuntimeWarning)
@@ -410,13 +417,13 @@
if name and name[0] == "_":
if name == '__dict__':
# use setdict to raise the error
- self.setdict(space, None)
+ self.setdict(space, space.w_None)
return
elif name == '__class__':
- # use setclass to raise the error
- self.setclass(space, None)
+ # use set_oldstyle_class to raise the error
+ self.set_oldstyle_class(space, None)
return
- w_meth = self.getattr_from_class(space, space.wrap('__delattr__'))
+ w_meth = self.getattr_from_class(space, '__delattr__')
if w_meth is not None:
space.call_function(w_meth, w_name)
else:
@@ -427,7 +434,7 @@
self.w_class.name, name)
def descr_repr(self, space):
- w_meth = self.getattr(space, space.wrap('__repr__'), False)
+ w_meth = self.getattr(space, '__repr__', False)
if w_meth is None:
w_class = self.w_class
mod = w_class.get_module_string(space)
@@ -435,19 +442,19 @@
return space.call_function(w_meth)
def descr_str(self, space):
- w_meth = self.getattr(space, space.wrap('__str__'), False)
+ w_meth = self.getattr(space, '__str__', False)
if w_meth is None:
return self.descr_repr(space)
return space.call_function(w_meth)
def descr_unicode(self, space):
- w_meth = self.getattr(space, space.wrap('__unicode__'), False)
+ w_meth = self.getattr(space, '__unicode__', False)
if w_meth is None:
return self.descr_str(space)
return space.call_function(w_meth)
def descr_len(self, space):
- w_meth = self.getattr(space, space.wrap('__len__'))
+ w_meth = self.getattr(space, '__len__')
w_result = space.call_function(w_meth)
if space.is_true(space.isinstance(w_result, space.w_int)):
if space.is_true(space.lt(w_result, space.wrap(0))):
@@ -460,22 +467,22 @@
space.wrap("__len__() should return an int"))
def descr_getitem(self, space, w_key):
- w_meth = self.getattr(space, space.wrap('__getitem__'))
+ w_meth = self.getattr(space, '__getitem__')
return space.call_function(w_meth, w_key)
def descr_setitem(self, space, w_key, w_value):
- w_meth = self.getattr(space, space.wrap('__setitem__'))
+ w_meth = self.getattr(space, '__setitem__')
space.call_function(w_meth, w_key, w_value)
def descr_delitem(self, space, w_key):
- w_meth = self.getattr(space, space.wrap('__delitem__'))
+ w_meth = self.getattr(space, '__delitem__')
space.call_function(w_meth, w_key)
def descr_iter(self, space):
- w_meth = self.getattr(space, space.wrap('__iter__'), False)
+ w_meth = self.getattr(space, '__iter__', False)
if w_meth is not None:
return space.call_function(w_meth)
- w_meth = self.getattr(space, space.wrap('__getitem__'), False)
+ w_meth = self.getattr(space, '__getitem__', False)
if w_meth is None:
raise OperationError(
space.w_TypeError,
@@ -485,14 +492,14 @@
# don't see the point
def descr_getslice(self, space, w_i, w_j):
- w_meth = self.getattr(space, space.wrap('__getslice__'), False)
+ w_meth = self.getattr(space, '__getslice__', False)
if w_meth is not None:
return space.call_function(w_meth, w_i, w_j)
else:
return space.getitem(self, space.newslice(w_i, w_j, space.w_None))
def descr_setslice(self, space, w_i, w_j, w_sequence):
- w_meth = self.getattr(space, space.wrap('__setslice__'), False)
+ w_meth = self.getattr(space, '__setslice__', False)
if w_meth is not None:
space.call_function(w_meth, w_i, w_j, w_sequence)
else:
@@ -500,20 +507,20 @@
w_sequence)
def descr_delslice(self, space, w_i, w_j):
- w_meth = self.getattr(space, space.wrap('__delslice__'), False)
+ w_meth = self.getattr(space, '__delslice__', False)
if w_meth is not None:
space.call_function(w_meth, w_i, w_j)
else:
return space.delitem(self, space.newslice(w_i, w_j, space.w_None))
def descr_call(self, space, __args__):
- w_meth = self.getattr(space, space.wrap('__call__'))
+ w_meth = self.getattr(space, '__call__')
return space.call_args(w_meth, __args__)
def descr_nonzero(self, space):
- w_func = self.getattr(space, space.wrap('__nonzero__'), False)
+ w_func = self.getattr(space, '__nonzero__', False)
if w_func is None:
- w_func = self.getattr(space, space.wrap('__len__'), False)
+ w_func = self.getattr(space, '__len__', False)
if w_func is None:
return space.w_True
w_result = space.call_function(w_func)
@@ -537,7 +544,7 @@
not isinstance(w_b, W_InstanceObject)):
return space.cmp(w_a, w_b)
if isinstance(w_a, W_InstanceObject):
- w_func = w_a.getattr(space, space.wrap('__cmp__'), False)
+ w_func = w_a.getattr(space, '__cmp__', False)
if w_func is not None:
w_res = space.call_function(w_func, w_b)
if space.is_w(w_res, space.w_NotImplemented):
@@ -556,7 +563,7 @@
return space.wrap(-1)
return space.wrap(0)
if isinstance(w_b, W_InstanceObject):
- w_func = w_b.getattr(space, space.wrap('__cmp__'), False)
+ w_func = w_b.getattr(space, '__cmp__', False)
if w_func is not None:
w_res = space.call_function(w_func, w_a)
if space.is_w(w_res, space.w_NotImplemented):
@@ -577,10 +584,10 @@
return space.w_NotImplemented
def descr_hash(self, space):
- w_func = self.getattr(space, space.wrap('__hash__'), False)
+ w_func = self.getattr(space, '__hash__', False)
if w_func is None:
- w_eq = self.getattr(space, space.wrap('__eq__'), False)
- w_cmp = self.getattr(space, space.wrap('__cmp__'), False)
+ w_eq = self.getattr(space, '__eq__', False)
+ w_cmp = self.getattr(space, '__cmp__', False)
if w_eq is not None or w_cmp is not None:
raise OperationError(space.w_TypeError,
space.wrap("unhashable instance"))
@@ -595,7 +602,7 @@
return w_ret
def descr_index(self, space):
- w_func = self.getattr(space, space.wrap('__index__'), False)
+ w_func = self.getattr(space, '__index__', False)
if w_func is not None:
return space.call_function(w_func)
raise OperationError(
@@ -603,7 +610,7 @@
space.wrap("object cannot be interpreted as an index"))
def descr_contains(self, space, w_obj):
- w_func = self.getattr(space, space.wrap('__contains__'), False)
+ w_func = self.getattr(space, '__contains__', False)
if w_func is not None:
return space.wrap(space.is_true(space.call_function(w_func, w_obj)))
# now do it ourselves
@@ -626,7 +633,7 @@
w_a = self
w_b = w_other
if w_a is self:
- w_func = self.getattr(space, space.wrap('__pow__'), False)
+ w_func = self.getattr(space, '__pow__', False)
if w_func is not None:
return space.call_function(w_func, w_other)
return space.w_NotImplemented
@@ -634,7 +641,7 @@
return space.pow(w_a, w_b, space.w_None)
else:
# CPython also doesn't try coercion in this case
- w_func = self.getattr(space, space.wrap('__pow__'), False)
+ w_func = self.getattr(space, '__pow__', False)
if w_func is not None:
return space.call_function(w_func, w_other, w_modulo)
return space.w_NotImplemented
@@ -646,7 +653,7 @@
w_a = self
w_b = w_other
if w_a is self:
- w_func = self.getattr(space, space.wrap('__rpow__'), False)
+ w_func = self.getattr(space, '__rpow__', False)
if w_func is not None:
return space.call_function(w_func, w_other)
return space.w_NotImplemented
@@ -654,13 +661,13 @@
return space.pow(w_b, w_a, space.w_None)
else:
# CPython also doesn't try coercion in this case
- w_func = self.getattr(space, space.wrap('__rpow__'), False)
+ w_func = self.getattr(space, '__rpow__', False)
if w_func is not None:
return space.call_function(w_func, w_other, w_modulo)
return space.w_NotImplemented
def descr_next(self, space):
- w_func = self.getattr(space, space.wrap('next'), False)
+ w_func = self.getattr(space, 'next', False)
if w_func is None:
raise OperationError(space.w_TypeError,
space.wrap("instance has no next() method"))
@@ -669,10 +676,9 @@
def descr_del(self, space):
# Note that this is called from executioncontext.UserDelAction
# via the space.userdel() method.
- w_name = space.wrap('__del__')
- w_func = space.finditem(self.w_dict, w_name)
+ w_func = self.getdictvalue(space, '__del__')
if w_func is None:
- w_func = self.getattr_from_class(space, w_name)
+ w_func = self.getattr_from_class(space, '__del__')
if w_func is not None:
space.call_function(w_func)
@@ -717,6 +723,14 @@
rmeth,
unwrap_spec=["self", ObjSpace, W_Root])
+
+def descr_del_dict(space, w_inst):
+ # use setdict to raise the error
+ w_inst.setdict(space, space.w_None)
+
+dict_descr = GetSetProperty(descr_get_dict, descr_set_dict, descr_del_dict)
+dict_descr.name = '__dict__'
+
W_InstanceObject.typedef = TypeDef("instance",
__new__ = interp2app(descr_instance_new),
__getattribute__ = interp2app(W_InstanceObject.descr_getattribute,
@@ -766,12 +780,9 @@
unwrap_spec=['self', ObjSpace, W_Root, W_Root]),
next = interp2app(W_InstanceObject.descr_next,
unwrap_spec=['self', ObjSpace]),
- __weakref__ = make_weakref_descr(W_InstanceObject),
__del__ = interp2app(W_InstanceObject.descr_del,
unwrap_spec=['self', ObjSpace]),
+ __dict__ = dict_descr,
**rawdict
)
-
-class W_InstanceObjectWithDel(W_InstanceObject):
- def __del__(self):
- self._enqueue_for_destruction(self.space)
+W_InstanceObject.typedef.acceptable_as_base_class = False
Modified: pypy/branch/jit-unroll-loops/pypy/module/__builtin__/test/test_classobj.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/__builtin__/test/test_classobj.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/__builtin__/test/test_classobj.py Sat Oct 30 16:41:17 2010
@@ -928,29 +928,33 @@
assert x is b
assert y == 5
-
-class AppTestOldStyleSharing(AppTestOldstyle):
- def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True})
- if option.runappdirect:
- py.test.skip("can only be run on py.py")
- def is_sharing(space, w_inst):
- from pypy.objspace.std.sharingdict import SharedDictImplementation
- w_d = w_inst.getdict()
- return space.wrap(isinstance(w_d, SharedDictImplementation) and w_d.r_dict_content is None)
- cls.w_is_sharing = cls.space.wrap(gateway.interp2app(is_sharing))
-
-
- def test_real_sharing(self):
+ def test_cant_subclass_instance(self):
class A:
- def __init__(self):
- self.x = 42
- A1, A2, A3 = A(), A(), A()
- assert self.is_sharing(A3)
- assert self.is_sharing(A2)
- assert self.is_sharing(A1)
+ pass
+ try:
+ class B(type(A())):
+ pass
+ except TypeError:
+ pass
+ else:
+ assert 0, "should have raised"
+
+ def test_dict_descriptor(self):
+ import sys
+ if not hasattr(sys, 'pypy_objspaceclass'):
+ skip("on CPython old-style instances don't have a __dict__ descriptor")
+ class A:
+ pass
+ a = A()
+ a.x = 1
+ descr = type(a).__dict__['__dict__']
+ assert descr.__get__(a) == {'x': 1}
+ descr.__set__(a, {'x': 2})
+ assert a.x == 2
+ raises(TypeError, descr.__delete__, a)
+
-class AppTestOldStyleModDict(object):
+class AppTestOldStyleClassStrDict(object):
def setup_class(cls):
if option.runappdirect:
py.test.skip("can only be run on py.py")
@@ -966,3 +970,22 @@
a = 1
b = 2
assert self.is_strdict(A)
+
+class AppTestOldStyleMapDict(AppTestOldstyle):
+ def setup_class(cls):
+ cls.space = gettestobjspace(**{"objspace.std.withmapdict": True})
+ if option.runappdirect:
+ py.test.skip("can only be run on py.py")
+ def has_mapdict(space, w_inst):
+ return space.wrap(w_inst._get_mapdict_map() is not None)
+ cls.w_has_mapdict = cls.space.wrap(gateway.interp2app(has_mapdict))
+
+
+ def test_has_mapdict(self):
+ class A:
+ def __init__(self):
+ self.x = 42
+ a = A()
+ assert a.x == 42
+ assert self.has_mapdict(a)
+
Modified: pypy/branch/jit-unroll-loops/pypy/module/_rawffi/__init__.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/_rawffi/__init__.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/_rawffi/__init__.py Sat Oct 30 16:41:17 2010
@@ -1,5 +1,5 @@
-""" Low-level interface to libffi
+""" Low-level interface to clibffi
"""
from pypy.interpreter.mixedmodule import MixedModule
@@ -39,11 +39,11 @@
if hasattr(interp_rawffi, 'check_HRESULT'):
Module.interpleveldefs['check_HRESULT'] = 'interp_rawffi.check_HRESULT'
- from pypy.rlib import libffi
+ from pypy.rlib import clibffi
for name in ['FUNCFLAG_STDCALL', 'FUNCFLAG_CDECL', 'FUNCFLAG_PYTHONAPI',
]:
- if hasattr(libffi, name):
- Module.interpleveldefs[name] = "space.wrap(%r)" % getattr(libffi, name)
+ if hasattr(clibffi, name):
+ Module.interpleveldefs[name] = "space.wrap(%r)" % getattr(clibffi, name)
super(Module, cls).buildloaders()
buildloaders = classmethod(buildloaders)
Modified: pypy/branch/jit-unroll-loops/pypy/module/_rawffi/array.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/_rawffi/array.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/_rawffi/array.py Sat Oct 30 16:41:17 2010
@@ -97,7 +97,15 @@
class W_ArrayInstance(W_DataInstance):
def __init__(self, space, shape, length, address=r_uint(0)):
- W_DataInstance.__init__(self, space, shape.size * length, address)
+ # Workaround for a strange behavior of libffi: make sure that
+ # we always have at least 8 bytes. For W_ArrayInstances that are
+ # used as the result value of a function call, ffi_call() writes
+ # 8 bytes into it even if the function's result type asks for less.
+ # This strange behavior is documented.
+ memsize = shape.size * length
+ if memsize < 8:
+ memsize = 8
+ W_DataInstance.__init__(self, space, memsize, address)
self.length = length
self.shape = shape
Modified: pypy/branch/jit-unroll-loops/pypy/module/_rawffi/callback.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/_rawffi/callback.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/_rawffi/callback.py Sat Oct 30 16:41:17 2010
@@ -8,8 +8,8 @@
from pypy.module._rawffi.array import get_elem, push_elem
from pypy.module._rawffi.interp_rawffi import W_DataInstance, letter2tp, \
wrap_value, unwrap_value, unwrap_truncate_int
-from pypy.rlib.libffi import USERDATA_P, CallbackFuncPtr, FUNCFLAG_CDECL
-from pypy.rlib.libffi import ffi_type_void
+from pypy.rlib.clibffi import USERDATA_P, CallbackFuncPtr, FUNCFLAG_CDECL
+from pypy.rlib.clibffi import ffi_type_void
from pypy.module._rawffi.tracker import tracker
from pypy.interpreter.error import OperationError
from pypy.interpreter import gateway
Modified: pypy/branch/jit-unroll-loops/pypy/module/_rawffi/interp_rawffi.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/_rawffi/interp_rawffi.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/_rawffi/interp_rawffi.py Sat Oct 30 16:41:17 2010
@@ -5,7 +5,7 @@
from pypy.interpreter.gateway import interp2app, NoneNotWrapped
from pypy.interpreter.typedef import TypeDef, GetSetProperty
-from pypy.rlib.libffi import *
+from pypy.rlib.clibffi import *
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.rlib.unroll import unrolling_iterable
Modified: pypy/branch/jit-unroll-loops/pypy/module/_rawffi/structure.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/_rawffi/structure.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/_rawffi/structure.py Sat Oct 30 16:41:17 2010
@@ -15,7 +15,7 @@
from pypy.module._rawffi.interp_rawffi import wrap_value, unwrap_value
from pypy.module._rawffi.interp_rawffi import unpack_shape_with_length
from pypy.module._rawffi.interp_rawffi import size_alignment
-from pypy.rlib import libffi
+from pypy.rlib import clibffi
from pypy.rlib.rarithmetic import intmask, r_uint
def unpack_fields(space, w_fields):
@@ -34,7 +34,7 @@
def round_up(size, alignment):
return (size + alignment - 1) & -alignment
-def size_alignment_pos(fields):
+def size_alignment_pos(fields, is_union=False):
size = 0
alignment = 1
pos = []
@@ -42,16 +42,20 @@
# fieldtype is a W_Array
fieldsize = fieldtype.size
fieldalignment = fieldtype.alignment
- size = round_up(size, fieldalignment)
alignment = max(alignment, fieldalignment)
- pos.append(size)
- size += intmask(fieldsize)
+ if is_union:
+ pos.append(0)
+ size = max(size, fieldsize)
+ else:
+ size = round_up(size, fieldalignment)
+ pos.append(size)
+ size += intmask(fieldsize)
size = round_up(size, alignment)
return size, alignment, pos
class W_Structure(W_DataShape):
- def __init__(self, space, fields, size, alignment):
+ def __init__(self, space, fields, size, alignment, is_union=False):
name_to_index = {}
if fields is not None:
for i in range(len(fields)):
@@ -60,7 +64,7 @@
raise operationerrfmt(space.w_ValueError,
"duplicate field name %s", name)
name_to_index[name] = i
- size, alignment, pos = size_alignment_pos(fields)
+ size, alignment, pos = size_alignment_pos(fields, is_union)
else: # opaque case
fields = []
pos = []
@@ -104,14 +108,14 @@
descr_fieldoffset.unwrap_spec = ['self', ObjSpace, str]
# get the corresponding ffi_type
- ffi_struct = lltype.nullptr(libffi.FFI_STRUCT_P.TO)
+ ffi_struct = lltype.nullptr(clibffi.FFI_STRUCT_P.TO)
def get_basic_ffi_type(self):
if not self.ffi_struct:
# Repeated fields are delicate. Consider for example
# struct { int a[5]; }
# or struct { struct {int x;} a[5]; }
- # Seeing no corresponding doc in libffi, let's just repeat
+ # Seeing no corresponding doc in clibffi, let's just repeat
# the field 5 times...
fieldtypes = []
for name, tp in self.fields:
@@ -122,7 +126,7 @@
while count + basic_size <= total_size:
fieldtypes.append(basic_ffi_type)
count += basic_size
- self.ffi_struct = libffi.make_struct_ffitype_e(self.size,
+ self.ffi_struct = clibffi.make_struct_ffitype_e(self.size,
self.alignment,
fieldtypes)
return self.ffi_struct.ffistruct
@@ -133,15 +137,17 @@
-def descr_new_structure(space, w_type, w_shapeinfo):
+def descr_new_structure(space, w_type, w_shapeinfo, union=0):
+ is_union = bool(union)
if space.is_true(space.isinstance(w_shapeinfo, space.w_tuple)):
w_size, w_alignment = space.fixedview(w_shapeinfo, expected_length=2)
S = W_Structure(space, None, space.int_w(w_size),
- space.int_w(w_alignment))
+ space.int_w(w_alignment), is_union)
else:
fields = unpack_fields(space, w_shapeinfo)
- S = W_Structure(space, fields, 0, 0)
+ S = W_Structure(space, fields, 0, 0, is_union)
return space.wrap(S)
+descr_new_structure.unwrap_spec = [ObjSpace, W_Root, W_Root, int]
W_Structure.typedef = TypeDef(
'Structure',
Modified: pypy/branch/jit-unroll-loops/pypy/module/_rawffi/test/test__rawffi.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/_rawffi/test/test__rawffi.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/_rawffi/test/test__rawffi.py Sat Oct 30 16:41:17 2010
@@ -70,7 +70,7 @@
return s[num];
}
- char *char_check(char x, char y)
+ const char *char_check(char x, char y)
{
if (y == static_str[0])
return static_str;
@@ -185,13 +185,14 @@
sum_x_y
give perturb get_s2a check_s2a
AAA_first_ordinal_function
+ ret_un_func
""".split()
eci = ExternalCompilationInfo(export_symbols=symbols)
return str(platform.compile([c_file], eci, 'x', standalone=False))
prepare_c_example = staticmethod(prepare_c_example)
def setup_class(cls):
- from pypy.rlib.libffi import get_libc_name
+ from pypy.rlib.clibffi import get_libc_name
space = gettestobjspace(usemodules=('_rawffi', 'struct'))
cls.space = space
cls.w_lib_name = space.wrap(cls.prepare_c_example())
@@ -945,14 +946,15 @@
assert a[4] == 't'
def test_union(self):
- skip("segfaulting")
import _rawffi
longsize = _rawffi.sizeof('l')
- S = _rawffi.Structure((longsize, longsize))
+ S = _rawffi.Structure([('x', 'h'), ('y', 'l')], union=True)
s = S(autofree=False)
+ s.x = 12345
lib = _rawffi.CDLL(self.lib_name)
f = lib.ptr('ret_un_func', [(S, 1)], (S, 1))
ret = f(s)
+ assert ret.y == 1234500, "ret.y == %d" % (ret.y,)
s.free()
class AppTestAutoFree:
Modified: pypy/branch/jit-unroll-loops/pypy/module/_socket/test/test_sock_app.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/_socket/test/test_sock_app.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/_socket/test/test_sock_app.py Sat Oct 30 16:41:17 2010
@@ -254,6 +254,7 @@
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
fd = s.fileno()
w_obj = rsocket.make_address(c_addr, addrlen).as_object(fd, space)
+ lltype.free(c_addr_ll, flavor='raw')
assert space.is_true(space.eq(w_obj, space.newtuple([
space.wrap('lo'),
space.wrap(socket.ntohs(8)),
Modified: pypy/branch/jit-unroll-loops/pypy/module/_sre/interp_sre.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/_sre/interp_sre.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/_sre/interp_sre.py Sat Oct 30 16:41:17 2010
@@ -290,10 +290,8 @@
def SRE_Pattern__new__(space, w_subtype, w_pattern, flags, w_code,
groups=0, w_groupindex=None, w_indexgroup=None):
n = space.int_w(space.len(w_code))
- code = [0] * n
- for i in range(n):
- x = space.uint_w(space.getitem(w_code, space.wrap(i)))
- code[i] = intmask(x)
+ code = [intmask(space.uint_w(space.getitem(w_code, space.wrap(i))))
+ for i in range(n)]
#
w_srepat = space.allocate_instance(W_SRE_Pattern, w_subtype)
srepat = space.interp_w(W_SRE_Pattern, w_srepat)
Modified: pypy/branch/jit-unroll-loops/pypy/module/_weakref/interp__weakref.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/_weakref/interp__weakref.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/_weakref/interp__weakref.py Sat Oct 30 16:41:17 2010
@@ -7,7 +7,7 @@
import weakref
-class WeakrefLifeline(object):
+class WeakrefLifeline(W_Root):
def __init__(self, space):
self.space = space # this is here for W_Root.clear_all_weakrefs()
self.refs_weak = []
Modified: pypy/branch/jit-unroll-loops/pypy/module/_winreg/interp_winreg.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/_winreg/interp_winreg.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/_winreg/interp_winreg.py Sat Oct 30 16:41:17 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
from pypy.interpreter.baseobjspace import Wrappable
from pypy.interpreter.baseobjspace import ObjSpace, W_Root
from pypy.interpreter.gateway import interp2app
@@ -212,13 +213,10 @@
subkey = None
else:
subkey = space.str_w(w_subkey)
- dataptr = rffi.str2charp(value)
- try:
+ with rffi.scoped_str2charp(value) as dataptr:
ret = rwinreg.RegSetValue(hkey, subkey, rwinreg.REG_SZ, dataptr, len(value))
- finally:
- rffi.free_charp(dataptr)
- if ret != 0:
- raiseWindowsError(space, ret, 'RegSetValue')
+ if ret != 0:
+ raiseWindowsError(space, ret, 'RegSetValue')
SetValue.unwrap_spec = [ObjSpace, W_Root, W_Root, int, str]
def QueryValue(space, w_hkey, w_subkey):
@@ -237,23 +235,15 @@
subkey = None
else:
subkey = space.str_w(w_subkey)
- bufsize_p = lltype.malloc(rwin32.PLONG.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.PLONG.TO, 1) as bufsize_p:
ret = rwinreg.RegQueryValue(hkey, subkey, None, bufsize_p)
if ret != 0:
raiseWindowsError(space, ret, 'RegQueryValue')
- buf = lltype.malloc(rffi.CCHARP.TO, bufsize_p[0], flavor='raw')
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, bufsize_p[0]) as buf:
ret = rwinreg.RegQueryValue(hkey, subkey, buf, bufsize_p)
if ret != 0:
raiseWindowsError(space, ret, 'RegQueryValue')
return space.wrap(rffi.charp2strn(buf, bufsize_p[0] - 1))
- finally:
- lltype.free(buf, flavor='raw')
- finally:
- lltype.free(bufsize_p, flavor='raw')
- if ret != 0:
- raiseWindowsError(space, ret, 'RegQueryValue')
QueryValue.unwrap_spec = [ObjSpace, W_Root, W_Root]
def convert_to_regdata(space, w_value, typ):
@@ -412,16 +402,14 @@
value_name is a string indicating the value to query"""
hkey = hkey_w(w_hkey, space)
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
- retDataSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retDataSize:
ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword, null_dword,
None, retDataSize)
if ret != 0:
raiseWindowsError(space, ret, 'RegQueryValueEx')
- databuf = lltype.malloc(rffi.CCHARP.TO, retDataSize[0], flavor='raw')
- try:
- retType = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+
+ with lltype.scoped_alloc(rffi.CCHARP.TO, retDataSize[0]) as databuf:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType:
ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword,
retType, databuf, retDataSize)
@@ -432,12 +420,6 @@
retDataSize[0], retType[0]),
space.wrap(retType[0]),
])
- finally:
- lltype.free(retType, flavor='raw')
- finally:
- lltype.free(databuf, flavor='raw')
- finally:
- lltype.free(retDataSize, flavor='raw')
QueryValueEx.unwrap_spec = [ObjSpace, W_Root, str]
@@ -454,14 +436,11 @@
The return value is the handle of the opened key.
If the function fails, an exception is raised."""
hkey = hkey_w(w_hkey, space)
- rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey:
ret = rwinreg.RegCreateKey(hkey, subkey, rethkey)
if ret != 0:
raiseWindowsError(space, ret, 'CreateKey')
return space.wrap(W_HKEY(rethkey[0]))
- finally:
- lltype.free(rethkey, flavor='raw')
CreateKey.unwrap_spec = [ObjSpace, W_Root, str]
def DeleteKey(space, w_hkey, subkey):
@@ -504,14 +483,11 @@
The result is a new handle to the specified key
If the function fails, an EnvironmentError exception is raised."""
hkey = hkey_w(w_hkey, space)
- rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey:
ret = rwinreg.RegOpenKeyEx(hkey, subkey, res, sam, rethkey)
if ret != 0:
raiseWindowsError(space, ret, 'RegOpenKeyEx')
return space.wrap(W_HKEY(rethkey[0]))
- finally:
- lltype.free(rethkey, flavor='raw')
OpenKey.unwrap_spec = [ObjSpace, W_Root, str, int, rffi.r_uint]
def EnumValue(space, w_hkey, index):
@@ -531,10 +507,8 @@
hkey = hkey_w(w_hkey, space)
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
- retValueSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
- retDataSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retValueSize:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retDataSize:
ret = rwinreg.RegQueryInfoKey(
hkey, None, null_dword, null_dword,
null_dword, null_dword, null_dword,
@@ -546,14 +520,9 @@
retValueSize[0] += 1
retDataSize[0] += 1
- valuebuf = lltype.malloc(rffi.CCHARP.TO, retValueSize[0],
- flavor='raw')
- try:
- databuf = lltype.malloc(rffi.CCHARP.TO, retDataSize[0],
- flavor='raw')
- try:
- retType = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, retValueSize[0]) as valuebuf:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, retDataSize[0]) as databuf:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType:
ret = rwinreg.RegEnumValue(
hkey, index, valuebuf, retValueSize,
null_dword, retType, databuf, retDataSize)
@@ -566,16 +535,6 @@
retDataSize[0], retType[0]),
space.wrap(retType[0]),
])
- finally:
- lltype.free(retType, flavor='raw')
- finally:
- lltype.free(databuf, flavor='raw')
- finally:
- lltype.free(valuebuf, flavor='raw')
- finally:
- lltype.free(retDataSize, flavor='raw')
- finally:
- lltype.free(retValueSize, flavor='raw')
EnumValue.unwrap_spec = [ObjSpace, W_Root, int]
@@ -592,10 +551,8 @@
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
# max key name length is 255
- buf = lltype.malloc(rffi.CCHARP.TO, 256, flavor='raw')
- try:
- retValueSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, 256) as buf:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retValueSize:
retValueSize[0] = 256 # includes NULL terminator
ret = rwinreg.RegEnumKeyEx(hkey, index, buf, retValueSize,
null_dword, None, null_dword,
@@ -603,10 +560,6 @@
if ret != 0:
raiseWindowsError(space, ret, 'RegEnumKeyEx')
return space.wrap(rffi.charp2str(buf))
- finally:
- lltype.free(retValueSize, flavor='raw')
- finally:
- lltype.free(buf, flavor='raw')
EnumKey.unwrap_spec = [ObjSpace, W_Root, int]
@@ -621,12 +574,9 @@
A long integer that identifies when the key was last modified (if available)
as 100's of nanoseconds since Jan 1, 1600."""
hkey = hkey_w(w_hkey, space)
- nSubKeys = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
- nValues = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw')
- try:
- ft = lltype.malloc(rwin32.PFILETIME.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as nSubKeys:
+ with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as nValues:
+ with lltype.scoped_alloc(rwin32.PFILETIME.TO, 1) as ft:
null_dword = lltype.nullptr(rwin32.LPDWORD.TO)
ret = rwinreg.RegQueryInfoKey(
hkey, None, null_dword, null_dword,
@@ -640,12 +590,6 @@
return space.newtuple([space.wrap(nSubKeys[0]),
space.wrap(nValues[0]),
space.wrap(l)])
- finally:
- lltype.free(ft, flavor='raw')
- finally:
- lltype.free(nValues, flavor='raw')
- finally:
- lltype.free(nSubKeys, flavor='raw')
QueryInfoKey.unwrap_spec = [ObjSpace, W_Root]
def str_or_None_w(space, w_obj):
@@ -666,12 +610,9 @@
If the function fails, an EnvironmentError exception is raised."""
machine = str_or_None_w(space, w_machine)
hkey = hkey_w(w_hkey, space)
- rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw')
- try:
+ with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey:
ret = rwinreg.RegConnectRegistry(machine, hkey, rethkey)
if ret != 0:
raiseWindowsError(space, ret, 'RegConnectRegistry')
return space.wrap(W_HKEY(rethkey[0]))
- finally:
- lltype.free(rethkey, flavor='raw')
ConnectRegistry.unwrap_spec = [ObjSpace, W_Root, W_Root]
Modified: pypy/branch/jit-unroll-loops/pypy/module/array/interp_array.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/array/interp_array.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/array/interp_array.py Sat Oct 30 16:41:17 2010
@@ -192,32 +192,30 @@
mytype.bytes
raise OperationError(space.w_OverflowError,
space.wrap(msg))
- elif mytype.unwrap == 'str_w' or mytype.unwrap == 'unicode_w':
+ return rffi.cast(mytype.itemtype, item)
+ if mytype.unwrap == 'str_w' or mytype.unwrap == 'unicode_w':
if len(item) != 1:
msg = 'array item must be char'
raise OperationError(space.w_TypeError, space.wrap(msg))
item = item[0]
-
+ return rffi.cast(mytype.itemtype, item)
+ #
+ # "regular" case: it fits in an rpython integer (lltype.Signed)
+ result = rffi.cast(mytype.itemtype, item)
if mytype.canoverflow:
- msg = None
- if mytype.signed:
- if item < -1 << (mytype.bytes * 8 - 1):
+ if rffi.cast(lltype.Signed, result) != item:
+ # overflow. build the correct message
+ if item < 0:
msg = ('signed %d-byte integer is less than minimum' %
mytype.bytes)
- elif item > (1 << (mytype.bytes * 8 - 1)) - 1:
+ else:
msg = ('signed %d-byte integer is greater than maximum'
% mytype.bytes)
- else:
- if item < 0:
- msg = ('unsigned %d-byte integer is less than minimum'
- % mytype.bytes)
- elif item > (1 << (mytype.bytes * 8)) - 1:
- msg = ('unsigned %d-byte integer is greater'
- ' than maximum' % mytype.bytes)
- if msg is not None:
+ if not mytype.signed:
+ msg = 'un' + msg # 'signed' => 'unsigned'
raise OperationError(space.w_OverflowError,
space.wrap(msg))
- return rffi.cast(mytype.itemtype, item)
+ return result
def __del__(self):
self.setlen(0)
Modified: pypy/branch/jit-unroll-loops/pypy/module/bz2/interp_bz2.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/bz2/interp_bz2.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/bz2/interp_bz2.py Sat Oct 30 16:41:17 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
from pypy.rpython.tool import rffi_platform as platform
from pypy.rpython.lltypesystem import rffi
from pypy.rpython.lltypesystem import lltype
@@ -225,6 +226,11 @@
if self.current_size > 0:
rffi.keep_buffer_alive_until_here(self.raw_buf, self.gc_buf)
+ def __enter__(self):
+ return self
+ def __exit__(self, *args):
+ self.free()
+
# ____________________________________________________________
#
# Make the BZ2File type by internally inheriting from W_File.
@@ -531,33 +537,30 @@
if not self.running:
raise OperationError(self.space.w_ValueError,
self.space.wrap("this object was already flushed"))
-
- out = OutBuffer(self.bzs)
+
in_bufsize = datasize
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(datasize):
- in_buf[i] = data[i]
- try:
-
- self.bzs.c_next_in = in_buf
- rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize)
+ with OutBuffer(self.bzs) as out:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
- while True:
- bzerror = BZ2_bzCompress(self.bzs, BZ_RUN)
- if bzerror != BZ_RUN_OK:
- _catch_bz2_error(self.space, bzerror)
+ for i in range(datasize):
+ in_buf[i] = data[i]
- if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
- break
- elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
+ self.bzs.c_next_in = in_buf
+ rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize)
- res = out.make_result_string()
- return self.space.wrap(res)
- finally:
- lltype.free(in_buf, flavor='raw')
- out.free()
+ while True:
+ bzerror = BZ2_bzCompress(self.bzs, BZ_RUN)
+ if bzerror != BZ_RUN_OK:
+ _catch_bz2_error(self.space, bzerror)
+
+ if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
+ break
+ elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
+
+ res = out.make_result_string()
+ return self.space.wrap(res)
compress.unwrap_spec = ['self', 'bufferstr']
@@ -566,9 +569,8 @@
raise OperationError(self.space.w_ValueError,
self.space.wrap("this object was already flushed"))
self.running = False
-
- out = OutBuffer(self.bzs)
- try:
+
+ with OutBuffer(self.bzs) as out:
while True:
bzerror = BZ2_bzCompress(self.bzs, BZ_FINISH)
if bzerror == BZ_STREAM_END:
@@ -581,8 +583,6 @@
res = out.make_result_string()
return self.space.wrap(res)
- finally:
- out.free()
flush.unwrap_spec = ['self']
W_BZ2Compressor.typedef = TypeDef("BZ2Compressor",
@@ -641,38 +641,37 @@
if not self.running:
raise OperationError(self.space.w_EOFError,
self.space.wrap("end of stream was already found"))
-
+
in_bufsize = len(data)
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(in_bufsize):
- in_buf[i] = data[i]
- out = OutBuffer(self.bzs)
- try:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
+ for i in range(in_bufsize):
+ in_buf[i] = data[i]
self.bzs.c_next_in = in_buf
rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize)
- while True:
- bzerror = BZ2_bzDecompress(self.bzs)
- if bzerror == BZ_STREAM_END:
- if rffi.getintfield(self.bzs, 'c_avail_in') != 0:
- unused = [self.bzs.c_next_in[i] for i in range(rffi.getintfield(self.bzs, 'c_avail_in'))]
- self.unused_data = "".join(unused)
- self.running = False
- break
- if bzerror != BZ_OK:
- _catch_bz2_error(self.space, bzerror)
-
- if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
- break
- elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
+ with OutBuffer(self.bzs) as out:
+ while True:
+ bzerror = BZ2_bzDecompress(self.bzs)
+ if bzerror == BZ_STREAM_END:
+ if rffi.getintfield(self.bzs, 'c_avail_in') != 0:
+ unused = [self.bzs.c_next_in[i]
+ for i in range(
+ rffi.getintfield(self.bzs,
+ 'c_avail_in'))]
+ self.unused_data = "".join(unused)
+ self.running = False
+ break
+ if bzerror != BZ_OK:
+ _catch_bz2_error(self.space, bzerror)
+
+ if rffi.getintfield(self.bzs, 'c_avail_in') == 0:
+ break
+ elif rffi.getintfield(self.bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
- res = out.make_result_string()
- return self.space.wrap(res)
- finally:
- lltype.free(in_buf, flavor='raw')
- out.free()
+ res = out.make_result_string()
+ return self.space.wrap(res)
decompress.unwrap_spec = ['self', 'bufferstr']
@@ -695,43 +694,39 @@
if compresslevel < 1 or compresslevel > 9:
raise OperationError(space.w_ValueError,
space.wrap("compresslevel must be between 1 and 9"))
-
- bzs = lltype.malloc(bz_stream.TO, flavor='raw', zero=True)
- in_bufsize = len(data)
- # conforming to bz2 manual, this is large enough to fit compressed
- # data in one shot. We will check it later anyway.
- out = OutBuffer(bzs, in_bufsize + (in_bufsize / 100 + 1) + 600)
-
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(in_bufsize):
- in_buf[i] = data[i]
-
- try:
- bzs.c_next_in = in_buf
- rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
- bzerror = BZ2_bzCompressInit(bzs, compresslevel, 0, 0)
- if bzerror != BZ_OK:
- _catch_bz2_error(space, bzerror)
+ with lltype.scoped_alloc(bz_stream.TO, zero=True) as bzs:
+ in_bufsize = len(data)
+
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
+ for i in range(in_bufsize):
+ in_buf[i] = data[i]
+ bzs.c_next_in = in_buf
+ rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
+
+ # conforming to bz2 manual, this is large enough to fit compressed
+ # data in one shot. We will check it later anyway.
+ with OutBuffer(bzs,
+ in_bufsize + (in_bufsize / 100 + 1) + 600) as out:
+
+ bzerror = BZ2_bzCompressInit(bzs, compresslevel, 0, 0)
+ if bzerror != BZ_OK:
+ _catch_bz2_error(space, bzerror)
- while True:
- bzerror = BZ2_bzCompress(bzs, BZ_FINISH)
- if bzerror == BZ_STREAM_END:
- break
- elif bzerror != BZ_FINISH_OK:
+ while True:
+ bzerror = BZ2_bzCompress(bzs, BZ_FINISH)
+ if bzerror == BZ_STREAM_END:
+ break
+ elif bzerror != BZ_FINISH_OK:
+ BZ2_bzCompressEnd(bzs)
+ _catch_bz2_error(space, bzerror)
+
+ if rffi.getintfield(bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
+
+ res = out.make_result_string()
BZ2_bzCompressEnd(bzs)
- _catch_bz2_error(space, bzerror)
-
- if rffi.getintfield(bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
-
- res = out.make_result_string()
- BZ2_bzCompressEnd(bzs)
- return space.wrap(res)
- finally:
- lltype.free(bzs, flavor='raw')
- lltype.free(in_buf, flavor='raw')
- out.free()
+ return space.wrap(res)
compress.unwrap_spec = [ObjSpace, 'bufferstr', int]
def decompress(space, data):
@@ -744,40 +739,34 @@
if in_bufsize == 0:
return space.wrap("")
- bzs = lltype.malloc(bz_stream.TO, flavor='raw', zero=True)
- in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw')
- for i in range(in_bufsize):
- in_buf[i] = data[i]
-
- out = OutBuffer(bzs)
- try:
- bzs.c_next_in = in_buf
- rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
-
- bzerror = BZ2_bzDecompressInit(bzs, 0, 0)
- if bzerror != BZ_OK:
- _catch_bz2_error(space, bzerror)
-
- while True:
- bzerror = BZ2_bzDecompress(bzs)
- if bzerror == BZ_STREAM_END:
- break
- if bzerror != BZ_OK:
- BZ2_bzDecompressEnd(bzs)
- _catch_bz2_error(space, bzerror)
-
- if rffi.getintfield(bzs, 'c_avail_in') == 0:
+ with lltype.scoped_alloc(bz_stream.TO, zero=True) as bzs:
+ with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf:
+ for i in range(in_bufsize):
+ in_buf[i] = data[i]
+ bzs.c_next_in = in_buf
+ rffi.setintfield(bzs, 'c_avail_in', in_bufsize)
+
+ with OutBuffer(bzs) as out:
+ bzerror = BZ2_bzDecompressInit(bzs, 0, 0)
+ if bzerror != BZ_OK:
+ _catch_bz2_error(space, bzerror)
+
+ while True:
+ bzerror = BZ2_bzDecompress(bzs)
+ if bzerror == BZ_STREAM_END:
+ break
+ if bzerror != BZ_OK:
+ BZ2_bzDecompressEnd(bzs)
+ _catch_bz2_error(space, bzerror)
+
+ if rffi.getintfield(bzs, 'c_avail_in') == 0:
+ BZ2_bzDecompressEnd(bzs)
+ raise OperationError(space.w_ValueError, space.wrap(
+ "couldn't find end of stream"))
+ elif rffi.getintfield(bzs, 'c_avail_out') == 0:
+ out.prepare_next_chunk()
+
+ res = out.make_result_string()
BZ2_bzDecompressEnd(bzs)
- raise OperationError(space.w_ValueError,
- space.wrap("couldn't find end of stream"))
- elif rffi.getintfield(bzs, 'c_avail_out') == 0:
- out.prepare_next_chunk()
-
- res = out.make_result_string()
- BZ2_bzDecompressEnd(bzs)
- return space.wrap(res)
- finally:
- lltype.free(bzs, flavor='raw')
- lltype.free(in_buf, flavor='raw')
- out.free()
+ return space.wrap(res)
decompress.unwrap_spec = [ObjSpace, 'bufferstr']
Modified: pypy/branch/jit-unroll-loops/pypy/module/cpyext/classobject.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/cpyext/classobject.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/cpyext/classobject.py Sat Oct 30 16:41:17 2010
@@ -15,16 +15,20 @@
class is the class of new object. The dict parameter will be used as the
object's __dict__; if NULL, a new dictionary will be created for the
instance."""
- if not PyClass_Check(space, w_class):
+ if not isinstance(w_class, W_ClassObject):
return PyErr_BadInternalCall(space)
- return W_InstanceObject(space, w_class, w_dict)
+ w_result = w_class.instantiate(space)
+ if w_dict is not None:
+ w_result.setdict(space, w_dict)
+ return w_result
@cpython_api([PyObject, PyObject], PyObject, error=CANNOT_FAIL)
def _PyInstance_Lookup(space, w_instance, w_name):
+ name = space.str_w(w_name)
assert isinstance(w_instance, W_InstanceObject)
- w_result = space.finditem(w_instance.w_dict, w_name)
+ w_result = w_instance.getdictvalue(space, name)
if w_result is not None:
return w_result
- return w_instance.w_class.lookup(space, w_name)
+ return w_instance.w_class.lookup(space, name)
Modified: pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_cpyext.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_cpyext.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_cpyext.py Sat Oct 30 16:41:17 2010
@@ -16,6 +16,7 @@
from pypy.module.cpyext.pyobject import Py_DecRef, InvalidPointerException
from pypy.translator.goal import autopath
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
@api.cpython_api([], api.PyObject)
def PyPy_Crash1(space):
@@ -78,7 +79,6 @@
self.frozen_refcounts[w_obj] = obj.c_ob_refcnt
#state.print_refcounts()
self.frozen_ll2callocations = set(ll2ctypes.ALLOCATED.values())
- lltype.start_tracking_allocations()
class LeakCheckingTest(object):
def check_and_print_leaks(self):
@@ -126,17 +126,8 @@
for w_obj in lost_objects_w:
print >>sys.stderr, "Lost object %r" % (w_obj, )
leaking = True
- for llvalue in set(ll2ctypes.ALLOCATED.values()) - self.frozen_ll2callocations:
- if getattr(llvalue, "_traceback", None): # this means that the allocation should be tracked
- leaking = True
- print >>sys.stderr, "Did not deallocate %r (ll2ctypes)" % (llvalue, )
- print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines())
- for llvalue in lltype.ALLOCATED.keys():
- leaking = True
- print >>sys.stderr, "Did not deallocate %r (llvalue)" % (llvalue, )
- print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines())
-
- lltype.stop_tracking_allocations()
+ # the actual low-level leak checking is done by pypy.tool.leakfinder,
+ # enabled automatically by pypy.conftest.
return leaking
class AppTestCpythonExtensionBase(LeakCheckingTest):
Modified: pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_unicodeobject.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_unicodeobject.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_unicodeobject.py Sat Oct 30 16:41:17 2010
@@ -177,13 +177,14 @@
encoded_charp = rffi.str2charp(encoded)
strict_charp = rffi.str2charp("strict")
if endian is not None:
- pendian = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
if endian < 0:
- pendian[0] = -1
+ value = -1
elif endian > 0:
- pendian[0] = 1
+ value = 1
else:
- pendian[0] = 0
+ value = 0
+ pendian = lltype.malloc(rffi.INTP.TO, 1, flavor='raw')
+ pendian[0] = rffi.cast(rffi.INT, value)
else:
pendian = None
Modified: pypy/branch/jit-unroll-loops/pypy/module/gc/interp_gc.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/gc/interp_gc.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/gc/interp_gc.py Sat Oct 30 16:41:17 2010
@@ -5,6 +5,11 @@
def collect(space):
"Run a full collection."
+ # First clear the method cache. See test_gc for an example of why.
+ if space.config.objspace.std.withmethodcache:
+ from pypy.objspace.std.typeobject import MethodCache
+ cache = space.fromcache(MethodCache)
+ cache.clear()
rgc.collect()
return space.wrap(0)
Modified: pypy/branch/jit-unroll-loops/pypy/module/gc/referents.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/gc/referents.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/gc/referents.py Sat Oct 30 16:41:17 2010
@@ -15,8 +15,16 @@
def try_cast_gcref_to_w_root(gcref):
w_obj = rgc.try_cast_gcref_to_instance(W_Root, gcref)
- if not we_are_translated() and not hasattr(w_obj, 'typedef'):
- w_obj = None
+ # Ignore the instances of W_Root that are not really valid as Python
+ # objects. There is e.g. WeakrefLifeline in module/_weakref that
+ # inherits from W_Root for internal reasons. Such instances don't
+ # have a typedef at all (or have a null typedef after translation).
+ if not we_are_translated():
+ if not hasattr(w_obj, 'typedef'):
+ return None
+ else:
+ if w_obj is None or not w_obj.typedef:
+ return None
return w_obj
def wrap(space, gcref):
Modified: pypy/branch/jit-unroll-loops/pypy/module/gc/test/test_gc.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/gc/test/test_gc.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/gc/test/test_gc.py Sat Oct 30 16:41:17 2010
@@ -103,3 +103,22 @@
import gc
gc.dump_heap_stats(self.fname)
+
+class AppTestGcMethodCache(object):
+ def setup_class(cls):
+ cls.space = gettestobjspace(**{"objspace.std.withmethodcache": True})
+
+ def test_clear_method_cache(self):
+ import gc, weakref
+ rlist = []
+ def f():
+ class C(object):
+ def f(self):
+ pass
+ C().f() # Fill the method cache
+ rlist.append(weakref.ref(C))
+ for i in range(5):
+ f()
+ gc.collect() # the classes C should all go away here
+ for r in rlist:
+ assert r() is None
Modified: pypy/branch/jit-unroll-loops/pypy/module/imp/importing.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/imp/importing.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/imp/importing.py Sat Oct 30 16:41:17 2010
@@ -122,7 +122,7 @@
n = len(ctxt_name_prefix_parts)-level+1
assert n>=0
ctxt_name_prefix_parts = ctxt_name_prefix_parts[:n]
- if ctxt_w_path is None: # plain module
+ if ctxt_name_prefix_parts and ctxt_w_path is None: # plain module
ctxt_name_prefix_parts.pop()
if ctxt_name_prefix_parts:
rel_modulename = '.'.join(ctxt_name_prefix_parts)
Modified: pypy/branch/jit-unroll-loops/pypy/module/imp/test/test_import.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/imp/test/test_import.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/imp/test/test_import.py Sat Oct 30 16:41:17 2010
@@ -360,6 +360,12 @@
""".rstrip()
raises(ValueError, imp)
+ def test_future_relative_import_error_when_in_non_package2(self):
+ exec """def imp():
+ from .. import inpackage
+ """.rstrip()
+ raises(ValueError, imp)
+
def test_relative_import_with___name__(self):
import sys
mydict = {'__name__': 'sys.foo'}
@@ -784,6 +790,7 @@
extrapath = udir.ensure("pythonpath", dir=1)
extrapath.join("urllib.py").write("print 42\n")
old = os.environ.get('PYTHONPATH', None)
+ oldlang = os.environ.pop('LANG', None)
try:
os.environ['PYTHONPATH'] = str(extrapath)
output = py.process.cmdexec('''"%s" "%s" -c "import urllib"''' %
@@ -792,6 +799,8 @@
finally:
if old:
os.environ['PYTHONPATH'] = old
+ if oldlang:
+ os.environ['LANG'] = oldlang
class AppTestImportHooks(object):
def test_meta_path(self):
Modified: pypy/branch/jit-unroll-loops/pypy/module/parser/test/test_parser.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/parser/test/test_parser.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/parser/test/test_parser.py Sat Oct 30 16:41:17 2010
@@ -10,6 +10,9 @@
cls.w_m = space.appexec([], """():
import parser
return parser""")
+ cls.w_symbol = space.appexec([], """():
+ import symbol
+ return symbol""")
class AppTestParser(ParserModuleTest):
@@ -36,7 +39,7 @@
seq = getattr(s, meth)()
assert isinstance(seq, tp)
assert len(seq) == 4
- assert seq[0] == 286
+ assert seq[0] == self.symbol.file_input
assert len(seq[2]) == 2
assert len(seq[3]) == 2
assert seq[2][0] == 4
Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/__init__.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/pypyjit/__init__.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/pypyjit/__init__.py Sat Oct 30 16:41:17 2010
@@ -6,6 +6,7 @@
interpleveldefs = {
'set_param': 'interp_jit.set_param',
+ 'residual_call': 'interp_jit.residual_call',
}
def setup_after_space_initialization(self):
Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/interp_jit.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/pypyjit/interp_jit.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/pypyjit/interp_jit.py Sat Oct 30 16:41:17 2010
@@ -5,10 +5,10 @@
from pypy.tool.pairtype import extendabletype
from pypy.rlib.rarithmetic import r_uint, intmask
-from pypy.rlib.jit import JitDriver, hint, we_are_jitted
+from pypy.rlib.jit import JitDriver, hint, we_are_jitted, dont_look_inside
import pypy.interpreter.pyopcode # for side-effects
from pypy.interpreter.error import OperationError, operationerrfmt
-from pypy.interpreter.gateway import ObjSpace, Arguments
+from pypy.interpreter.gateway import ObjSpace, Arguments, W_Root
from pypy.interpreter.pycode import PyCode, CO_GENERATOR
from pypy.interpreter.pyframe import PyFrame
from pypy.interpreter.pyopcode import ExitFrame
@@ -131,3 +131,10 @@
"no JIT parameter '%s'", key)
set_param.unwrap_spec = [ObjSpace, Arguments]
+
+ at dont_look_inside
+def residual_call(space, w_callable, args):
+ '''For testing. Invokes callable(...), but without letting
+ the JIT follow the call.'''
+ return space.call_args(w_callable, args)
+residual_call.unwrap_spec = [ObjSpace, W_Root, Arguments]
Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/policy.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/pypyjit/policy.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/pypyjit/policy.py Sat Oct 30 16:41:17 2010
@@ -11,7 +11,7 @@
if '.' in modname:
modname, _ = modname.split('.', 1)
if modname in ['pypyjit', 'signal', 'micronumpy', 'math', 'exceptions',
- 'imp', 'sys', 'array']:
+ 'imp', 'sys', 'array', '_ffi']:
return True
return False
Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py Sat Oct 30 16:41:17 2010
@@ -79,8 +79,11 @@
class PyPyCJITTests(object):
- def run_source(self, source, expected_max_ops, *testcases):
+ def run_source(self, source, expected_max_ops, *testcases, **kwds):
assert isinstance(expected_max_ops, int)
+ threshold = kwds.pop('threshold', 3)
+ if kwds:
+ raise TypeError, 'Unsupported keyword arguments: %s' % kwds.keys()
source = py.code.Source(source)
filepath = self.tmpdir.join('case%d.py' % self.counter)
logfilepath = filepath.new(ext='.log')
@@ -92,7 +95,7 @@
import sys
try: # make the file runnable by CPython
import pypyjit
- pypyjit.set_param(threshold=3)
+ pypyjit.set_param(threshold=%d)
except ImportError:
pass
@@ -102,7 +105,7 @@
print >> sys.stderr, 'got:', repr(result)
assert result == expected
assert type(result) is type(expected)
- """)
+ """ % threshold)
for testcase in testcases * 2:
print >> f, "check(%r, %r)" % testcase
print >> f, "print 'OK :-)'"
@@ -116,6 +119,8 @@
result = child_stdout.read()
child_stdout.close()
assert result
+ if result.strip().startswith('SKIP:'):
+ py.test.skip(result.strip())
assert result.splitlines()[-1].strip() == 'OK :-)'
self.parse_loops(logfilepath)
self.print_loops()
@@ -123,16 +128,18 @@
if self.total_ops > expected_max_ops:
assert 0, "too many operations: got %d, expected maximum %d" % (
self.total_ops, expected_max_ops)
+ return result
def parse_loops(self, opslogfile):
- from pypy.jit.metainterp.test.oparser import parse
+ from pypy.jit.tool.oparser import parse
from pypy.tool import logparser
assert opslogfile.check()
log = logparser.parse_log_file(str(opslogfile))
parts = logparser.extract_category(log, 'jit-log-opt-')
+ self.rawloops = [part for part in parts
+ if not from_entry_bridge(part, parts)]
# skip entry bridges, they can contain random things
- self.loops = [parse(part, no_namespace=True) for part in parts
- if not from_entry_bridge(part, parts)]
+ self.loops = [parse(part, no_namespace=True) for part in self.rawloops]
self.sliced_loops = [] # contains all bytecodes of all loops
self.total_ops = 0
for loop in self.loops:
@@ -156,12 +163,11 @@
return [ops for ops in self.sliced_loops if ops.bytecode == name]
def print_loops(self):
- for loop in self.loops:
+ for rawloop in self.rawloops:
print
print '@' * 79
print
- for op in loop.operations:
- print op
+ print rawloop.rstrip()
print
print '@' * 79
@@ -272,7 +278,7 @@
assert len(ops) == 2
assert not ops[0].get_opnames("call")
assert not ops[0].get_opnames("new")
- assert len(ops[0].get_opnames("guard")) <= 7
+ assert len(ops[0].get_opnames("guard")) <= 2
assert not ops[1] # second LOOKUP_METHOD folded away
ops = self.get_by_bytecode("CALL_METHOD")
@@ -283,12 +289,16 @@
else:
assert not bytecode.get_opnames("call")
assert not bytecode.get_opnames("new")
- assert len(bytecode.get_opnames("guard")) <= 9
+ assert len(bytecode.get_opnames("guard")) <= 6
assert len(ops[1]) < len(ops[0])
ops = self.get_by_bytecode("LOAD_ATTR")
assert len(ops) == 2
- assert ops[0].get_opnames() == ["getfield_gc", "getarrayitem_gc",
+ # With mapdict, we get fast access to (so far) the 5 first
+ # attributes, which means it is done with only the following
+ # operations. (For the other attributes there is additionally
+ # a getarrayitem_gc.)
+ assert ops[0].get_opnames() == ["getfield_gc",
"guard_nonnull_class"]
assert not ops[1] # second LOAD_ATTR folded away
@@ -317,8 +327,8 @@
assert len(ops) == 2
assert not ops[0].get_opnames("call")
assert not ops[0].get_opnames("new")
- assert len(ops[0].get_opnames("guard")) <= 7
- assert len(ops[0].get_opnames("getfield")) < 6
+ assert len(ops[0].get_opnames("guard")) <= 2
+ assert len(ops[0].get_opnames("getfield")) <= 4
assert not ops[1] # second LOOKUP_METHOD folded away
def test_default_and_kw(self):
@@ -382,7 +392,7 @@
a.x = 2
i = i + a.x
return i
- ''', 67,
+ ''', 69,
([20], 20),
([31], 32))
@@ -390,7 +400,7 @@
self.get_by_bytecode("CALL_FUNCTION"))
assert not callA.get_opnames("call")
assert not callA.get_opnames("new")
- assert len(callA.get_opnames("guard")) <= 8
+ assert len(callA.get_opnames("guard")) <= 2
assert not callisinstance1.get_opnames("call")
assert not callisinstance1.get_opnames("new")
assert len(callisinstance1.get_opnames("guard")) <= 2
@@ -554,17 +564,13 @@
def test_blockstack_virtualizable(self):
self.run_source('''
- def g(k):
- s = 0
- for i in range(k, k+2):
- s += 1
- return s
+ from pypyjit import residual_call
def main():
i = 0
while i < 100:
try:
- g(i)
+ residual_call(len, [])
except:
pass
i += 1
@@ -606,16 +612,17 @@
# call that can raise is not exchanged into getarrayitem_gc
def test_overflow_checking(self):
+ startvalue = sys.maxint - 2147483647
self.run_source('''
def main():
def f(a,b):
if a < 0: return -1
return a-b
- total = 0
+ total = %d
for i in range(100000):
total += f(i, 5)
return total
- ''', 170, ([], 4999450000L))
+ ''' % startvalue, 170, ([], startvalue + 4999450000L))
def test_boolrewrite_invers(self):
for a, b, res, ops in (('2000', '2000', 20001000, 51),
@@ -742,6 +749,8 @@
'''%(op1, float(a)/4.0, float(b)/4.0, op2), 109, ([], res))
def test_boolrewrite_ptr(self):
+ # XXX this test is way too imprecise in what it is actually testing
+ # it should count the number of guards instead
compares = ('a == b', 'b == a', 'a != b', 'b != a', 'a == c', 'c != b')
for e1 in compares:
for e2 in compares:
@@ -765,7 +774,7 @@
print
print 'Test:', e1, e2, n, res
self.run_source('''
- class tst:
+ class tst(object):
pass
def main():
a = tst()
@@ -781,24 +790,6 @@
return sa
'''%(e1, e2), n, ([], res))
- def test_boolrewrite_ptr_single(self):
- self.run_source('''
- class tst:
- pass
- def main():
- a = tst()
- b = tst()
- c = tst()
- sa = 0
- for i in range(1000):
- if a == b: sa += 1
- else: sa += 2
- if a != b: sa += 10000
- else: sa += 20000
- if i > 750: a = b
- return sa
- ''', 215, ([], 12481752))
-
def test_array_sum(self):
for tc, maxops in zip('bhilBHILfd', (38,) * 6 + (40, 40, 41, 38)):
res = 19352859
@@ -847,7 +838,12 @@
''', 65, ([], 122880))
def test_array_intimg(self):
- for tc, maxops in zip('ilILd', (67, 67, 69, 69, 61)):
+ # XXX this test is way too imprecise in what it is actually testing
+ # it should count the number of guards instead
+ for tc, maxops in zip('ilILd', (67, 67, 70, 70, 61)):
+ print
+ print '='*65
+ print '='*20, 'running test for tc=%r' % (tc,), '='*20
res = 73574560
if tc in 'IL':
res = long(res)
@@ -1130,6 +1126,44 @@
return sa
''', 88, ([], 1997001))
+ def test__ffi_call(self):
+ from pypy.rlib.test.test_libffi import get_libm_name
+ libm_name = get_libm_name(sys.platform)
+ out = self.run_source('''
+ def main():
+ try:
+ from _ffi import CDLL, types
+ except ImportError:
+ sys.stdout.write('SKIP: cannot import _ffi')
+ return 0
+
+ libm = CDLL('%(libm_name)s')
+ pow = libm.getfunc('pow', [types.double, types.double],
+ types.double)
+ print pow.getaddr()
+ i = 0
+ res = 0
+ while i < 2000:
+ res += pow(2, 3)
+ i += 1
+ return res
+ ''' % locals(),
+ 76, ([], 8.0*2000), threshold=1000)
+ pow_addr = int(out.splitlines()[0])
+ ops = self.get_by_bytecode('CALL_FUNCTION')
+ assert len(ops) == 2 # we get two loops, because of specialization
+ call_function = ops[0]
+ last_ops = [op.getopname() for op in call_function[-5:]]
+ assert last_ops == ['force_token',
+ 'setfield_gc',
+ 'call_may_force',
+ 'guard_not_forced',
+ 'guard_no_exception']
+ call = call_function[-3]
+ assert call.getarg(0).value == pow_addr
+ assert call.getarg(1).value == 2.0
+ assert call.getarg(2).value == 3.0
+
# test_circular
class AppTestJIT(PyPyCJITTests):
@@ -1155,6 +1189,17 @@
cls.pypy_c = option.pypy_c
+def test_interface_residual_call():
+ space = gettestobjspace(usemodules=['pypyjit'])
+ space.appexec([], """():
+ import pypyjit
+ def f(*args, **kwds):
+ return (args, kwds)
+ res = pypyjit.residual_call(f, 4, x=6)
+ assert res == ((4,), {'x': 6})
+ """)
+
+
def has_info(pypy_c, option):
g = os.popen('"%s" --info' % pypy_c, 'r')
lines = g.readlines()
Modified: pypy/branch/jit-unroll-loops/pypy/module/signal/interp_signal.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/signal/interp_signal.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/signal/interp_signal.py Sat Oct 30 16:41:17 2010
@@ -1,7 +1,6 @@
from pypy.interpreter.error import OperationError
from pypy.interpreter.baseobjspace import W_Root, ObjSpace
from pypy.interpreter.executioncontext import AsyncAction, AbstractActionFlag
-from pypy.rlib.rarithmetic import LONG_BIT, intmask
import signal as cpy_signal
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.translator.tool.cbuild import ExternalCompilationInfo
@@ -64,8 +63,8 @@
class CheckSignalAction(AsyncAction):
"""An action that is automatically invoked when a signal is received."""
- # The C-level signal handler sets the highest bit of pypysig_occurred:
- bitmask = intmask(1 << (LONG_BIT-1))
+ # The C-level signal handler sets the bit 30 of pypysig_occurred:
+ bitmask = 1 << 30
def __init__(self, space):
AsyncAction.__init__(self, space)
Modified: pypy/branch/jit-unroll-loops/pypy/module/sys/__init__.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/sys/__init__.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/sys/__init__.py Sat Oct 30 16:41:17 2010
@@ -7,13 +7,15 @@
"""Sys Builtin Module. """
def __init__(self, space, w_name):
"""NOT_RPYTHON""" # because parent __init__ isn't
+ if space.config.translating:
+ del self.__class__.interpleveldefs['pypy_getudir']
super(Module, self).__init__(space, w_name)
self.checkinterval = 100
self.recursionlimit = 100
self.w_default_encoder = None
self.defaultencoding = "ascii"
self.filesystemencoding = None
-
+
interpleveldefs = {
'__name__' : '(space.wrap("sys"))',
'__doc__' : '(space.wrap("PyPy sys module"))',
@@ -37,7 +39,7 @@
'argv' : 'state.get(space).w_argv',
'warnoptions' : 'state.get(space).w_warnoptions',
'builtin_module_names' : 'state.w_None',
- 'pypy_getudir' : 'state.pypy_getudir',
+ 'pypy_getudir' : 'state.pypy_getudir', # not translated
'pypy_initial_path' : 'state.pypy_initial_path',
'_getframe' : 'vm._getframe',
Modified: pypy/branch/jit-unroll-loops/pypy/module/sys/state.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/sys/state.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/sys/state.py Sat Oct 30 16:41:17 2010
@@ -95,15 +95,8 @@
def getio(space):
return space.fromcache(IOState)
-def _pypy_getudir(space):
- """NOT_RPYTHON"""
+def pypy_getudir(space):
+ """NOT_RPYTHON
+ (should be removed from interpleveldefs before translation)"""
from pypy.tool.udir import udir
return space.wrap(str(udir))
-_pypy_getudir._annspecialcase_ = "override:ignore"
-
-# we need the indirection because this function will live in a dictionary with other
-# RPYTHON functions and share call sites with them. Better it not be a special-case
-# directly.
-def pypy_getudir(space):
- return _pypy_getudir(space)
-
Modified: pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c Sat Oct 30 16:41:17 2010
@@ -533,3 +533,7 @@
return inp;
}
+int my_unused_function(void)
+{
+ return 42;
+}
Modified: pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py Sat Oct 30 16:41:17 2010
@@ -148,3 +148,7 @@
# but it segfaults for some reason.
if sys.platform == 'win32':
assert f() == 0x12345678
+
+ def test_restype(self):
+ foo = lib.my_unused_function
+ assert foo.restype is c_int # by default
Modified: pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py Sat Oct 30 16:41:17 2010
@@ -29,6 +29,7 @@
class RECT(Structure):
_fields_ = [("left", c_int), ("top", c_int),
("right", c_int), ("bottom", c_int)]
+
class TestFunctions(BaseCTypesTestChecker):
def test_mro(self):
@@ -392,6 +393,18 @@
result = f("abcd", ord("b"))
assert result == "bcd"
+ def test_caching_bug_1(self):
+ # the same test as test_call_some_args, with two extra lines
+ # in the middle that trigger caching in f._ptr, which then
+ # makes the last two lines fail
+ f = dll.my_strchr
+ f.argtypes = [c_char_p, c_int]
+ f.restype = c_char_p
+ result = f("abcd", ord("b"))
+ assert result == "bcd"
+ result = f("abcd", ord("b"), 42)
+ assert result == "bcd"
+
def test_sf1651235(self):
py.test.skip("we are less strict in checking callback parameters")
# see http://www.python.org/sf/1651235
Modified: pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py Sat Oct 30 16:41:17 2010
@@ -10,7 +10,7 @@
"""
def test_array_of_pointers(self):
# tests array item assignements & pointer.contents = ...
- A = POINTER(c_int) * 24
+ A = POINTER(c_long) * 24
a = A()
l = c_long(2)
p = pointer(l)
Modified: pypy/branch/jit-unroll-loops/pypy/module/thread/ll_thread.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/module/thread/ll_thread.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/module/thread/ll_thread.py Sat Oct 30 16:41:17 2010
@@ -111,7 +111,7 @@
c_thread_releaselock(self._lock)
def __del__(self):
- lltype.free(self._lock, flavor='raw')
+ lltype.free(self._lock, flavor='raw', track_allocation=False)
# ____________________________________________________________
#
@@ -128,10 +128,13 @@
null_ll_lock = lltype.nullptr(TLOCKP.TO)
def allocate_ll_lock():
- ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw')
+ # track_allocation=False here; be careful to lltype.free() it. The
+ # reason it is set to False is that we get it from all app-level
+ # lock objects, as well as from the GIL, which exists at shutdown.
+ ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw', track_allocation=False)
res = c_thread_lock_init(ll_lock)
if res == -1:
- lltype.free(ll_lock, flavor='raw')
+ lltype.free(ll_lock, flavor='raw', track_allocation=False)
raise error("out of resources")
return ll_lock
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/descroperation.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/descroperation.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/descroperation.py Sat Oct 30 16:41:17 2010
@@ -51,7 +51,7 @@
return space.is_w(w_typ1, w_typ2)
-class Object:
+class Object(object):
def descr__getattribute__(space, w_obj, w_name):
name = space.str_w(w_name)
w_descr = space.lookup(w_obj, name)
@@ -64,9 +64,7 @@
w_type = space.type(w_obj)
return space.get_and_call_function(w_get, w_descr, w_obj,
w_type)
- w_value = w_obj.getdictvalue_attr_is_in_class(space, name)
- else:
- w_value = w_obj.getdictvalue(space, name)
+ w_value = w_obj.getdictvalue(space, name)
if w_value is not None:
return w_value
if w_descr is not None:
@@ -76,13 +74,11 @@
def descr__setattr__(space, w_obj, w_name, w_value):
name = space.str_w(w_name)
w_descr = space.lookup(w_obj, name)
- shadows_type = False
if w_descr is not None:
if space.is_data_descr(w_descr):
space.set(w_descr, w_obj, w_value)
return
- shadows_type = True
- if w_obj.setdictvalue(space, name, w_value, shadows_type):
+ if w_obj.setdictvalue(space, name, w_value):
return
raiseattrerror(space, w_obj, name, w_descr)
@@ -100,7 +96,7 @@
def descr__init__(space, w_obj, __args__):
pass
-class DescrOperation:
+class DescrOperation(object):
_mixin_ = True
def is_data_descr(space, w_obj):
@@ -545,7 +541,7 @@
# what is the maximum value slices can get on CPython?
# we need to stick to that value, because fake.py etc.
-class Temp:
+class Temp(object):
def __getslice__(self, i, j):
return j
slice_max = Temp()[:]
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/flow/flowcontext.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/flow/flowcontext.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/flow/flowcontext.py Sat Oct 30 16:41:17 2010
@@ -460,7 +460,22 @@
else:
v = self.peekvalue(oparg - 1)
self.space.call_method(v, 'append', w)
-
+
+ # `with` statement
+
+ def SETUP_WITH(self, offsettoend, next_instr):
+ # A simpler version than the 'real' 2.7 one:
+ # directly call manager.__enter__(), don't use special lookup functions
+ # which don't make sense on the RPython type system.
+ from pypy.interpreter.pyopcode import WithBlock
+ w_manager = self.peekvalue()
+ w_exit = self.space.getattr(w_manager, self.space.wrap("__exit__"))
+ self.settopvalue(w_exit)
+ w_result = self.space.call_method(w_manager, "__enter__")
+ block = WithBlock(self, next_instr + offsettoend)
+ self.append_block(block)
+ self.pushvalue(w_result)
+
# XXX Unimplemented 2.7 opcodes ----------------
# Set literals, set comprehensions
@@ -476,12 +491,6 @@
def MAP_ADD(self, oparg, next_instr):
raise NotImplementedError("MAP_ADD")
- # `with` statement
-
- def SETUP_WITH(self, oparg, next_instr):
- raise NotImplementedError("SETUP_WITH")
-
-
def make_arguments(self, nargs):
return ArgumentsForTranslation(self.space, self.peekvalues(nargs))
def argument_factory(self, *args):
@@ -493,3 +502,12 @@
raise operr
return pyframe.PyFrame.handle_operation_error(self, ec, operr,
*args, **kwds)
+
+ def call_contextmanager_exit_function(self, w_func, w_typ, w_val, w_tb):
+ if w_typ is not self.space.w_None:
+ # The annotator won't allow to merge exception types with None.
+ # Replace it with an object which will break translation when used
+ # (except maybe with 'exc_typ is None')
+ w_typ = self.space.wrap(self.space)
+ return self.space.call_function(w_func, w_typ, w_val, w_tb)
+
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/flow/test/test_objspace.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/flow/test/test_objspace.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/flow/test/test_objspace.py Sat Oct 30 16:41:17 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
import new
import py
from pypy.objspace.flow.model import Constant, Block, Link, Variable, traverse
@@ -828,6 +829,25 @@
simplify_graph(graph)
assert self.all_operations(graph) == {'getitem': 1}
+ def test_context_manager(self):
+ def f(c, x):
+ with x:
+ pass
+ graph = self.codetest(f)
+ # 2 method calls: x.__enter__() and x.__exit__(None, None, None)
+ assert self.all_operations(graph) == {'getattr': 2,
+ 'simple_call': 2}
+ #
+ def g(): pass
+ def f(c, x):
+ with x:
+ g()
+ graph = self.codetest(f)
+ assert self.all_operations(graph) == {
+ 'getattr': 2, # __enter__ and __exit__
+ 'simple_call': 4, # __enter__, g and 2 possible calls to __exit__
+ 'is_true': 1} # check the result of __exit__()
+
def monkey_patch_code(self, code, stacksize, flags, codestring, names, varnames):
c = code
return new.code(c.co_argcount, c.co_nlocals, stacksize, flags,
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/callmethod.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/callmethod.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/callmethod.py Sat Oct 30 16:41:17 2010
@@ -44,7 +44,7 @@
else:
typ = type(w_descr)
if typ is function.Function or typ is function.FunctionWithFixedCode:
- w_value = w_obj.getdictvalue_attr_is_in_class(space, name)
+ w_value = w_obj.getdictvalue(space, name)
if w_value is None:
# fast method path: a function object in the class,
# nothing in the instance
@@ -103,7 +103,7 @@
w_descr = space.lookup(w_obj, methname)
typ = type(w_descr)
if typ is function.Function or typ is function.FunctionWithFixedCode:
- w_value = w_obj.getdictvalue_attr_is_in_class(space, methname)
+ w_value = w_obj.getdictvalue(space, methname)
if w_value is None:
# fast method path: a function object in the class,
# nothing in the instance
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/celldict.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/celldict.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/celldict.py Sat Oct 30 16:41:17 2010
@@ -45,9 +45,9 @@
if space.is_w(space.type(w_key), space.w_str):
self.impl_setitem_str(self.space.str_w(w_key), w_value)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
- def impl_setitem_str(self, name, w_value, shadows_type=True):
+ def impl_setitem_str(self, name, w_value):
self.getcell(name, True).w_value = w_value
def impl_delitem(self, w_key):
@@ -66,7 +66,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_length(self):
# inefficient, but do we care?
@@ -85,7 +85,7 @@
elif _is_sane_hash(space, w_lookup_type):
return None
else:
- return self._as_rdict().getitem(w_lookup)
+ return self._as_rdict().impl_fallback_getitem(w_lookup)
def impl_getitem_str(self, lookup):
res = self.getcell(lookup, False)
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/dictmultiobject.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/dictmultiobject.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/dictmultiobject.py Sat Oct 30 16:41:17 2010
@@ -49,14 +49,6 @@
elif space.config.objspace.std.withdictmeasurement:
assert w_type is None
return MeasuringDictImplementation(space)
- elif space.config.objspace.std.withsharingdict and instance:
- from pypy.objspace.std.sharingdict import SharedDictImplementation
- assert w_type is None
- return SharedDictImplementation(space)
- elif (space.config.objspace.std.withshadowtracking and instance and
- classofinstance is not None):
- assert w_type is None
- return ShadowDetectingDictImplementation(space, classofinstance)
elif instance or strdict or module:
assert w_type is None
return StrDictImplementation(space)
@@ -102,17 +94,17 @@
else:
return None
- # _________________________________________________________________
+ # _________________________________________________________________
# implementation methods
def impl_getitem(self, w_key):
#return w_value or None
raise NotImplementedError("abstract base class")
- def impl_getitem_str(self, w_key):
+ def impl_getitem_str(self, key):
#return w_value or None
raise NotImplementedError("abstract base class")
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
raise NotImplementedError("abstract base class")
def impl_setitem(self, w_key, w_value):
@@ -120,7 +112,7 @@
def impl_delitem(self, w_key):
raise NotImplementedError("abstract base class")
-
+
def impl_length(self):
raise NotImplementedError("abstract base class")
@@ -165,12 +157,15 @@
key = OPTIMIZED_BUILTINS[i]
return self.impl_getitem_str(key)
- # this method will only be seen whan a certain config option is used
- def impl_shadows_anything(self):
- return True
-
- def impl_set_shadows_anything(self):
- pass
+ def impl_popitem(self):
+ # default implementation
+ space = self.space
+ iterator = self.impl_iter()
+ w_key, w_value = iterator.next()
+ if w_key is None:
+ raise KeyError
+ self.impl_delitem(w_key)
+ return w_key, w_value
# _________________________________________________________________
# fallback implementation methods
@@ -178,7 +173,7 @@
def impl_fallback_setitem(self, w_key, w_value):
self.r_dict_content[w_key] = w_value
- def impl_fallback_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_fallback_setitem_str(self, key, w_value):
return self.impl_fallback_setitem(self.space.wrap(key), w_value)
def impl_fallback_delitem(self, w_key):
@@ -211,18 +206,15 @@
key = OPTIMIZED_BUILTINS[i]
return self.impl_fallback_getitem_str(key)
- def impl_fallback_shadows_anything(self):
- return True
-
- def impl_fallback_set_shadows_anything(self):
- pass
+ def impl_fallback_popitem(self):
+ return self.r_dict_content.popitem()
implementation_methods = [
("getitem", 1),
("getitem_str", 1),
("length", 0),
- ("setitem_str", 3),
+ ("setitem_str", 2),
("setitem", 2),
("delitem", 1),
("iter", 0),
@@ -231,8 +223,7 @@
("keys", 0),
("clear", 0),
("get_builtin_indexed", 1),
- ("shadows_anything", 0),
- ("set_shadows_anything", 0),
+ ("popitem", 0),
]
@@ -310,9 +301,9 @@
if space.is_w(space.type(w_key), space.w_str):
self.impl_setitem_str(self.space.str_w(w_key), w_value)
else:
- self._as_rdict().setitem(w_key, w_value)
+ self._as_rdict().impl_fallback_setitem(w_key, w_value)
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
self.content[key] = w_value
def impl_delitem(self, w_key):
@@ -324,7 +315,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_length(self):
return len(self.content)
@@ -344,7 +335,7 @@
elif _is_sane_hash(space, w_lookup_type):
return None
else:
- return self._as_rdict().getitem(w_key)
+ return self._as_rdict().impl_fallback_getitem(w_key)
def impl_iter(self):
return StrIteratorImplementation(self.space, self)
@@ -388,47 +379,12 @@
return None, None
-class ShadowDetectingDictImplementation(StrDictImplementation):
- def __init__(self, space, w_type):
- StrDictImplementation.__init__(self, space)
- self.w_type = w_type
- self.original_version_tag = w_type.version_tag()
- if self.original_version_tag is None:
- self._shadows_anything = True
- else:
- self._shadows_anything = False
-
- def impl_setitem_str(self, key, w_value, shadows_type=True):
- if shadows_type:
- self._shadows_anything = True
- StrDictImplementation.impl_setitem_str(
- self, key, w_value, shadows_type)
-
- def impl_setitem(self, w_key, w_value):
- space = self.space
- if space.is_w(space.type(w_key), space.w_str):
- if not self._shadows_anything:
- w_obj = self.w_type.lookup(space.str_w(w_key))
- if w_obj is not None:
- self._shadows_anything = True
- StrDictImplementation.impl_setitem_str(
- self, self.space.str_w(w_key), w_value, False)
- else:
- self._as_rdict().setitem(w_key, w_value)
-
- def impl_shadows_anything(self):
- return (self._shadows_anything or
- self.w_type.version_tag() is not self.original_version_tag)
-
- def impl_set_shadows_anything(self):
- self._shadows_anything = True
-
class WaryDictImplementation(StrDictImplementation):
def __init__(self, space):
StrDictImplementation.__init__(self, space)
self.shadowed = [None] * len(BUILTIN_TO_INDEX)
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
i = BUILTIN_TO_INDEX.get(key, -1)
if i != -1:
self.shadowed[i] = w_value
@@ -446,7 +402,7 @@
elif _is_sane_hash(space, w_key_type):
raise KeyError
else:
- self._as_rdict().delitem(w_key)
+ self._as_rdict().impl_fallback_delitem(w_key)
def impl_get_builtin_indexed(self, i):
return self.shadowed[i]
@@ -558,7 +514,7 @@
self.info.writes += 1
self.content[w_key] = w_value
self.info.maxcontents = max(self.info.maxcontents, len(self.content))
- def impl_setitem_str(self, key, w_value, shadows_type=True):
+ def impl_setitem_str(self, key, w_value):
self.info.setitem_strs += 1
self.impl_setitem(self.space.wrap(key), w_value)
def impl_delitem(self, w_key):
@@ -641,30 +597,50 @@
init_signature = Signature(['seq_or_map'], None, 'kwargs')
init_defaults = [None]
-def init__DictMulti(space, w_dict, __args__):
- w_src, w_kwds = __args__.parse_obj(
- None, 'dict',
- init_signature, # signature
- init_defaults) # default argument
- if w_src is None:
- pass
- elif space.findattr(w_src, space.wrap("keys")) is None:
- list_of_w_pairs = space.listview(w_src)
- for w_pair in list_of_w_pairs:
+def update1(space, w_dict, w_data):
+ if space.findattr(w_data, space.wrap("keys")) is None:
+ # no 'keys' method, so we assume it is a sequence of pairs
+ for w_pair in space.listview(w_data):
pair = space.fixedview(w_pair)
- if len(pair)!=2:
+ if len(pair) != 2:
raise OperationError(space.w_ValueError,
- space.wrap("dict() takes a sequence of pairs"))
- w_k, w_v = pair
- w_dict.setitem(w_k, w_v)
+ space.wrap("sequence of pairs expected"))
+ w_key, w_value = pair
+ w_dict.setitem(w_key, w_value)
else:
- if space.is_true(w_src):
- from pypy.objspace.std.dicttype import update1
- update1(space, w_dict, w_src)
+ if isinstance(w_data, W_DictMultiObject): # optimization case only
+ update1_dict_dict(space, w_dict, w_data)
+ else:
+ # general case -- "for k in o.keys(): dict.__setitem__(d, k, o[k])"
+ w_keys = space.call_method(w_data, "keys")
+ for w_key in space.listview(w_keys):
+ w_value = space.getitem(w_data, w_key)
+ w_dict.setitem(w_key, w_value)
+
+def update1_dict_dict(space, w_dict, w_data):
+ iterator = w_data.iter()
+ while 1:
+ w_key, w_value = iterator.next()
+ if w_key is None:
+ break
+ w_dict.setitem(w_key, w_value)
+
+def init_or_update(space, w_dict, __args__, funcname):
+ w_src, w_kwds = __args__.parse_obj(
+ None, funcname,
+ init_signature, # signature
+ init_defaults) # default argument
+ if w_src is not None:
+ update1(space, w_dict, w_src)
if space.is_true(w_kwds):
- from pypy.objspace.std.dicttype import update1
update1(space, w_dict, w_kwds)
+def init__DictMulti(space, w_dict, __args__):
+ init_or_update(space, w_dict, __args__, 'dict')
+
+def dict_update__DictMulti(space, w_dict, __args__):
+ init_or_update(space, w_dict, __args__, 'dict.update')
+
def getitem__DictMulti_ANY(space, w_dict, w_key):
w_value = w_dict.getitem(w_key)
if w_value is not None:
@@ -758,9 +734,8 @@
return w_res
def dict_copy__DictMulti(space, w_self):
- from pypy.objspace.std.dicttype import update1
w_new = W_DictMultiObject.allocate_and_init_instance(space)
- update1(space, w_new, w_self)
+ update1_dict_dict(space, w_new, w_self)
return w_new
def dict_items__DictMulti(space, w_self):
@@ -791,6 +766,15 @@
else:
return w_default
+def dict_setdefault__DictMulti_ANY_ANY(space, w_dict, w_key, w_default):
+ # XXX should be more efficient, with only one dict lookup
+ w_value = w_dict.getitem(w_key)
+ if w_value is not None:
+ return w_value
+ else:
+ w_dict.setitem(w_key, w_default)
+ return w_default
+
def dict_pop__DictMulti_ANY(space, w_dict, w_key, w_defaults):
defaults = space.listview(w_defaults)
len_defaults = len(defaults)
@@ -808,6 +792,14 @@
w_dict.delitem(w_key)
return w_item
+def dict_popitem__DictMulti(space, w_dict):
+ try:
+ w_key, w_value = w_dict.popitem()
+ except KeyError:
+ raise OperationError(space.w_KeyError,
+ space.wrap("popitem(): dictionary is empty"))
+ return space.newtuple([w_key, w_value])
+
app = gateway.applevel('''
def dictrepr(currently_in_repr, d):
# Now we only handle one implementation of dicts, this one.
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/dicttype.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/dicttype.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/dicttype.py Sat Oct 30 16:41:17 2010
@@ -53,106 +53,21 @@
# This can return when multimethods have been fixed
#dict_str = StdObjSpace.str
-# default application-level implementations for some operations
-# most of these (notably not popitem and update*) are overwritten
-# in dictmultiobject
-# gateway is imported in the stdtypedef module
-app = gateway.applevel('''
-
- # in the following functions we use dict.__setitem__ instead of
- # d[k]=... because when a subclass of dict override __setitem__,
- # CPython does not call it when doing builtin operations. The
- # same for other operations.
-
- def update1(d, o):
- if hasattr(o, 'keys'):
- for k in o.keys():
- dict.__setitem__(d, k, o[k])
- else:
- for k,v in o:
- dict.__setitem__(d, k, v)
-
- def update(d, *args, **kwargs):
- len_args = len(args)
- if len_args == 1:
- update1(d, args[0])
- elif len_args > 1:
- raise TypeError("update takes at most 1 (non-keyword) argument")
- if kwargs:
- update1(d, kwargs)
-
- def popitem(d):
- for k in dict.iterkeys(d):
- break
- else:
- raise KeyError("popitem(): dictionary is empty")
- v = dict.__getitem__(d, k)
- dict.__delitem__(d, k)
- return k, v
-
- def get(d, k, v=None):
- if k in d:
- return dict.__getitem__(d, k)
- else:
- return v
-
- def setdefault(d, k, v=None):
- if k in d:
- return dict.__getitem__(d, k)
- else:
- dict.__setitem__(d, k, v)
- return v
-
- def pop(d, k, defaults): # XXX defaults is actually *defaults
- if len(defaults) > 1:
- raise TypeError, "pop expected at most 2 arguments, got %d" % (
- 1 + len(defaults))
- try:
- v = dict.__getitem__(d, k)
- dict.__delitem__(d, k)
- except KeyError, e:
- if defaults:
- return defaults[0]
- else:
- raise e
- return v
-
- def iteritems(d):
- return iter(dict.items(d))
-
- def iterkeys(d):
- return iter(dict.keys(d))
-
- def itervalues(d):
- return iter(dict.values(d))
-''', filename=__file__)
-
-dict_update__ANY = app.interphook("update")
-dict_popitem__ANY = app.interphook("popitem")
-dict_get__ANY_ANY_ANY = app.interphook("get")
-dict_setdefault__ANY_ANY_ANY = app.interphook("setdefault")
-dict_pop__ANY_ANY = app.interphook("pop")
-dict_iteritems__ANY = app.interphook("iteritems")
-dict_iterkeys__ANY = app.interphook("iterkeys")
-dict_itervalues__ANY = app.interphook("itervalues")
-update1 = app.interphook("update1")
-
register_all(vars(), globals())
@gateway.unwrap_spec(ObjSpace, W_Root, W_Root, W_Root)
def descr_fromkeys(space, w_type, w_keys, w_fill=None):
+ from pypy.objspace.std.dictmultiobject import W_DictMultiObject
if w_fill is None:
w_fill = space.w_None
- w_dict = space.call_function(w_type)
- w_iter = space.iter(w_keys)
- while True:
- try:
- w_key = space.next(w_iter)
- except OperationError, e:
- if not e.match(space, space.w_StopIteration):
- raise
- break
- space.setitem(w_dict, w_key, w_fill)
+ if w_type is space.w_dict:
+ w_dict = W_DictMultiObject.allocate_and_init_instance(space, w_type)
+ for w_key in space.listview(w_keys):
+ w_dict.setitem(w_key, w_fill)
+ else:
+ w_dict = space.call_function(w_type)
+ for w_key in space.listview(w_keys):
+ space.setitem(w_dict, w_key, w_fill)
return w_dict
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/fake.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/fake.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/fake.py Sat Oct 30 16:41:17 2010
@@ -21,7 +21,6 @@
#debug_print("faking obj %s" % x)
ft = fake_type(type(x))
return ft(space, x)
-fake_object._annspecialcase_ = "override:fake_object"
import sys
@@ -47,7 +46,6 @@
w_exc = space.wrap(exc)
w_value = space.wrap(value)
raise OperationError, OperationError(w_exc, w_value), tb
-wrap_exception._annspecialcase_ = "override:ignore"
def fake_type(cpy_type):
assert type(cpy_type) is type
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/frozensettype.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/frozensettype.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/frozensettype.py Sat Oct 30 16:41:17 2010
@@ -39,13 +39,13 @@
def descr__frozenset__new__(space, w_frozensettype,
w_iterable=gateway.NoneNotWrapped):
from pypy.objspace.std.setobject import W_FrozensetObject
- from pypy.objspace.std.setobject import _is_frozenset_exact
+ from pypy.objspace.std.setobject import make_setdata_from_w_iterable
if (space.is_w(w_frozensettype, space.w_frozenset) and
- _is_frozenset_exact(w_iterable)):
+ w_iterable is not None and type(w_iterable) is W_FrozensetObject):
return w_iterable
w_obj = space.allocate_instance(W_FrozensetObject, w_frozensettype)
- W_FrozensetObject.__init__(w_obj, space, None)
-
+ data = make_setdata_from_w_iterable(space, w_iterable)
+ W_FrozensetObject.__init__(w_obj, space, data)
return w_obj
frozenset_typedef = StdTypeDef("frozenset",
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/inttype.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/inttype.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/inttype.py Sat Oct 30 16:41:17 2010
@@ -1,7 +1,7 @@
from pypy.interpreter import gateway
from pypy.interpreter.error import OperationError
from pypy.objspace.std.stdtypedef import StdTypeDef
-from pypy.objspace.std.strutil import (string_to_int, string_to_w_long,
+from pypy.objspace.std.strutil import (string_to_int, string_to_bigint,
ParseStringError,
ParseStringOverflowError)
from pypy.rlib.rarithmetic import r_uint
@@ -45,10 +45,12 @@
def retry_to_w_long(space, parser, base=0):
parser.rewind()
try:
- return string_to_w_long(space, None, base=base, parser=parser)
+ bigint = string_to_bigint(None, base=base, parser=parser)
except ParseStringError, e:
raise OperationError(space.w_ValueError,
space.wrap(e.msg))
+ from pypy.objspace.std.longobject import W_LongObject
+ return W_LongObject(bigint)
def descr__new__(space, w_inttype, w_x=0, w_base=gateway.NoneNotWrapped):
from pypy.objspace.std.intobject import W_IntObject
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/listobject.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/listobject.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/listobject.py Sat Oct 30 16:41:17 2010
@@ -283,7 +283,7 @@
elif start >= 0:
del items[start:start+delta]
else:
- assert delta==0
+ assert delta==0 # start<0 is only possible with slicelength==0
elif len2 != slicelength: # No resize for extended slices
raise operationerrfmt(space.w_ValueError, "attempt to "
"assign sequence of size %d to extended slice of size %d",
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/longtype.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/longtype.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/longtype.py Sat Oct 30 16:41:17 2010
@@ -1,7 +1,7 @@
from pypy.interpreter.error import OperationError
from pypy.interpreter import gateway
from pypy.objspace.std.stdtypedef import StdTypeDef
-from pypy.objspace.std.strutil import string_to_w_long, ParseStringError
+from pypy.objspace.std.strutil import string_to_bigint, ParseStringError
def descr__new__(space, w_longtype, w_x=0, w_base=gateway.NoneNotWrapped):
from pypy.objspace.std.longobject import W_LongObject
@@ -9,10 +9,10 @@
if w_base is None:
# check for easy cases
if type(w_value) is W_LongObject:
- pass
+ bigint = w_value.num
elif space.is_true(space.isinstance(w_value, space.w_str)):
try:
- w_value = string_to_w_long(space, space.str_w(w_value))
+ bigint = string_to_bigint(space.str_w(w_value))
except ParseStringError, e:
raise OperationError(space.w_ValueError,
space.wrap(e.msg))
@@ -22,7 +22,7 @@
from pypy.objspace.std.ropeunicodeobject import unicode_to_decimal_w
else:
from pypy.objspace.std.unicodeobject import unicode_to_decimal_w
- w_value = string_to_w_long(space, unicode_to_decimal_w(space, w_value))
+ bigint = string_to_bigint(unicode_to_decimal_w(space, w_value))
except ParseStringError, e:
raise OperationError(space.w_ValueError,
space.wrap(e.msg))
@@ -35,10 +35,11 @@
if space.is_true(space.isinstance(w_obj, space.w_long)):
assert isinstance(w_obj, W_LongObject) # XXX this could fail!
# XXX find a way to do that even if w_obj is not a W_LongObject
- w_value = w_obj
+ bigint = w_obj.num
elif space.is_true(space.isinstance(w_obj, space.w_int)):
+ from pypy.rlib.rbigint import rbigint
intval = space.int_w(w_obj)
- w_value = W_LongObject.fromint(space, intval)
+ bigint = rbigint.fromint(intval)
else:
raise OperationError(space.w_ValueError,
space.wrap("value can't be converted to long"))
@@ -56,13 +57,13 @@
space.wrap("long() can't convert non-string "
"with explicit base"))
try:
- w_value = string_to_w_long(space, s, base)
+ bigint = string_to_bigint(s, base)
except ParseStringError, e:
raise OperationError(space.w_ValueError,
space.wrap(e.msg))
w_obj = space.allocate_instance(W_LongObject, w_longtype)
- W_LongObject.__init__(w_obj, w_value.num)
+ W_LongObject.__init__(w_obj, bigint)
return w_obj
# ____________________________________________________________
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/model.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/model.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/model.py Sat Oct 30 16:41:17 2010
@@ -18,6 +18,7 @@
"withsmallint" : ["smallintobject.W_SmallIntObject"],
"withstrslice" : ["strsliceobject.W_StringSliceObject"],
"withstrjoin" : ["strjoinobject.W_StringJoinObject"],
+ "withstrbuf" : ["strbufobject.W_StringBufferObject"],
"withrope" : ["ropeobject.W_RopeObject",
"ropeobject.W_RopeIterObject"],
"withropeunicode": ["ropeunicodeobject.W_RopeUnicodeObject",
@@ -75,6 +76,7 @@
from pypy.objspace.std import ropeunicodeobject
from pypy.objspace.std import strsliceobject
from pypy.objspace.std import strjoinobject
+ from pypy.objspace.std import strbufobject
from pypy.objspace.std import typeobject
from pypy.objspace.std import sliceobject
from pypy.objspace.std import longobject
@@ -222,6 +224,13 @@
(unicodeobject.W_UnicodeObject,
strjoinobject.delegate_join2unicode)
]
+ elif config.objspace.std.withstrbuf:
+ self.typeorder[strbufobject.W_StringBufferObject] += [
+ (stringobject.W_StringObject,
+ strbufobject.delegate_buf2str),
+ (unicodeobject.W_UnicodeObject,
+ strbufobject.delegate_buf2unicode)
+ ]
if config.objspace.std.withrangelist:
self.typeorder[rangeobject.W_RangeListObject] += [
(listobject.W_ListObject,
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/objspace.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/objspace.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/objspace.py Sat Oct 30 16:41:17 2010
@@ -23,6 +23,7 @@
from pypy.objspace.std.listobject import W_ListObject
from pypy.objspace.std.longobject import W_LongObject
from pypy.objspace.std.noneobject import W_NoneObject
+from pypy.objspace.std.objectobject import W_ObjectObject
from pypy.objspace.std.ropeobject import W_RopeObject
from pypy.objspace.std.iterobject import W_SeqIterObject
from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject
@@ -242,7 +243,6 @@
w_result = getattr(self, 'w_' + x.__name__)
return w_result
return None
- wrap_exception_cls._annspecialcase_ = "override:wrap_exception_cls"
def unwrap(self, w_obj):
if isinstance(w_obj, Wrappable):
@@ -318,9 +318,14 @@
w_subtype = w_type.check_user_subclass(w_subtype)
if cls.typedef.applevel_subclasses_base is not None:
cls = cls.typedef.applevel_subclasses_base
- subcls = get_unique_interplevel_subclass(
- self.config, cls, w_subtype.hasdict, w_subtype.nslots != 0,
- w_subtype.needsdel, w_subtype.weakrefable)
+ if (self.config.objspace.std.withmapdict and cls is W_ObjectObject
+ and not w_subtype.needsdel):
+ from pypy.objspace.std.mapdict import get_subclass_of_correct_size
+ subcls = get_subclass_of_correct_size(self, cls, w_subtype)
+ else:
+ subcls = get_unique_interplevel_subclass(
+ self.config, cls, w_subtype.hasdict, w_subtype.nslots != 0,
+ w_subtype.needsdel, w_subtype.weakrefable)
instance = instantiate(subcls)
assert isinstance(instance, cls)
instance.user_setup(self, w_subtype)
@@ -432,7 +437,7 @@
if is_data:
w_get = self.lookup(w_descr, "__get__")
if w_get is None:
- w_value = w_obj.getdictvalue_attr_is_in_class(self, name)
+ w_value = w_obj.getdictvalue(self, name)
if w_value is not None:
return w_value
if not is_data:
@@ -484,14 +489,12 @@
return w_obj.getitem(w_key)
return ObjSpace.finditem(self, w_obj, w_key)
- def setitem_str(self, w_obj, key, w_value, shadows_type=True):
+ def setitem_str(self, w_obj, key, w_value):
""" Same as setitem, but takes string instead of any wrapped object
-
- XXX what shadows_type means???
"""
if (isinstance(w_obj, W_DictMultiObject) and
not w_obj.user_overridden_class):
- w_obj.setitem_str(key, w_value, shadows_type)
+ w_obj.setitem_str(key, w_value)
else:
self.setitem(w_obj, self.wrap(key), w_value)
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/proxyobject.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/proxyobject.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/proxyobject.py Sat Oct 30 16:41:17 2010
@@ -43,7 +43,7 @@
raise
return None
- def setdictvalue(self, space, attr, w_value, shadows_type=True):
+ def setdictvalue(self, space, attr, w_value):
try:
space.call_function(self.w_controller, space.wrap('__setattr__'),
space.wrap(attr), w_value)
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/setobject.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/setobject.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/setobject.py Sat Oct 30 16:41:17 2010
@@ -21,11 +21,10 @@
return False
- def __init__(w_self, space, setdata=None):
- if setdata is None:
- w_self.setdata = r_dict(space.eq_w, space.hash_w)
- else:
- w_self.setdata = setdata.copy()
+ def __init__(w_self, space, setdata):
+ """Initialize the set by taking ownership of 'setdata'."""
+ assert setdata is not None
+ w_self.setdata = setdata
def __repr__(w_self):
"""representation for debugging purposes"""
@@ -33,6 +32,7 @@
return "<%s(%s)>" % (w_self.__class__.__name__, ', '.join(reprlist))
def _newobj(w_self, space, rdict_w=None):
+ """Make a new set or frozenset by taking ownership of 'rdict_w'."""
#return space.call(space.type(w_self),W_SetIterObject(rdict_w))
objtype = type(w_self)
if objtype is W_SetObject:
@@ -55,10 +55,7 @@
class W_FrozensetObject(W_BaseSetObject):
from pypy.objspace.std.frozensettype import frozenset_typedef as typedef
-
- def __init__(w_self, space, setdata):
- W_BaseSetObject.__init__(w_self, space, setdata)
- w_self.hash = -1
+ hash = 0
registerimplementation(W_BaseSetObject)
registerimplementation(W_SetObject)
@@ -109,8 +106,14 @@
# some helper functions
+def newset(space):
+ return r_dict(space.eq_w, space.hash_w)
+
def make_setdata_from_w_iterable(space, w_iterable=None):
- data = r_dict(space.eq_w, space.hash_w)
+ """Return a new r_dict with the content of w_iterable."""
+ if isinstance(w_iterable, W_BaseSetObject):
+ return w_iterable.setdata.copy()
+ data = newset(space)
if w_iterable is not None:
for w_item in space.listview(w_iterable):
data[w_item] = None
@@ -119,7 +122,7 @@
def _initialize_set(space, w_obj, w_iterable=None):
w_obj.setdata.clear()
if w_iterable is not None:
- w_obj.setdata.update(make_setdata_from_w_iterable(space, w_iterable))
+ w_obj.setdata = make_setdata_from_w_iterable(space, w_iterable)
def _convert_set_to_frozenset(space, w_obj):
if space.is_true(space.isinstance(w_obj, space.w_set)):
@@ -130,12 +133,6 @@
# helper functions for set operation on dicts
-def _is_frozenset_exact(w_obj):
- if (w_obj is not None) and (type(w_obj) is W_FrozensetObject):
- return True
- else:
- return False
-
def _is_eq(ld, rd):
if len(ld) != len(rd):
return False
@@ -144,66 +141,41 @@
return False
return True
-def _union_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
- else:
- ld = ldict.copy()
- ld.update(rdict)
- return ld, rdict
-
-def _difference_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
- else:
- ld = ldict.copy()
- del_list_w = []
+def _difference_dict(space, ld, rd):
+ result = newset(space)
for w_key in ld:
- if w_key in rdict:
- del_list_w.append(w_key)
- for w_key in del_list_w:
- del ld[w_key]
-
- return ld, rdict
-
-def _intersection_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
- else:
- ld = ldict.copy()
- del_list_w = []
- for w_key in ld:
- if w_key not in rdict:
- del_list_w.append(w_key)
-
- for w_key in del_list_w:
- del ld[w_key]
-
- return ld, rdict
-
+ if w_key not in rd:
+ result[w_key] = None
+ return result
-def _symmetric_difference_dict(ldict, rdict, isupdate):
- if isupdate:
- ld = ldict
+def _difference_dict_update(space, ld, rd):
+ if ld is rd:
+ ld.clear() # for the case 'a.difference_update(a)'
else:
- ld = ldict.copy()
- del_list_w = []
- add_list_w = []
+ for w_key in rd:
+ try:
+ del ld[w_key]
+ except KeyError:
+ pass
+
+def _intersection_dict(space, ld, rd):
+ result = newset(space)
+ if len(ld) > len(rd):
+ ld, rd = rd, ld # loop over the smaller dict
for w_key in ld:
- if w_key in rdict:
- del_list_w.append(w_key)
+ if w_key in rd:
+ result[w_key] = None
+ return result
- for w_key in rdict:
+def _symmetric_difference_dict(space, ld, rd):
+ result = newset(space)
+ for w_key in ld:
+ if w_key not in rd:
+ result[w_key] = None
+ for w_key in rd:
if w_key not in ld:
- add_list_w.append(w_key)
-
- for w_key in del_list_w:
- del ld[w_key]
-
- for w_key in add_list_w:
- ld[w_key] = None
-
- return ld, rdict
+ result[w_key] = None
+ return result
def _issubset_dict(ldict, rdict):
if len(ldict) > len(rdict):
@@ -220,14 +192,13 @@
def set_update__Set_BaseSet(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _union_dict(ld, rd, True)
- return space.w_None
+ ld.update(rd)
def set_update__Set_ANY(space, w_left, w_other):
"""Update a set with the union of itself and another."""
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _union_dict(ld, rd, True)
- return space.w_None
+ ld = w_left.setdata
+ for w_item in space.listview(w_other):
+ ld[w_item] = None
def inplace_or__Set_Set(space, w_left, w_other):
set_update__Set_BaseSet(space, w_left, w_other)
@@ -241,25 +212,23 @@
This has no effect if the element is already present.
"""
w_left.setdata[w_other] = None
- return space.w_None
def set_copy__Set(space, w_set):
- return w_set._newobj(space,w_set.setdata)
+ return w_set._newobj(space, w_set.setdata.copy())
def frozenset_copy__Frozenset(space, w_left):
- if _is_frozenset_exact(w_left):
+ if type(w_left) is W_FrozensetObject:
return w_left
else:
- return set_copy__Set(space,w_left)
+ return set_copy__Set(space, w_left)
def set_clear__Set(space, w_left):
w_left.setdata.clear()
- return space.w_None
def set_difference__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _difference_dict(ld, rd, False)
+ new_ld = _difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
set_difference__Set_Frozenset = set_difference__Set_Set
@@ -272,7 +241,7 @@
def set_difference__Set_ANY(space, w_left, w_other):
ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _difference_dict(ld, rd, False)
+ new_ld = _difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
frozenset_difference__Frozenset_ANY = set_difference__Set_ANY
@@ -281,15 +250,17 @@
def set_difference_update__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _difference_dict(ld, rd, True)
- return space.w_None
+ _difference_dict_update(space, ld, rd)
set_difference_update__Set_Frozenset = set_difference_update__Set_Set
def set_difference_update__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _difference_dict(ld, rd, True)
- return space.w_None
+ ld = w_left.setdata
+ for w_key in space.listview(w_other):
+ try:
+ del ld[w_key]
+ except KeyError:
+ pass
def inplace_sub__Set_Set(space, w_left, w_other):
set_difference_update__Set_Set(space, w_left, w_other)
@@ -467,7 +438,7 @@
def hash__Frozenset(space, w_set):
multi = r_uint(1822399083) + r_uint(1822399083) + 1
- if w_set.hash != -1:
+ if w_set.hash != 0:
return space.wrap(w_set.hash)
hash = 1927868237
hash *= (len(w_set.setdata) + 1)
@@ -476,7 +447,7 @@
value = ((h ^ (h << 16) ^ 89869747) * multi)
hash = intmask(hash ^ value)
hash = hash * 69069 + 907133923
- if hash == -1:
+ if hash == 0:
hash = 590923713
hash = intmask(hash)
w_set.hash = hash
@@ -484,29 +455,31 @@
return space.wrap(hash)
def set_pop__Set(space, w_left):
- if len(w_left.setdata) == 0:
+ for w_key in w_left.setdata:
+ break
+ else:
raise OperationError(space.w_KeyError,
space.wrap('pop from an empty set'))
- w_keys = w_left.setdata.keys()
- w_value = w_keys[0]
- del w_left.setdata[w_value]
-
- return w_value
+ del w_left.setdata[w_key]
+ return w_key
def set_intersection__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _intersection_dict(ld, rd, False)
- return w_left._newobj(space,new_ld)
+ new_ld = _intersection_dict(space, ld, rd)
+ return w_left._newobj(space, new_ld)
set_intersection__Set_Frozenset = set_intersection__Set_Set
set_intersection__Frozenset_Frozenset = set_intersection__Set_Set
set_intersection__Frozenset_Set = set_intersection__Set_Set
def set_intersection__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _intersection_dict(ld, rd, False)
- return w_left._newobj(space,new_ld)
+ result = newset(space)
+ ld = w_left.setdata
+ for w_key in space.listview(w_other):
+ if w_key in ld:
+ result[w_key] = None
+ return w_left._newobj(space, result)
frozenset_intersection__Frozenset_ANY = set_intersection__Set_ANY
@@ -518,15 +491,18 @@
def set_intersection_update__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _intersection_dict(ld, rd, True)
- return space.w_None
+ new_ld = _intersection_dict(space, ld, rd)
+ w_left.setdata = new_ld
set_intersection_update__Set_Frozenset = set_intersection_update__Set_Set
def set_intersection_update__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _intersection_dict(ld, rd, True)
- return space.w_None
+ result = newset(space)
+ ld = w_left.setdata
+ for w_key in space.listview(w_other):
+ if w_key in ld:
+ result[w_key] = None
+ w_left.setdata = result
def inplace_and__Set_Set(space, w_left, w_other):
set_intersection_update__Set_Set(space, w_left, w_other)
@@ -537,7 +513,7 @@
def set_symmetric_difference__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _symmetric_difference_dict(ld, rd, False)
+ new_ld = _symmetric_difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
set_symmetric_difference__Set_Frozenset = set_symmetric_difference__Set_Set
@@ -553,7 +529,7 @@
def set_symmetric_difference__Set_ANY(space, w_left, w_other):
ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _symmetric_difference_dict(ld, rd, False)
+ new_ld = _symmetric_difference_dict(space, ld, rd)
return w_left._newobj(space, new_ld)
frozenset_symmetric_difference__Frozenset_ANY = \
@@ -562,16 +538,16 @@
def set_symmetric_difference_update__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _symmetric_difference_dict(ld, rd, True)
- return space.w_None
+ new_ld = _symmetric_difference_dict(space, ld, rd)
+ w_left.setdata = new_ld
set_symmetric_difference_update__Set_Frozenset = \
set_symmetric_difference_update__Set_Set
def set_symmetric_difference_update__Set_ANY(space, w_left, w_other):
ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _symmetric_difference_dict(ld, rd, True)
- return space.w_None
+ new_ld = _symmetric_difference_dict(space, ld, rd)
+ w_left.setdata = new_ld
def inplace_xor__Set_Set(space, w_left, w_other):
set_symmetric_difference_update__Set_Set(space, w_left, w_other)
@@ -582,8 +558,9 @@
def set_union__Set_Set(space, w_left, w_other):
# optimization only (the general case works too)
ld, rd = w_left.setdata, w_other.setdata
- new_ld, rd = _union_dict(ld, rd, False)
- return w_left._newobj(space, new_ld)
+ result = ld.copy()
+ result.update(rd)
+ return w_left._newobj(space, result)
set_union__Set_Frozenset = set_union__Set_Set
set_union__Frozenset_Set = set_union__Set_Set
@@ -595,9 +572,11 @@
def set_union__Set_ANY(space, w_left, w_other):
- ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other)
- new_ld, rd = _union_dict(ld, rd, False)
- return w_left._newobj(space, new_ld)
+ ld = w_left.setdata
+ result = ld.copy()
+ for w_key in space.listview(w_other):
+ result[w_key] = None
+ return w_left._newobj(space, result)
frozenset_union__Frozenset_ANY = set_union__Set_ANY
@@ -629,15 +608,6 @@
init_defaults)
_initialize_set(space, w_set, w_iterable)
-def init__Frozenset(space, w_set, __args__):
- w_iterable, = __args__.parse_obj(
- None, 'set',
- init_signature,
- init_defaults)
- if w_set.hash == -1:
- _initialize_set(space, w_set, w_iterable)
- hash__Frozenset(space, w_set)
-
app = gateway.applevel("""
def setrepr(currently_in_repr, s):
'The app-level part of repr().'
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/settype.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/settype.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/settype.py Sat Oct 30 16:41:17 2010
@@ -66,9 +66,9 @@
register_all(vars(), globals())
def descr__new__(space, w_settype, __args__):
- from pypy.objspace.std.setobject import W_SetObject
+ from pypy.objspace.std.setobject import W_SetObject, newset
w_obj = space.allocate_instance(W_SetObject, w_settype)
- W_SetObject.__init__(w_obj, space, None)
+ W_SetObject.__init__(w_obj, space, newset(space))
return w_obj
set_typedef = StdTypeDef("set",
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/smallintobject.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/smallintobject.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/smallintobject.py Sat Oct 30 16:41:17 2010
@@ -2,18 +2,13 @@
Implementation of small ints, stored as odd-valued pointers in the
translated PyPy. To enable them, see inttype.py.
"""
-import types
-from pypy.interpreter.error import OperationError
from pypy.objspace.std import intobject
from pypy.objspace.std.model import registerimplementation, W_Object
from pypy.objspace.std.register_all import register_all
-from pypy.objspace.std.multimethod import FailedToImplementArgs
from pypy.objspace.std.noneobject import W_NoneObject
-from pypy.rlib.rarithmetic import ovfcheck, ovfcheck_lshift, LONG_BIT, r_uint
-from pypy.objspace.std.inttype import wrapint
-from pypy.objspace.std.intobject import W_IntObject, _impl_int_int_pow
+from pypy.objspace.std.intobject import W_IntObject
from pypy.rlib.objectmodel import UnboxedValue
-from pypy.rlib.rbigint import rbigint
+from pypy.tool.sourcetools import func_with_new_name
class W_SmallIntObject(W_Object, UnboxedValue):
@@ -46,9 +41,7 @@
new_name = name.replace("Int", "SmallInt")
# Copy the function, so the annotator specializes it for
# W_SmallIntObject.
- ns[new_name] = types.FunctionType(func.func_code, ns, new_name,
- func.func_defaults,
- func.func_closure)
+ ns[new_name] = func_with_new_name(func, new_name)
ns["get_integer"] = ns["pos__SmallInt"] = ns["int__SmallInt"]
ns["get_negint"] = ns["neg__SmallInt"]
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/stringobject.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/stringobject.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/stringobject.py Sat Oct 30 16:41:17 2010
@@ -14,7 +14,7 @@
from pypy.rlib.rstring import StringBuilder, string_repeat
from pypy.interpreter.buffer import StringBuffer
-from pypy.objspace.std.stringtype import sliced, joined, wrapstr, wrapchar, \
+from pypy.objspace.std.stringtype import sliced, wrapstr, wrapchar, \
stringendswith, stringstartswith, joined2
from pypy.objspace.std.formatting import mod_format
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/stringtype.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/stringtype.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/stringtype.py Sat Oct 30 16:41:17 2010
@@ -55,19 +55,14 @@
return W_StringSliceObject(s, start, stop)
return wrapstr(space, s[start:stop])
-def joined(space, strlist):
- assert not space.config.objspace.std.withrope
- if space.config.objspace.std.withstrjoin:
- from pypy.objspace.std.strjoinobject import W_StringJoinObject
- return W_StringJoinObject(strlist)
- else:
- return wrapstr(space, "".join(strlist))
-
def joined2(space, str1, str2):
assert not space.config.objspace.std.withrope
if space.config.objspace.std.withstrjoin:
from pypy.objspace.std.strjoinobject import W_StringJoinObject
return W_StringJoinObject([str1, str2])
+ elif space.config.objspace.std.withstrbuf:
+ from pypy.objspace.std.strbufobject import joined2
+ return joined2(str1, str2)
else:
return wrapstr(space, str1 + str2)
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/strutil.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/strutil.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/strutil.py Sat Oct 30 16:41:17 2010
@@ -4,6 +4,7 @@
from pypy.rlib.rarithmetic import ovfcheck, break_up_float, parts_to_float,\
INFINITY, NAN
+from pypy.rlib.rbigint import rbigint, parse_digit_string
from pypy.interpreter.error import OperationError
import math
@@ -91,9 +92,10 @@
return -1
def string_to_int(s, base=10):
- """Utility to converts a string to an integer (or possibly a long).
+ """Utility to converts a string to an integer.
If base is 0, the proper base is guessed based on the leading
characters of 's'. Raises ParseStringError in case of error.
+ Raises ParseStringOverflowError in case the result does not fit.
"""
s = literal = strip_spaces(s)
p = NumberStringParser(s, literal, base, 'int')
@@ -113,11 +115,9 @@
except OverflowError:
raise ParseStringOverflowError(p)
-def string_to_long(space, s, base=10, parser=None):
- return string_to_w_long(space, s, base, parser).longval()
-
-def string_to_w_long(space, s, base=10, parser=None):
- """As string_to_int(), but ignores an optional 'l' or 'L' suffix."""
+def string_to_bigint(s, base=10, parser=None):
+ """As string_to_int(), but ignores an optional 'l' or 'L' suffix
+ and returns an rbigint."""
if parser is None:
s = literal = strip_spaces(s)
if (s.endswith('l') or s.endswith('L')) and base < 22:
@@ -126,18 +126,7 @@
p = NumberStringParser(s, literal, base, 'long')
else:
p = parser
- w_base = space.newlong(p.base)
- w_result = space.newlong(0)
- while True:
- digit = p.next_digit()
- if digit == -1:
- if p.sign == -1:
- w_result = space.neg(w_result)
- # XXX grumble
- from pypy.objspace.std.longobject import W_LongObject
- assert isinstance(w_result, W_LongObject)
- return w_result
- w_result = space.add(space.mul(w_result,w_base), space.newlong(digit))
+ return parse_digit_string(p)
# Tim's comment:
# 57 bits are more than needed in any case.
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_dictmultiobject.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_dictmultiobject.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_dictmultiobject.py Sat Oct 30 16:41:17 2010
@@ -4,7 +4,6 @@
StrDictImplementation
from pypy.objspace.std.celldict import ModuleDictImplementation
-from pypy.objspace.std.sharingdict import SharedDictImplementation
from pypy.conftest import gettestobjspace
@@ -238,7 +237,17 @@
it1 = d.popitem()
assert len(d) == 0
assert (it!=it1) and (it1==(1,2) or it1==(3,4))
-
+ raises(KeyError, d.popitem)
+
+ def test_popitem_2(self):
+ class A(object):
+ pass
+ d = A().__dict__
+ d['x'] = 5
+ it1 = d.popitem()
+ assert it1 == ('x', 5)
+ raises(KeyError, d.popitem)
+
def test_setdefault(self):
d = {1:2, 3:4}
dd = d.copy()
@@ -446,7 +455,9 @@
d1 = {}
d2 = D(a='foo')
d1.update(d2)
- assert d1['a'] == 42 # fails on CPython, d1['a'] == 'foo'
+ assert d1['a'] == 'foo'
+ # a bit of an obscure case: now (from r78295) we get the same result
+ # as CPython does
def test_index_keyerror_unpacking(self):
d = {}
@@ -510,32 +521,6 @@
assert getattr(a, s) == 42
-class TestW_DictSharing(TestW_DictObject):
- def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True})
-
-class AppTest_DictSharing(AppTest_DictObject):
- def setup_class(cls):
- cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True})
-
- def test_values_does_not_share(self):
- class A(object):
- pass
- a = A()
- a.abc = 12
- l = a.__dict__.values()
- assert l == [12]
- l[0] = 24
- assert a.abc == 12
-
- def test_items(self):
- class A(object):
- pass
- a = A()
- a.abc = 12
- a.__dict__.items() == [("abc", 12)]
-
-
class AppTestModuleDict(object):
def setup_class(cls):
cls.space = gettestobjspace(**{"objspace.std.withcelldict": True})
@@ -602,6 +587,15 @@
classofinstance=classofinstance,
from_strdict_shared=from_strdict_shared)
+ def finditem_str(self, w_dict, s):
+ return w_dict.getitem_str(s) # assume it's a multidict
+
+ def setitem_str(self, w_dict, s, w_value):
+ return w_dict.setitem_str(s, w_value) # assume it's a multidict
+
+ def delitem(self, w_dict, w_s):
+ return w_dict.delitem(w_s) # assume it's a multidict
+
def allocate_instance(self, cls, type):
return object.__new__(cls)
@@ -611,7 +605,7 @@
w_StopIteration = StopIteration
w_None = None
StringObjectCls = FakeString
- w_dict = None
+ w_dict = W_DictMultiObject
iter = iter
fixedview = list
listview = list
@@ -620,10 +614,8 @@
class objspace:
class std:
withdictmeasurement = False
- withsharingdict = False
withsmalldicts = False
withcelldict = False
- withshadowtracking = False
class opcodes:
CALL_LIKELY_BUILTIN = False
@@ -687,6 +679,14 @@
assert self.impl.length() == 0
self.check_not_devolved()
+ def test_clear(self):
+ self.fill_impl()
+ assert self.impl.length() == 2
+ self.impl.clear()
+ assert self.impl.length() == 0
+ self.check_not_devolved()
+
+
def test_keys(self):
self.fill_impl()
keys = self.impl.keys()
@@ -750,9 +750,6 @@
string = "int"
string2 = "isinstance"
-class TestSharedDictImplementation(BaseTestRDictImplementation):
- ImplementionClass = SharedDictImplementation
-
class BaseTestDevolvedDictImplementation(BaseTestRDictImplementation):
def fill_impl(self):
@@ -774,8 +771,6 @@
string = "int"
string2 = "isinstance"
-class TestDevolvedSharedDictImplementation(BaseTestDevolvedDictImplementation):
- ImplementionClass = SharedDictImplementation
def test_module_uses_strdict():
fakespace = FakeSpace()
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_longobject.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_longobject.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_longobject.py Sat Oct 30 16:41:17 2010
@@ -41,56 +41,64 @@
class AppTestLong:
def test_add(self):
- assert int(123L + 12443L) == 123 + 12443
- assert -20 + 2 + 3L + True == -14L
+ x = 123L
+ assert int(x + 12443L) == 123 + 12443
+ x = -20
+ assert x + 2 + 3L + True == -14L
def test_sub(self):
- assert int(58543L - 12332L) == 58543 - 12332
- assert 237123838281233L * 12 == 237123838281233L * 12L
+ x = 58543L
+ assert int(x - 12332L) == 58543 - 12332
+ x = 237123838281233L
+ assert x * 12 == x * 12L
def test_mul(self):
- assert 363L * 2 ** 40 == 363L << 40
+ x = 363L
+ assert x * 2 ** 40 == x << 40
def test_truediv(self):
exec "from __future__ import division; a = 31415926L / 10000000L"
assert a == 3.1415926
def test_floordiv(self):
- a = 31415926L // 10000000L
+ x = 31415926L
+ a = x // 10000000L
assert a == 3L
def test_compare(self):
+ Z = 0
+ ZL = 0L
for BIG in (1L, 1L << 62, 1L << 9999):
- assert 0 == 0L
- assert not (0 != 0L)
- assert 0L == 0
- assert not (0L != 0)
- assert not (0 == BIG)
- assert 0 != BIG
- assert not (BIG == 0)
- assert BIG != 0
- assert not (0L == BIG)
- assert 0L != BIG
- assert 0 <= 0L
- assert not (0 < 0L)
- assert 0 <= BIG
- assert 0 < BIG
- assert not (BIG <= 0)
- assert not (BIG < 0)
- assert 0L <= 0L
- assert not (0L < 0L)
- assert 0L <= BIG
- assert 0L < BIG
- assert not (BIG <= 0L)
- assert not (BIG < 0L)
- assert not (0 <= -BIG)
- assert not (0 < -BIG)
- assert -BIG <= 0
- assert -BIG < 0
- assert not (0L <= -BIG)
- assert not (0L < -BIG)
- assert -BIG <= 0L
- assert -BIG < 0L
+ assert Z == ZL
+ assert not (Z != ZL)
+ assert ZL == Z
+ assert not (ZL != Z)
+ assert not (Z == BIG)
+ assert Z != BIG
+ assert not (BIG == Z)
+ assert BIG != Z
+ assert not (ZL == BIG)
+ assert ZL != BIG
+ assert Z <= ZL
+ assert not (Z < ZL)
+ assert Z <= BIG
+ assert Z < BIG
+ assert not (BIG <= Z)
+ assert not (BIG < Z)
+ assert ZL <= ZL
+ assert not (ZL < ZL)
+ assert ZL <= BIG
+ assert ZL < BIG
+ assert not (BIG <= ZL)
+ assert not (BIG < ZL)
+ assert not (Z <= -BIG)
+ assert not (Z < -BIG)
+ assert -BIG <= Z
+ assert -BIG < Z
+ assert not (ZL <= -BIG)
+ assert not (ZL < -BIG)
+ assert -BIG <= ZL
+ assert -BIG < ZL
#
assert not (BIG < int(BIG))
assert (BIG <= int(BIG))
@@ -137,7 +145,8 @@
def test_conversion(self):
class long2(long):
pass
- x = long2(1L<<100)
+ x = 1L
+ x = long2(x<<100)
y = int(x)
assert type(y) == long
assert type(+long2(5)) is long
@@ -152,7 +161,8 @@
assert type(long2(5) // 1) is long
def test_pow(self):
- assert pow(0L, 0L, 1L) == 0L
+ x = 0L
+ assert pow(x, 0L, 1L) == 0L
def test_getnewargs(self):
assert 0L .__getnewargs__() == (0L,)
@@ -193,13 +203,14 @@
assert oct(01234567012345670L) == '01234567012345670L'
def test_bits(self):
- assert 0xAAAAAAAAL | 0x55555555L == 0xFFFFFFFFL
- assert 0xAAAAAAAAL & 0x55555555L == 0x00000000L
- assert 0xAAAAAAAAL ^ 0x55555555L == 0xFFFFFFFFL
- assert -0xAAAAAAAAL | 0x55555555L == -0xAAAAAAA9L
- assert 0xAAAAAAAAL | 0x555555555L == 0x5FFFFFFFFL
- assert 0xAAAAAAAAL & 0x555555555L == 0x000000000L
- assert 0xAAAAAAAAL ^ 0x555555555L == 0x5FFFFFFFFL
+ x = 0xAAAAAAAAL
+ assert x | 0x55555555L == 0xFFFFFFFFL
+ assert x & 0x55555555L == 0x00000000L
+ assert x ^ 0x55555555L == 0xFFFFFFFFL
+ assert -x | 0x55555555L == -0xAAAAAAA9L
+ assert x | 0x555555555L == 0x5FFFFFFFFL
+ assert x & 0x555555555L == 0x000000000L
+ assert x ^ 0x555555555L == 0x5FFFFFFFFL
def test_hash(self):
# ints have the same hash as equal longs
@@ -229,7 +240,8 @@
def test_huge_longs(self):
import operator
- huge = 1L << 40000L
+ x = 1L
+ huge = x << 40000L
raises(OverflowError, float, huge)
raises(OverflowError, operator.truediv, huge, 3)
raises(OverflowError, operator.truediv, huge, 3L)
@@ -242,3 +254,7 @@
class myotherlong(long):
pass
assert long(myotherlong(21)) == 21L
+
+ def test_negative_zero(self):
+ x = eval("-0L")
+ assert x == 0L
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_setobject.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_setobject.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_setobject.py Sat Oct 30 16:41:17 2010
@@ -10,6 +10,7 @@
import py.test
from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject
from pypy.objspace.std.setobject import _initialize_set
+from pypy.objspace.std.setobject import newset, make_setdata_from_w_iterable
from pypy.objspace.std.setobject import set_intersection__Set_Set
from pypy.objspace.std.setobject import set_intersection__Set_ANY
from pypy.objspace.std.setobject import eq__Set_Set
@@ -28,12 +29,12 @@
self.false = self.space.w_False
def test_and(self):
- s = W_SetObject(self.space, None)
+ s = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, s, self.word)
- t0 = W_SetObject(self.space, None)
+ t0 = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, t0, self.otherword)
- t1 = W_FrozensetObject(self.space, None)
- _initialize_set(self.space, t1, self.otherword)
+ t1 = W_FrozensetObject(self.space,
+ make_setdata_from_w_iterable(self.space, self.otherword))
r0 = set_intersection__Set_Set(self.space, s, t0)
r1 = set_intersection__Set_Set(self.space, s, t1)
assert eq__Set_Set(self.space, r0, r1) == self.true
@@ -41,9 +42,9 @@
assert eq__Set_Set(self.space, r0, sr) == self.true
def test_compare(self):
- s = W_SetObject(self.space, None)
+ s = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, s, self.word)
- t = W_SetObject(self.space, None)
+ t = W_SetObject(self.space, newset(self.space))
_initialize_set(self.space, t, self.word)
assert self.space.eq_w(s,t)
u = self.space.wrap(set('simsalabim'))
@@ -56,6 +57,11 @@
b = a | set('abc')
assert type(b) is subset
+ def test_union(self):
+ a = set([4, 5])
+ b = a.union([5, 7])
+ assert sorted(b) == [4, 5, 7]
+
def test_compare(self):
raises(TypeError, cmp, set('abc'), set('abd'))
assert set('abc') != 'abc'
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_strutil.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_strutil.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_strutil.py Sat Oct 30 16:41:17 2010
@@ -25,7 +25,7 @@
]
for s, expected in cases:
assert string_to_int(s) == expected
- assert string_to_w_long(space, s).longval() == expected
+ assert string_to_bigint(s).tolong() == expected
def test_string_to_int_base(self):
space = self.space
@@ -119,17 +119,16 @@
raises(ParseStringError, string_to_int, '+'+s, base)
raises(ParseStringError, string_to_int, '-'+s, base)
- def test_string_to_w_long(self):
- space = self.space
- assert string_to_w_long(space, '123L').longval() == 123
- assert string_to_w_long(space, '123L ').longval() == 123
- raises(ParseStringError, string_to_w_long, space, 'L')
- raises(ParseStringError, string_to_w_long, space, 'L ')
- assert string_to_w_long(space, '123L', 4).longval() == 27
- assert string_to_w_long(space, '123L', 30).longval() == 27000 + 1800 + 90 + 21
- assert string_to_w_long(space, '123L', 22).longval() == 10648 + 968 + 66 + 21
- assert string_to_w_long(space, '123L', 21).longval() == 441 + 42 + 3
- assert string_to_w_long(space, '1891234174197319').longval() == 1891234174197319
+ def test_string_to_bigint(self):
+ assert string_to_bigint('123L').tolong() == 123
+ assert string_to_bigint('123L ').tolong() == 123
+ raises(ParseStringError, string_to_bigint, 'L')
+ raises(ParseStringError, string_to_bigint, 'L ')
+ assert string_to_bigint('123L', 4).tolong() == 27
+ assert string_to_bigint('123L', 30).tolong() == 27000 + 1800 + 90 + 21
+ assert string_to_bigint('123L', 22).tolong() == 10648 + 968 + 66 + 21
+ assert string_to_bigint('123L', 21).tolong() == 441 + 42 + 3
+ assert string_to_bigint('1891234174197319').tolong() == 1891234174197319
def test_string_to_float(self):
def string_to_float(x):
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_typeobject.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_typeobject.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_typeobject.py Sat Oct 30 16:41:17 2010
@@ -1085,4 +1085,12 @@
assert b == 1
-
+ def test_slots_with_method_in_class(self):
+ # this works in cpython...
+ class A(object):
+ __slots__ = ["f"]
+ def f(self, x):
+ return x + 1
+ a = A()
+ assert a.f(1) == 2
+
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_versionedtype.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_versionedtype.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_versionedtype.py Sat Oct 30 16:41:17 2010
@@ -12,10 +12,12 @@
def f(self): pass
class B(A):
pass
- class metatype(type):
+ class X:
+ pass
+ class Y(object):
+ pass
+ class C(Y, X):
pass
- class C(object):
- __metaclass__ = metatype
return A, B, C
""")
return space.unpackiterable(w_types)
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/typeobject.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/std/typeobject.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/std/typeobject.py Sat Oct 30 16:41:17 2010
@@ -60,6 +60,15 @@
self.hits = {}
self.misses = {}
+ def clear(self):
+ None_None = (None, None)
+ for i in range(len(self.versions)):
+ self.versions[i] = None
+ for i in range(len(self.names)):
+ self.names[i] = None
+ for i in range(len(self.lookup_where)):
+ self.lookup_where[i] = None_None
+
class W_TypeObject(W_Object):
from pypy.objspace.std.typetype import type_typedef as typedef
@@ -75,7 +84,9 @@
'weakrefable',
'hasdict',
'nslots',
- 'instancetypedef']
+ 'instancetypedef',
+ 'terminator',
+ ]
# for config.objspace.std.getattributeshortcut
# (False is a conservative default, fixed during real usage)
@@ -102,20 +113,24 @@
if overridetypedef is not None:
setup_builtin_type(w_self)
- custom_metaclass = False
else:
setup_user_defined_type(w_self)
- custom_metaclass = not space.is_w(space.type(w_self), space.w_type)
w_self.w_same_layout_as = get_parent_layout(w_self)
if space.config.objspace.std.withtypeversion:
- if custom_metaclass or not is_mro_purely_of_types(w_self.mro_w):
+ if not is_mro_purely_of_types(w_self.mro_w):
pass
else:
# the _version_tag should change, whenever the content of
# dict_w of any of the types in the mro changes, or if the mro
# itself changes
w_self._version_tag = VersionTag()
+ if space.config.objspace.std.withmapdict:
+ from pypy.objspace.std.mapdict import DictTerminator, NoDictTerminator
+ if w_self.hasdict:
+ w_self.terminator = DictTerminator(space, w_self)
+ else:
+ w_self.terminator = NoDictTerminator(space, w_self)
def mutated(w_self):
space = w_self.space
@@ -551,11 +566,13 @@
space.wrap('__slots__ must be identifiers'))
# create member
slot_name = _mangle(slot_name, w_self.name)
- # Force interning of slot names.
- slot_name = space.str_w(space.new_interned_str(slot_name))
- member = Member(w_self.nslots, slot_name, w_self)
- w_self.dict_w[slot_name] = space.wrap(member)
- w_self.nslots += 1
+ if slot_name not in w_self.dict_w:
+ # Force interning of slot names.
+ slot_name = space.str_w(space.new_interned_str(slot_name))
+ # in cpython it is ignored less, but we probably don't care
+ member = Member(w_self.nslots, slot_name, w_self)
+ w_self.dict_w[slot_name] = space.wrap(member)
+ w_self.nslots += 1
def create_dict_slot(w_self):
if not w_self.hasdict:
Modified: pypy/branch/jit-unroll-loops/pypy/objspace/test/test_descroperation.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/objspace/test/test_descroperation.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/objspace/test/test_descroperation.py Sat Oct 30 16:41:17 2010
@@ -400,7 +400,7 @@
class D(object):
def __init__(self, a):
self.a = a
-
+
assert A(1) == B(1)
assert B(1) == A(1)
assert A(1) == C(1)
@@ -457,6 +457,20 @@
else:
assert False, "did not raise"
+ def test_attribute_error(self):
+ class classmethodonly(classmethod):
+ def __get__(self, instance, type):
+ if instance is not None:
+ raise AttributeError("Must be called on a class, not an instance.")
+ return super(classmethodonly, self).__get__(instance, type)
+
+ class A(object):
+ @classmethodonly
+ def a(cls):
+ return 3
+
+ raises(AttributeError, lambda: A().a)
+
class AppTestWithBuiltinShortcut(AppTest_Descroperation):
OPTIONS = {'objspace.std.builtinshortcut': True}
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/_rsocket_rffi.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/_rsocket_rffi.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/_rsocket_rffi.py Sat Oct 30 16:41:17 2010
@@ -324,10 +324,11 @@
('sll_hatype', rffi.INT),
('sll_addr', rffi.CFixedArray(rffi.CHAR, 8)),
('sll_halen', rffi.INT)],
- )
+ ifdef='AF_PACKET')
CConfig.ifreq = platform.Struct('struct ifreq', [('ifr_ifindex', rffi.INT),
- ('ifr_name', rffi.CFixedArray(rffi.CHAR, 8))])
+ ('ifr_name', rffi.CFixedArray(rffi.CHAR, 8))],
+ ifdef='AF_PACKET')
if _WIN32:
CConfig.WSAEVENT = platform.SimpleType('WSAEVENT', rffi.VOIDP)
@@ -532,8 +533,9 @@
socketpair_t = rffi.CArray(socketfd_type)
socketpair = external('socketpair', [rffi.INT, rffi.INT, rffi.INT,
lltype.Ptr(socketpair_t)], rffi.INT)
- ioctl = external('ioctl', [socketfd_type, rffi.INT, lltype.Ptr(ifreq)],
- rffi.INT)
+ if ifreq is not None:
+ ioctl = external('ioctl', [socketfd_type, rffi.INT, lltype.Ptr(ifreq)],
+ rffi.INT)
if _WIN32:
ioctlsocket = external('ioctlsocket',
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/jit.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/jit.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/jit.py Sat Oct 30 16:41:17 2010
@@ -77,6 +77,12 @@
return result
return decorator
+def oopspec(spec):
+ def decorator(func):
+ func.oopspec = spec
+ return func
+ return decorator
+
class Entry(ExtRegistryEntry):
_about_ = hint
@@ -139,6 +145,24 @@
return hop.inputconst(lltype.Signed, _we_are_jitted)
+def jit_debug(string, arg1=-sys.maxint-1, arg2=-sys.maxint-1,
+ arg3=-sys.maxint-1, arg4=-sys.maxint-1):
+ """When JITted, cause an extra operation DEBUG_MERGE_POINT to appear in
+ the graphs. Should not be left after debugging."""
+ keepalive_until_here(string) # otherwise the whole function call is removed
+jit_debug.oopspec = 'jit.debug(string, arg1, arg2, arg3, arg4)'
+
+def assert_green(value):
+ """Very strong assert: checks that 'value' is a green
+ (a JIT compile-time constant)."""
+ keepalive_until_here(value)
+assert_green._annspecialcase_ = 'specialize:argtype(0)'
+assert_green.oopspec = 'jit.assert_green(value)'
+
+class AssertGreenFailed(Exception):
+ pass
+
+
##def force_virtualizable(virtualizable):
## pass
@@ -251,8 +275,9 @@
several independent JITting interpreters in it.
"""
+ active = True # if set to False, this JitDriver is ignored
virtualizables = []
-
+
def __init__(self, greens=None, reds=None, virtualizables=None,
get_jitcell_at=None, set_jitcell_at=None,
get_printable_location=None, confirm_enter_jit=None,
@@ -267,7 +292,8 @@
self.virtualizables = virtualizables
for v in self.virtualizables:
assert v in self.reds
- self._alllivevars = dict.fromkeys(self.greens + self.reds)
+ self._alllivevars = dict.fromkeys(
+ [name for name in self.greens + self.reds if '.' not in name])
self._make_extregistryentries()
self.get_jitcell_at = get_jitcell_at
self.set_jitcell_at = set_jitcell_at
@@ -356,10 +382,16 @@
def compute_result_annotation(self, **kwds_s):
from pypy.annotation import model as annmodel
+
+ if self.instance.__name__ == 'jit_merge_point':
+ if not self.annotate_hooks(**kwds_s):
+ return None # wrong order, try again later
+
driver = self.instance.im_self
keys = kwds_s.keys()
keys.sort()
- expected = ['s_' + name for name in driver.greens + driver.reds]
+ expected = ['s_' + name for name in driver.greens + driver.reds
+ if '.' not in name]
expected.sort()
if keys != expected:
raise JitHintError("%s expects the following keyword "
@@ -383,30 +415,35 @@
key[2:])
cache[key] = s_value
- if self.instance.__name__ == 'jit_merge_point':
- self.annotate_hooks(**kwds_s)
-
return annmodel.s_None
def annotate_hooks(self, **kwds_s):
driver = self.instance.im_self
s_jitcell = self.bookkeeper.valueoftype(BaseJitCell)
- self.annotate_hook(driver.get_jitcell_at, driver.greens, **kwds_s)
- self.annotate_hook(driver.set_jitcell_at, driver.greens, [s_jitcell],
- **kwds_s)
- self.annotate_hook(driver.get_printable_location, driver.greens, **kwds_s)
+ h = self.annotate_hook
+ return (h(driver.get_jitcell_at, driver.greens, **kwds_s)
+ and h(driver.set_jitcell_at, driver.greens, [s_jitcell], **kwds_s)
+ and h(driver.get_printable_location, driver.greens, **kwds_s))
def annotate_hook(self, func, variables, args_s=[], **kwds_s):
if func is None:
- return
+ return True
bk = self.bookkeeper
s_func = bk.immutablevalue(func)
uniquekey = 'jitdriver.%s' % func.func_name
args_s = args_s[:]
for name in variables:
- s_arg = kwds_s['s_' + name]
+ if '.' not in name:
+ s_arg = kwds_s['s_' + name]
+ else:
+ objname, fieldname = name.split('.')
+ s_instance = kwds_s['s_' + objname]
+ s_arg = s_instance.classdef.about_attribute(fieldname)
+ if s_arg is None:
+ return False # wrong order, try again later
args_s.append(s_arg)
bk.emulate_pbc_call(uniquekey, s_func, args_s)
+ return True
def specialize_call(self, hop, **kwds_i):
# XXX to be complete, this could also check that the concretetype
@@ -417,9 +454,42 @@
greens_v = []
reds_v = []
for name in driver.greens:
- i = kwds_i['i_' + name]
- r_green = hop.args_r[i]
- v_green = hop.inputarg(r_green, arg=i)
+ if '.' not in name:
+ i = kwds_i['i_' + name]
+ r_green = hop.args_r[i]
+ v_green = hop.inputarg(r_green, arg=i)
+ else:
+ if hop.rtyper.type_system.name == 'ootypesystem':
+ py.test.skip("lltype only")
+ objname, fieldname = name.split('.') # see test_green_field
+ assert objname in driver.reds
+ i = kwds_i['i_' + objname]
+ s_red = hop.args_s[i]
+ r_red = hop.args_r[i]
+ while True:
+ try:
+ mangled_name, r_field = r_red._get_field(fieldname)
+ break
+ except KeyError:
+ pass
+ assert r_red.rbase is not None, (
+ "field %r not found in %r" % (name,
+ r_red.lowleveltype.TO))
+ r_red = r_red.rbase
+ GTYPE = r_red.lowleveltype.TO
+ assert GTYPE._immutable_field(mangled_name), (
+ "field %r must be declared as immutable" % name)
+ if not hasattr(driver, 'll_greenfields'):
+ driver.ll_greenfields = {}
+ driver.ll_greenfields[name] = GTYPE, mangled_name
+ #
+ v_red = hop.inputarg(r_red, arg=i)
+ c_llname = hop.inputconst(lltype.Void, mangled_name)
+ v_green = hop.genop('getfield', [v_red, c_llname],
+ resulttype = r_field)
+ s_green = s_red.classdef.about_attribute(fieldname)
+ assert s_green is not None
+ hop.rtyper.annotator.setbinding(v_green, s_green)
greens_v.append(v_green)
for name in driver.reds:
i = kwds_i['i_' + name]
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/objectmodel.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/objectmodel.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/objectmodel.py Sat Oct 30 16:41:17 2010
@@ -475,6 +475,10 @@
def setdefault(self, key, default):
return self._dict.setdefault(_r_dictkey(self, key), default)
+ def popitem(self):
+ dk, value = self._dict.popitem()
+ return dk.key, value
+
def copy(self):
result = r_dict(self.key_eq, self.key_hash)
result.update(self)
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/rbigint.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/rbigint.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/rbigint.py Sat Oct 30 16:41:17 2010
@@ -1596,11 +1596,16 @@
# a few internal helpers
-DEC_PER_DIGIT = 1
-while int('9' * DEC_PER_DIGIT) < MASK:
- DEC_PER_DIGIT += 1
-DEC_PER_DIGIT -= 1
-DEC_MAX = 10 ** DEC_PER_DIGIT
+def digits_max_for_base(base):
+ dec_per_digit = 1
+ while base ** dec_per_digit < MASK:
+ dec_per_digit += 1
+ dec_per_digit -= 1
+ return base ** dec_per_digit
+
+BASE_MAX = [0, 0] + [digits_max_for_base(_base) for _base in range(2, 37)]
+DEC_MAX = digits_max_for_base(10)
+assert DEC_MAX == BASE_MAX[10]
def _decimalstr_to_bigint(s):
# a string that has been already parsed to be decimal and valid,
@@ -1615,7 +1620,6 @@
p += 1
a = rbigint.fromint(0)
- cnt = DEC_PER_DIGIT
tens = 1
dig = 0
ord0 = ord('0')
@@ -1627,8 +1631,26 @@
a = _muladd1(a, tens, dig)
tens = 1
dig = 0
- if sign:
+ if sign and a.sign == 1:
a.sign = -1
return a
-
+def parse_digit_string(parser):
+ # helper for objspace.std.strutil
+ a = rbigint.fromint(0)
+ base = parser.base
+ digitmax = BASE_MAX[base]
+ tens, dig = 1, 0
+ while True:
+ digit = parser.next_digit()
+ if tens == digitmax or digit < 0:
+ a = _muladd1(a, tens, dig)
+ if digit < 0:
+ break
+ dig = digit
+ tens = base
+ else:
+ dig = dig * base + digit
+ tens *= base
+ a.sign *= parser.sign
+ return a
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/rgc.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/rgc.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/rgc.py Sat Oct 30 16:41:17 2010
@@ -170,7 +170,14 @@
return hop.genop('gc_set_max_heap_size', [v_nbytes],
resulttype=lltype.Void)
-def can_move(p): # NB. must not be called with NULL pointers
+def can_move(p):
+ """Check if the GC object 'p' is at an address that can move.
+ Must not be called with None. With non-moving GCs, it is always False.
+ With some moving GCs like the SemiSpace GC, it is always True.
+ With other moving GCs like the MiniMark GC, it can be True for some
+ time, then False for the same object, when we are sure that it won't
+ move any more.
+ """
return True
class CanMoveEntry(ExtRegistryEntry):
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/rmmap.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/rmmap.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/rmmap.py Sat Oct 30 16:41:17 2010
@@ -50,7 +50,7 @@
constant_names = ['MAP_SHARED', 'MAP_PRIVATE',
'PROT_READ', 'PROT_WRITE',
'MS_SYNC']
- opt_constant_names = ['MAP_ANON', 'MAP_ANONYMOUS',
+ opt_constant_names = ['MAP_ANON', 'MAP_ANONYMOUS', 'MAP_NORESERVE',
'PROT_EXEC',
'MAP_DENYWRITE', 'MAP_EXECUTABLE']
for name in constant_names:
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/rsre/rsre_char.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/rsre/rsre_char.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/rsre/rsre_char.py Sat Oct 30 16:41:17 2010
@@ -4,6 +4,7 @@
import sys
from pypy.rlib.rlocale import tolower, isalnum
from pypy.rlib.unroll import unrolling_iterable
+from pypy.rlib import jit
# Note: the unicode parts of this module require you to call
# rsre_char.set_unicode_db() first, to select one of the modules
@@ -43,6 +44,7 @@
# XXX can we import those safely from sre_constants?
SRE_INFO_PREFIX = 1
SRE_INFO_LITERAL = 2
+SRE_INFO_CHARSET = 4
SRE_FLAG_LOCALE = 4 # honour system locale
SRE_FLAG_UNICODE = 32 # use unicode locale
OPCODE_INFO = 17
@@ -64,33 +66,27 @@
#### Category helpers
-ascii_char_info = [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 6, 2,
-2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0,
-0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 25, 25, 25, 25, 25, 25, 25, 25,
-25, 25, 0, 0, 0, 0, 0, 0, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0,
-0, 0, 16, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
-24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0, 0, 0, 0 ]
-
+is_a_word = [(chr(i).isalnum() or chr(i) == '_') for i in range(256)]
linebreak = ord("\n")
underline = ord("_")
def is_digit(code):
- return code < 128 and (ascii_char_info[code] & 1 != 0)
+ return code <= 57 and code >= 48
def is_uni_digit(code):
assert unicodedb is not None
return unicodedb.isdigit(code)
def is_space(code):
- return code < 128 and (ascii_char_info[code] & 2 != 0)
+ return code == 32 or (code <= 13 and code >= 9)
def is_uni_space(code):
assert unicodedb is not None
return unicodedb.isspace(code)
def is_word(code):
- return code < 128 and (ascii_char_info[code] & 16 != 0)
+ assert code >= 0
+ return code < 256 and is_a_word[code]
def is_uni_word(code):
assert unicodedb is not None
@@ -142,6 +138,7 @@
SET_OK = -1
SET_NOT_OK = -2
+ at jit.unroll_safe
def check_charset(pattern, ppos, char_code):
"""Checks whether a character matches set of arbitrary length.
The set starts at pattern[ppos]."""
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/rsre/rsre_core.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/rsre/rsre_core.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/rsre/rsre_core.py Sat Oct 30 16:41:17 2010
@@ -1,9 +1,11 @@
import sys
-from pypy.rlib.debug import check_nonneg
+from pypy.rlib.debug import check_nonneg, make_sure_not_modified
from pypy.rlib.unroll import unrolling_iterable
from pypy.rlib.rsre import rsre_char
from pypy.tool.sourcetools import func_with_new_name
from pypy.rlib.objectmodel import we_are_translated
+from pypy.rlib import jit
+from pypy.rlib.rsre.rsre_jit import install_jitdriver, install_jitdriver_spec
OPCODE_FAILURE = 0
@@ -56,16 +58,19 @@
_seen_specname[specname] = True
# Install a copy of the function under the name '_spec_funcname' in each
# concrete subclass
+ specialized_methods = []
for prefix, concreteclass in [('str', StrMatchContext),
('uni', UnicodeMatchContext)]:
newfunc = func_with_new_name(func, prefix + specname)
assert not hasattr(concreteclass, specname)
setattr(concreteclass, specname, newfunc)
+ specialized_methods.append(newfunc)
# Return a dispatcher function, specialized on the exact type of 'ctx'
def dispatch(ctx, *args):
return getattr(ctx, specname)(*args)
dispatch._annspecialcase_ = 'specialize:argtype(0)'
- return dispatch
+ dispatch._specialized_methods_ = specialized_methods
+ return func_with_new_name(dispatch, specname)
# ____________________________________________________________
@@ -75,6 +80,7 @@
class AbstractMatchContext(object):
"""Abstract base class"""
+ _immutable_fields_ = ['pattern[*]', 'flags', 'end']
match_start = 0
match_end = 0
match_marks = None
@@ -164,6 +170,8 @@
def __init__(self, pattern, string, match_start, end, flags):
AbstractMatchContext.__init__(self, pattern, match_start, end, flags)
self._string = string
+ if not we_are_translated() and isinstance(string, unicode):
+ self.flags |= rsre_char.SRE_FLAG_UNICODE # for rsre_re.py
def str(self, index):
check_nonneg(index)
@@ -238,8 +246,9 @@
self.start_ptr = ptr
self.start_marks = marks
+ @jit.unroll_safe
def find_first_result(self, ctx):
- ppos = self.ppos
+ ppos = jit.hint(self.ppos, promote=True)
while ctx.pat(ppos):
result = sre_match(ctx, ppos + 1, self.start_ptr, self.start_marks)
ppos += ctx.pat(ppos)
@@ -250,6 +259,10 @@
find_next_result = find_first_result
class RepeatOneMatchResult(MatchResult):
+ install_jitdriver('RepeatOne',
+ greens=['nextppos', 'ctx.pattern'],
+ reds=['ptr', 'self', 'ctx'],
+ debugprint=(1, 0)) # indices in 'greens'
def __init__(self, nextppos, minptr, ptr, marks):
self.nextppos = nextppos
@@ -259,8 +272,11 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
+ nextppos = self.nextppos
while ptr >= self.minptr:
- result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
+ ctx.jitdriver_RepeatOne.jit_merge_point(
+ self=self, ptr=ptr, ctx=ctx, nextppos=nextppos)
+ result = sre_match(ctx, nextppos, ptr, self.start_marks)
ptr -= 1
if result is not None:
self.subresult = result
@@ -270,6 +286,10 @@
class MinRepeatOneMatchResult(MatchResult):
+ install_jitdriver('MinRepeatOne',
+ greens=['nextppos', 'ppos3', 'ctx.pattern'],
+ reds=['ptr', 'self', 'ctx'],
+ debugprint=(2, 0)) # indices in 'greens'
def __init__(self, nextppos, ppos3, maxptr, ptr, marks):
self.nextppos = nextppos
@@ -280,29 +300,32 @@
def find_first_result(self, ctx):
ptr = self.start_ptr
+ nextppos = self.nextppos
+ ppos3 = self.ppos3
while ptr <= self.maxptr:
- result = sre_match(ctx, self.nextppos, ptr, self.start_marks)
+ ctx.jitdriver_MinRepeatOne.jit_merge_point(
+ self=self, ptr=ptr, ctx=ctx, nextppos=nextppos, ppos3=ppos3)
+ result = sre_match(ctx, nextppos, ptr, self.start_marks)
if result is not None:
self.subresult = result
self.start_ptr = ptr
return self
- if not self.next_char_ok(ctx, ptr):
+ if not self.next_char_ok(ctx, ptr, ppos3):
break
ptr += 1
def find_next_result(self, ctx):
ptr = self.start_ptr
- if not self.next_char_ok(ctx, ptr):
+ if not self.next_char_ok(ctx, ptr, self.ppos3):
return
self.start_ptr = ptr + 1
return self.find_first_result(ctx)
- def next_char_ok(self, ctx, ptr):
+ def next_char_ok(self, ctx, ptr, ppos):
if ptr == ctx.end:
return False
- ppos = self.ppos3
op = ctx.pat(ppos)
- for op1, (checkerfn, _) in unroll_char_checker:
+ for op1, checkerfn in unroll_char_checker:
if op1 == op:
return checkerfn(ctx, ptr, ppos)
raise Error("next_char_ok[%d]" % op)
@@ -325,41 +348,34 @@
self.next = next # chained list
class MaxUntilMatchResult(AbstractUntilMatchResult):
+ install_jitdriver('MaxUntil',
+ greens=['ppos', 'tailppos', 'match_more', 'ctx.pattern'],
+ reds=['ptr', 'marks', 'self', 'ctx'],
+ debugprint=(3, 0, 2))
def find_first_result(self, ctx):
- enum = sre_match(ctx, self.ppos + 3, self.cur_ptr, self.cur_marks)
- return self.search_next(ctx, enum, resume=False)
+ return self.search_next(ctx, match_more=True)
def find_next_result(self, ctx):
- return self.search_next(ctx, None, resume=True)
+ return self.search_next(ctx, match_more=False)
- def search_next(self, ctx, enum, resume):
+ def search_next(self, ctx, match_more):
ppos = self.ppos
- min = ctx.pat(ppos+1)
- max = ctx.pat(ppos+2)
+ tailppos = self.tailppos
ptr = self.cur_ptr
marks = self.cur_marks
while True:
- while True:
- if (enum is not None and
- (ptr != ctx.match_end or self.num_pending < min)):
- # ^^^^^^^^^^ zero-width match protection
- # matched one more 'item'. record it and continue.
- self.pending = Pending(ptr, marks, enum, self.pending)
- self.num_pending += 1
- ptr = ctx.match_end
- marks = ctx.match_marks
- break
- # 'item' no longer matches.
- if not resume and self.num_pending >= min:
- # try to match 'tail' if we have enough 'item'
- result = sre_match(ctx, self.tailppos, ptr, marks)
- if result is not None:
- self.subresult = result
- self.cur_ptr = ptr
- self.cur_marks = marks
- return self
- resume = False
+ ctx.jitdriver_MaxUntil.jit_merge_point(
+ ppos=ppos, tailppos=tailppos, match_more=match_more,
+ ptr=ptr, marks=marks, self=self, ctx=ctx)
+ if match_more:
+ max = ctx.pat(ppos+2)
+ if max == 65535 or self.num_pending < max:
+ # try to match one more 'item'
+ enum = sre_match(ctx, ppos + 3, ptr, marks)
+ else:
+ enum = None # 'max' reached, no more matches
+ else:
p = self.pending
if p is None:
return
@@ -369,11 +385,27 @@
marks = p.marks
enum = p.enum.move_to_next_result(ctx)
#
- if max == 65535 or self.num_pending < max:
- # try to match one more 'item'
- enum = sre_match(ctx, ppos + 3, ptr, marks)
+ min = ctx.pat(ppos+1)
+ if (enum is not None and
+ (ptr != ctx.match_end or self.num_pending < min)):
+ # ^^^^^^^^^^ zero-width match protection
+ # matched one more 'item'. record it and continue.
+ self.pending = Pending(ptr, marks, enum, self.pending)
+ self.num_pending += 1
+ ptr = ctx.match_end
+ marks = ctx.match_marks
+ match_more = True
else:
- enum = None # 'max' reached, no more matches
+ # 'item' no longer matches.
+ if self.num_pending >= min:
+ # try to match 'tail' if we have enough 'item'
+ result = sre_match(ctx, tailppos, ptr, marks)
+ if result is not None:
+ self.subresult = result
+ self.cur_ptr = ptr
+ self.cur_marks = marks
+ return self
+ match_more = False
class MinUntilMatchResult(AbstractUntilMatchResult):
@@ -384,6 +416,7 @@
return self.search_next(ctx, resume=True)
def search_next(self, ctx, resume):
+ # XXX missing jit support here
ppos = self.ppos
min = ctx.pat(ppos+1)
max = ctx.pat(ppos+2)
@@ -429,6 +462,7 @@
# ____________________________________________________________
@specializectx
+ at jit.unroll_safe
def sre_match(ctx, ppos, ptr, marks):
"""Returns either None or a MatchResult object. Usually we only need
the first result, but there is the case of REPEAT...UNTIL where we
@@ -437,6 +471,13 @@
while True:
op = ctx.pat(ppos)
ppos += 1
+ make_sure_not_modified(ctx.pattern)
+
+ #jit.jit_debug("sre_match", op, ppos, ptr)
+ #
+ # When using the JIT, calls to sre_match() must always have a constant
+ # (green) argument for 'ppos'. If not, the following assert fails.
+ jit.assert_green(op)
if op == OPCODE_FAILURE:
return
@@ -712,13 +753,23 @@
@specializectx
def find_repetition_end(ctx, ppos, ptr, maxcount):
end = ctx.end
- # adjust end
- if maxcount != 65535:
+ if maxcount <= 1:
+ if maxcount == 1 and ptr < end:
+ # Relatively common case: maxcount == 1. If we are not at the
+ # end of the string, it's done by a single direct check.
+ op = ctx.pat(ppos)
+ for op1, checkerfn in unroll_char_checker:
+ if op1 == op:
+ if checkerfn(ctx, ptr, ppos):
+ return ptr + 1
+ return ptr
+ elif maxcount != 65535:
+ # adjust end
end1 = ptr + maxcount
if end1 <= end:
end = end1
op = ctx.pat(ppos)
- for op1, (_, fre) in unroll_char_checker:
+ for op1, fre in unroll_fre_checker:
if op1 == op:
return fre(ctx, ptr, end, ppos)
raise Error("rsre.find_repetition_end[%d]" % op)
@@ -751,23 +802,60 @@
if checkerfn == match_ANY_ALL:
def fre(ctx, ptr, end, ppos):
return end
+ elif checkerfn == match_IN:
+ install_jitdriver_spec('MatchIn',
+ greens=['ppos', 'ctx.pattern'],
+ reds=['ptr', 'end', 'ctx'],
+ debugprint=(1, 0))
+ @specializectx
+ def fre(ctx, ptr, end, ppos):
+ while True:
+ ctx.jitdriver_MatchIn.jit_merge_point(ctx=ctx, ptr=ptr,
+ end=end, ppos=ppos)
+ if ptr < end and checkerfn(ctx, ptr, ppos):
+ ptr += 1
+ else:
+ return ptr
+ elif checkerfn == match_IN_IGNORE:
+ install_jitdriver_spec('MatchInIgnore',
+ greens=['ppos', 'ctx.pattern'],
+ reds=['ptr', 'end', 'ctx'],
+ debugprint=(1, 0))
+ @specializectx
+ def fre(ctx, ptr, end, ppos):
+ while True:
+ ctx.jitdriver_MatchInIgnore.jit_merge_point(ctx=ctx, ptr=ptr,
+ end=end, ppos=ppos)
+ if ptr < end and checkerfn(ctx, ptr, ppos):
+ ptr += 1
+ else:
+ return ptr
else:
+ # in the other cases, the fre() function is not JITted at all
+ # and is present as a residual call.
+ @specializectx
def fre(ctx, ptr, end, ppos):
while ptr < end and checkerfn(ctx, ptr, ppos):
ptr += 1
return ptr
- return checkerfn, fre
+ fre = func_with_new_name(fre, 'fre_' + checkerfn.__name__)
+ return fre
+
+unroll_char_checker = [
+ (OPCODE_ANY, match_ANY),
+ (OPCODE_ANY_ALL, match_ANY_ALL),
+ (OPCODE_IN, match_IN),
+ (OPCODE_IN_IGNORE, match_IN_IGNORE),
+ (OPCODE_LITERAL, match_LITERAL),
+ (OPCODE_LITERAL_IGNORE, match_LITERAL_IGNORE),
+ (OPCODE_NOT_LITERAL, match_NOT_LITERAL),
+ (OPCODE_NOT_LITERAL_IGNORE, match_NOT_LITERAL_IGNORE),
+ ]
+unroll_fre_checker = [(_op, _make_fre(_fn))
+ for (_op, _fn) in unroll_char_checker]
-unroll_char_checker = unrolling_iterable([
- (OPCODE_ANY, _make_fre(match_ANY)),
- (OPCODE_ANY_ALL, _make_fre(match_ANY_ALL)),
- (OPCODE_IN, _make_fre(match_IN)),
- (OPCODE_IN_IGNORE, _make_fre(match_IN_IGNORE)),
- (OPCODE_LITERAL, _make_fre(match_LITERAL)),
- (OPCODE_LITERAL_IGNORE, _make_fre(match_LITERAL_IGNORE)),
- (OPCODE_NOT_LITERAL, _make_fre(match_NOT_LITERAL)),
- (OPCODE_NOT_LITERAL_IGNORE, _make_fre(match_NOT_LITERAL_IGNORE)),
- ])
+unroll_char_checker = unrolling_iterable(unroll_char_checker)
+unroll_fre_checker = unrolling_iterable(unroll_fre_checker)
##### At dispatch
@@ -873,74 +961,139 @@
else:
return None
+install_jitdriver('Match',
+ greens=['ctx.pattern'], reds=['ctx'],
+ debugprint=(0,))
+
def match_context(ctx):
ctx.original_pos = ctx.match_start
if ctx.end < ctx.match_start:
return False
+ ctx.jitdriver_Match.jit_merge_point(ctx=ctx)
return sre_match(ctx, 0, ctx.match_start, None) is not None
def search_context(ctx):
ctx.original_pos = ctx.match_start
if ctx.end < ctx.match_start:
return False
- if ctx.pat(0) == OPCODE_INFO:
- if ctx.pat(2) & rsre_char.SRE_INFO_PREFIX and ctx.pat(5) > 1:
- return fast_search(ctx)
- return regular_search(ctx)
+ base = 0
+ charset = False
+ if ctx.pat(base) == OPCODE_INFO:
+ flags = ctx.pat(2)
+ if flags & rsre_char.SRE_INFO_PREFIX:
+ if ctx.pat(5) > 1:
+ return fast_search(ctx)
+ else:
+ charset = (flags & rsre_char.SRE_INFO_CHARSET)
+ base += 1 + ctx.pat(1)
+ if ctx.pat(base) == OPCODE_LITERAL:
+ return literal_search(ctx, base)
+ if charset:
+ return charset_search(ctx, base)
+ return regular_search(ctx, base)
+
+install_jitdriver('RegularSearch',
+ greens=['base', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(1, 0))
-def regular_search(ctx):
+def regular_search(ctx, base):
start = ctx.match_start
while start <= ctx.end:
- if sre_match(ctx, 0, start, None) is not None:
+ ctx.jitdriver_RegularSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base)
+ if sre_match(ctx, base, start, None) is not None:
ctx.match_start = start
return True
start += 1
return False
+install_jitdriver_spec("LiteralSearch",
+ greens=['base', 'character', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(2, 0, 1))
+ at specializectx
+def literal_search(ctx, base):
+ # pattern starts with a literal character. this is used
+ # for short prefixes, and if fast search is disabled
+ character = ctx.pat(base + 1)
+ base += 2
+ start = ctx.match_start
+ while start < ctx.end:
+ ctx.jitdriver_LiteralSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base, character=character)
+ if ctx.str(start) == character:
+ if sre_match(ctx, base, start + 1, None) is not None:
+ ctx.match_start = start
+ return True
+ start += 1
+ return False
+
+install_jitdriver_spec("CharsetSearch",
+ greens=['base', 'ctx.pattern'],
+ reds=['start', 'ctx'],
+ debugprint=(1, 0))
+ at specializectx
+def charset_search(ctx, base):
+ # pattern starts with a character from a known set
+ start = ctx.match_start
+ while start < ctx.end:
+ ctx.jitdriver_CharsetSearch.jit_merge_point(ctx=ctx, start=start,
+ base=base)
+ if rsre_char.check_charset(ctx.pattern, 5, ctx.str(start)):
+ if sre_match(ctx, base, start, None) is not None:
+ ctx.match_start = start
+ return True
+ start += 1
+ return False
+
+install_jitdriver_spec('FastSearch',
+ greens=['i', 'prefix_len', 'ctx.pattern'],
+ reds=['string_position', 'ctx'],
+ debugprint=(2, 0))
@specializectx
def fast_search(ctx):
# skips forward in a string as fast as possible using information from
# an optimization info block
# <1=skip> <2=flags> <3=min> <4=...>
# <5=length> <6=skip> <7=prefix data>
- flags = ctx.pat(2)
+ string_position = ctx.match_start
+ if string_position >= ctx.end:
+ return False
prefix_len = ctx.pat(5)
assert prefix_len >= 0
- prefix_skip = ctx.pat(6)
- assert prefix_skip >= 0
- overlap_offset = 7 + prefix_len - 1
- assert overlap_offset >= 0
- pattern_offset = ctx.pat(1) + 1
- ppos_start = pattern_offset + 2 * prefix_skip
- assert ppos_start >= 0
i = 0
- string_position = ctx.match_start
- end = ctx.end
- while string_position < end:
- while True:
- char_ord = ctx.str(string_position)
- if char_ord != ctx.pat(7 + i):
- if i == 0:
- break
- else:
- i = ctx.pat(overlap_offset + i)
- else:
- i += 1
- if i == prefix_len:
- # found a potential match
- start = string_position + 1 - prefix_len
- assert start >= 0
- ptr = start + prefix_skip
- if flags & rsre_char.SRE_INFO_LITERAL:
- # matched all of pure literal pattern
- ctx.match_start = start
- ctx.match_end = ptr
- ctx.match_marks = None
- return True
- if sre_match(ctx, ppos_start, ptr, None) is not None:
- ctx.match_start = start
- return True
- i = ctx.pat(overlap_offset + i)
- break
+ while True:
+ ctx.jitdriver_FastSearch.jit_merge_point(ctx=ctx,
+ string_position=string_position, i=i, prefix_len=prefix_len)
+ char_ord = ctx.str(string_position)
+ if char_ord != ctx.pat(7 + i):
+ if i > 0:
+ overlap_offset = prefix_len + (7 - 1)
+ i = ctx.pat(overlap_offset + i)
+ continue
+ else:
+ i += 1
+ if i == prefix_len:
+ # found a potential match
+ start = string_position + 1 - prefix_len
+ assert start >= 0
+ prefix_skip = ctx.pat(6)
+ ptr = start + prefix_skip
+ #flags = ctx.pat(2)
+ #if flags & rsre_char.SRE_INFO_LITERAL:
+ # # matched all of pure literal pattern
+ # ctx.match_start = start
+ # ctx.match_end = ptr
+ # ctx.match_marks = None
+ # return True
+ pattern_offset = ctx.pat(1) + 1
+ ppos_start = pattern_offset + 2 * prefix_skip
+ if sre_match(ctx, ppos_start, ptr, None) is not None:
+ ctx.match_start = start
+ return True
+ overlap_offset = prefix_len + (7 - 1)
+ i = ctx.pat(overlap_offset + i)
string_position += 1
- return False
+ if string_position >= ctx.end:
+ return False
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/rsre/test/test_match.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/rsre/test/test_match.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/rsre/test/test_match.py Sat Oct 30 16:41:17 2010
@@ -1,24 +1,49 @@
-import _sre, re, sre_compile
-from pypy.rlib.rsre import rsre_core
+import re
+from pypy.rlib.rsre import rsre_core, rsre_char
-def get_code(regexp, flags=0, allargs=False):
- class GotIt(Exception):
- pass
- def my_compile(pattern, flags, code, *args):
- print code
- raise GotIt(code, flags, args)
- saved = _sre.compile
- try:
- _sre.compile = my_compile
- try:
- sre_compile.compile(regexp, flags)
- except GotIt, e:
- pass
+def get_hacked_sre_compile(my_compile):
+ """Return a copy of the sre_compile module for which the _sre
+ module is a custom module that has _sre.compile == my_compile
+ and CODESIZE == rsre_char.CODESIZE.
+ """
+ import sre_compile, __builtin__, new
+ sre_hacked = new.module("_sre_hacked")
+ sre_hacked.compile = my_compile
+ sre_hacked.MAGIC = sre_compile.MAGIC
+ sre_hacked.CODESIZE = rsre_char.CODESIZE
+ sre_hacked.getlower = rsre_char.getlower
+ def my_import(name, *args):
+ if name == '_sre':
+ return sre_hacked
else:
- raise ValueError("did not reach _sre.compile()!")
+ return default_import(name, *args)
+ src = sre_compile.__file__
+ if src.lower().endswith('.pyc') or src.lower().endswith('.pyo'):
+ src = src[:-1]
+ mod = new.module("sre_compile_hacked")
+ default_import = __import__
+ try:
+ __builtin__.__import__ = my_import
+ execfile(src, mod.__dict__)
finally:
- _sre.compile = saved
+ __builtin__.__import__ = default_import
+ return mod
+
+class GotIt(Exception):
+ pass
+def my_compile(pattern, flags, code, *args):
+ print code
+ raise GotIt(code, flags, args)
+sre_compile_hacked = get_hacked_sre_compile(my_compile)
+
+def get_code(regexp, flags=0, allargs=False):
+ try:
+ sre_compile_hacked.compile(regexp, flags)
+ except GotIt, e:
+ pass
+ else:
+ raise ValueError("did not reach _sre.compile()!")
if allargs:
return e.args
else:
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/rstring.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/rstring.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/rstring.py Sat Oct 30 16:41:17 2010
@@ -54,6 +54,7 @@
self.l = []
def append(self, s):
+ assert isinstance(s, self._type)
self.l.append(s)
def append_slice(self, s, start, end):
@@ -63,11 +64,16 @@
def append_multiple_char(self, c, times):
self.l.append(c * times)
+ def getlength(self):
+ return len(self.build())
+
class StringBuilder(AbstractStringBuilder):
+ _type = str
def build(self):
return "".join(self.l)
class UnicodeBuilder(AbstractStringBuilder):
+ _type = unicode
def build(self):
return u''.join(self.l)
@@ -121,9 +127,12 @@
assert s_times.nonneg
return s_None
+ def method_getlength(self):
+ return SomeInteger(nonneg=True)
+
def method_build(self):
return SomeString()
-
+
def rtyper_makerepr(self, rtyper):
return rtyper.type_system.rbuilder.stringbuilder_repr
@@ -146,6 +155,9 @@
assert s_times.nonneg
return s_None
+ def method_getlength(self):
+ return SomeInteger(nonneg=True)
+
def method_build(self):
return SomeUnicodeString()
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/test/test_jit.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/test/test_jit.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/test/test_jit.py Sat Oct 30 16:41:17 2010
@@ -1,10 +1,17 @@
import py
+from pypy.conftest import option
from pypy.rlib.jit import hint, we_are_jitted, JitDriver, purefunction_promote
-from pypy.rlib.jit import JitHintError
+from pypy.rlib.jit import JitHintError, oopspec
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.rpython.lltypesystem import lltype
+def test_oopspec():
+ @oopspec('foobar')
+ def fn():
+ pass
+ assert fn.oopspec == 'foobar'
+
class BaseTestJIT(BaseRtypingTest):
def test_hint(self):
def f():
@@ -104,6 +111,26 @@
return n
py.test.raises(JitHintError, self.gengraph, fn, [int])
+ def test_green_field(self):
+ def get_printable_location(xfoo):
+ return str(ord(xfoo)) # xfoo must be annotated as a character
+ myjitdriver = JitDriver(greens=['x.foo'], reds=['n', 'x'],
+ get_printable_location=get_printable_location)
+ class A(object):
+ _immutable_fields_ = ['foo']
+ def fn(n):
+ x = A()
+ x.foo = chr(n)
+ while n > 0:
+ myjitdriver.can_enter_jit(x=x, n=n)
+ myjitdriver.jit_merge_point(x=x, n=n)
+ n -= 1
+ return n
+ t = self.gengraph(fn, [int])[0]
+ if option.view:
+ t.view()
+ # assert did not raise
+
class TestJITLLtype(BaseTestJIT, LLRtypeMixin):
pass
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rbigint.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rbigint.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rbigint.py Sat Oct 30 16:41:17 2010
@@ -112,6 +112,23 @@
assert rbigint.fromrarith_int(r_uint(2*sys.maxint+1)).eq(
rbigint.fromlong(2*sys.maxint+1))
+ def test_fromdecimalstr(self):
+ x = rbigint.fromdecimalstr("12345678901234567890523897987")
+ assert x.tolong() == 12345678901234567890523897987L
+ assert x.tobool() is True
+ x = rbigint.fromdecimalstr("+12345678901234567890523897987")
+ assert x.tolong() == 12345678901234567890523897987L
+ assert x.tobool() is True
+ x = rbigint.fromdecimalstr("-12345678901234567890523897987")
+ assert x.tolong() == -12345678901234567890523897987L
+ assert x.tobool() is True
+ x = rbigint.fromdecimalstr("+0")
+ assert x.tolong() == 0
+ assert x.tobool() is False
+ x = rbigint.fromdecimalstr("-0")
+ assert x.tolong() == 0
+ assert x.tobool() is False
+
def test_add(self):
x = 123456789123456789000000L
y = 123858582373821923936744221L
@@ -448,6 +465,35 @@
assert (rbigint.fromlong(-9**50).ulonglongmask() ==
r_ulonglong(-9**50))
+ def test_parse_digit_string(self):
+ from pypy.rlib.rbigint import parse_digit_string
+ class Parser:
+ def __init__(self, base, sign, digits):
+ self.base = base
+ self.sign = sign
+ self.next_digit = iter(digits + [-1]).next
+ x = parse_digit_string(Parser(10, 1, [6]))
+ assert x.eq(rbigint.fromint(6))
+ x = parse_digit_string(Parser(10, 1, [6, 2, 3]))
+ assert x.eq(rbigint.fromint(623))
+ x = parse_digit_string(Parser(10, -1, [6, 2, 3]))
+ assert x.eq(rbigint.fromint(-623))
+ x = parse_digit_string(Parser(16, 1, [0xA, 0x4, 0xF]))
+ assert x.eq(rbigint.fromint(0xA4F))
+ num = 0
+ for i in range(36):
+ x = parse_digit_string(Parser(36, 1, range(i)))
+ assert x.eq(rbigint.fromlong(num))
+ num = num * 36 + i
+ x = parse_digit_string(Parser(16, -1, range(15,-1,-1)*99))
+ assert x.eq(rbigint.fromlong(long('-0x' + 'FEDCBA9876543210'*99, 16)))
+ assert x.tobool() is True
+ x = parse_digit_string(Parser(7, 1, [0, 0, 0]))
+ assert x.tobool() is False
+ x = parse_digit_string(Parser(7, -1, [0, 0, 0]))
+ assert x.tobool() is False
+
+
BASE = 2 ** SHIFT
class TestTranslatable(object):
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rdynload.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rdynload.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rdynload.py Sat Oct 30 16:41:17 2010
@@ -1,15 +1,22 @@
from pypy.rlib.rdynload import *
-from pypy.rlib.libffi import get_libc_name
+from pypy.rlib.clibffi import get_libc_name
from pypy.rpython.lltypesystem import rffi, lltype
import py
class TestDLOperations:
def test_dlopen(self):
- py.test.raises(DLOpenError, "dlopen(rffi.str2charp('xxxxxxxxxxxx'))")
- assert dlopen(rffi.str2charp(get_libc_name()))
+ s = rffi.str2charp('xxxxxxxxxxxx')
+ py.test.raises(DLOpenError, "dlopen(s)")
+ rffi.free_charp(s)
+ #
+ s = rffi.str2charp(get_libc_name())
+ assert dlopen(s)
+ rffi.free_charp(s)
def test_dlsym(self):
- lib = dlopen(rffi.str2charp(get_libc_name()))
+ s = rffi.str2charp(get_libc_name())
+ lib = dlopen(s)
+ rffi.free_charp(s)
handle = rffi.cast(lltype.Ptr(lltype.FuncType([lltype.Signed],
lltype.Signed)), dlsym(lib, 'abs'))
assert 1 == handle(1)
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rsocket.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rsocket.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rsocket.py Sat Oct 30 16:41:17 2010
@@ -437,3 +437,31 @@
foo = self.serv.accept()
py.test.raises(SocketError, raise_error)
+def _test_cond_include(cond):
+ # Test that _rsocket_rffi is importable even on platforms where
+ # AF_PACKET or AF_NETLINK is not defined.
+ import re
+ from pypy.rlib import _rsocket_rffi
+ srcfile = _rsocket_rffi.__file__
+ if srcfile.lower().endswith('c') or srcfile.lower().endswith('o'):
+ srcfile = srcfile[:-1] # .pyc => .py
+ assert srcfile.lower().endswith('.py')
+ sourcelines = open(srcfile, 'rb').read().splitlines()
+ found = False
+ for i, line in enumerate(sourcelines):
+ line2 = re.sub(r"(\s*COND_HEADER\s*=)",
+ r"\1'#undef %s\\n'+" % cond,
+ line)
+ if line2 != line:
+ found = True
+ sourcelines[i] = line2
+ assert found
+ d = {}
+ sourcelines.append('')
+ exec '\n'.join(sourcelines) in d
+
+def test_no_AF_PACKET():
+ _test_cond_include('AF_PACKET')
+
+def test_no_AF_NETLINK():
+ _test_cond_include('AF_NETLINK')
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rstring.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rstring.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rstring.py Sat Oct 30 16:41:17 2010
@@ -29,6 +29,7 @@
s = StringBuilder()
s.append("a")
s.append("abc")
+ assert s.getlength() == len('aabc')
s.append("a")
s.append_slice("abc", 1, 2)
s.append_multiple_char('d', 4)
@@ -39,6 +40,7 @@
s.append(u'a')
s.append(u'abc')
s.append_slice(u'abcdef', 1, 2)
+ assert s.getlength() == len('aabcb')
s.append_multiple_char('d', 4)
assert s.build() == 'aabcbdddd'
assert isinstance(s.build(), unicode)
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/test/test_runicode.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/test/test_runicode.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/test/test_runicode.py Sat Oct 30 16:41:17 2010
@@ -76,7 +76,7 @@
assert start == startingpos
assert stop == endingpos
return u"42424242", stop
- return "", endingpos
+ return u"", endingpos
decoder = self.getdecoder(encoding)
if addstuff:
s += "some rest in ascii"
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rzlib.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rzlib.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rzlib.py Sat Oct 30 16:41:17 2010
@@ -189,6 +189,8 @@
assert unused3 == len('more_garbage')
assert data3 == ''
+ rzlib.deflateEnd(stream)
+
def test_decompress_max_length():
"""
@@ -205,6 +207,8 @@
assert finished2 is True
assert unused2 == 0
+ rzlib.deflateEnd(stream)
+
def test_cornercases():
"""
@@ -215,6 +219,7 @@
bytes += rzlib.compress(stream, "")
bytes += rzlib.compress(stream, "", rzlib.Z_FINISH)
assert zlib.decompress(bytes) == ""
+ rzlib.deflateEnd(stream)
stream = rzlib.inflateInit()
data, finished, unused = rzlib.decompress(stream, "")
@@ -228,3 +233,4 @@
assert finished is False
assert unused > 0
buf = buf[-unused:]
+ rzlib.deflateEnd(stream)
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/llinterp.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/llinterp.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/llinterp.py Sat Oct 30 16:41:17 2010
@@ -48,8 +48,7 @@
current_interpreter = None
- def __init__(self, typer, tracing=True, exc_data_ptr=None,
- malloc_check=True):
+ def __init__(self, typer, tracing=True, exc_data_ptr=None):
self.bindings = {}
self.typer = typer
# 'heap' is module or object that provides malloc, etc for lltype ops
@@ -57,9 +56,7 @@
self.exc_data_ptr = exc_data_ptr
self.frame_stack = []
self.tracer = None
- self.malloc_check = malloc_check
self.frame_class = LLFrame
- self.mallocs = {}
if tracing:
self.tracer = Tracer()
@@ -163,24 +160,6 @@
return self.exc_data_ptr
return None
- def remember_malloc(self, ptr, llframe):
- # err....
- self.mallocs[ptr._obj] = llframe
-
- def remember_free(self, ptr):
- try:
- del self.mallocs[ptr._obj]
- except KeyError:
- self._rehash_mallocs()
- del self.mallocs[ptr._obj]
-
- def _rehash_mallocs(self):
- # rehashing is needed because some objects' hash may change
- # when being turned to
- items = self.mallocs.items()
- self.mallocs = {}
- self.mallocs.update(items)
-
def _store_exception(self, exc):
raise PleaseOverwriteStoreException("You just invoked ll2ctypes callback without overwriting _store_exception on llinterpreter")
@@ -726,23 +705,23 @@
def op_malloc(self, obj, flags):
flavor = flags['flavor']
zero = flags.get('zero', False)
+ track_allocation = flags.get('track_allocation', True)
if flavor == "stack":
result = self.heap.malloc(obj, zero=zero, flavor='raw')
self.alloca_objects.append(result)
return result
- ptr = self.heap.malloc(obj, zero=zero, flavor=flavor)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_malloc(ptr, self)
+ ptr = self.heap.malloc(obj, zero=zero, flavor=flavor,
+ track_allocation=track_allocation)
return ptr
def op_malloc_varsize(self, obj, flags, size):
flavor = flags['flavor']
zero = flags.get('zero', False)
+ track_allocation = flags.get('track_allocation', True)
assert flavor in ('gc', 'raw')
try:
- ptr = self.heap.malloc(obj, size, zero=zero, flavor=flavor)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_malloc(ptr, self)
+ ptr = self.heap.malloc(obj, size, zero=zero, flavor=flavor,
+ track_allocation=track_allocation)
return ptr
except MemoryError:
self.make_llexception()
@@ -759,11 +738,10 @@
zero = flags.get('zero', False)
return self.heap.malloc_nonmovable(TYPE, size, zero=zero)
- def op_free(self, obj, flavor):
- assert isinstance(flavor, str)
- if flavor == 'raw' and self.llinterpreter.malloc_check:
- self.llinterpreter.remember_free(obj)
- self.heap.free(obj, flavor=flavor)
+ def op_free(self, obj, flags):
+ assert flags['flavor'] == 'raw'
+ track_allocation = flags.get('track_allocation', True)
+ self.heap.free(obj, flavor='raw', track_allocation=track_allocation)
def op_shrink_array(self, obj, smallersize):
return self.heap.shrink_array(obj, smallersize)
@@ -1037,6 +1015,13 @@
def op_stack_malloc(self, size): # mmh
raise NotImplementedError("backend only")
+ def op_track_alloc_start(self, addr):
+ # we don't do tracking at this level
+ checkadr(addr)
+
+ def op_track_alloc_stop(self, addr):
+ checkadr(addr)
+
# ____________________________________________________________
# Overflow-detecting variants
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/ll2ctypes.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/ll2ctypes.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/ll2ctypes.py Sat Oct 30 16:41:17 2010
@@ -69,7 +69,7 @@
PIECESIZE = 0x08000000
PIECES = 10
m = rmmap.mmap(-1, PIECES * PIECESIZE,
- rmmap.MAP_PRIVATE|rmmap.MAP_ANONYMOUS,
+ rmmap.MAP_PRIVATE|rmmap.MAP_ANONYMOUS|rmmap.MAP_NORESERVE,
rmmap.PROT_READ|rmmap.PROT_WRITE)
m.close = lambda : None # leak instead of giving a spurious
# error at CPython's shutdown
@@ -823,6 +823,8 @@
except (ValueError, OverflowError):
for tc in 'HIL':
if array(tc).itemsize == array('u').itemsize:
+ import struct
+ cobj &= 256 ** struct.calcsize(tc) - 1
llobj = array('u', array(tc, (cobj,)).tostring())[0]
break
else:
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llarena.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llarena.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llarena.py Sat Oct 30 16:41:17 2010
@@ -440,7 +440,7 @@
[rffi.INT],
rffi.INT,
sandboxsafe=True, _nowrapper=True)
- _dev_zero = rffi.str2charp('/dev/zero') # prebuilt
+ _dev_zero = rffi.str2charp_immortal('/dev/zero') # prebuilt
def clear_large_memory_chunk(baseaddr, size):
# on some Unixy platforms, reading from /dev/zero is the fastest way
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llmemory.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llmemory.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llmemory.py Sat Oct 30 16:41:17 2010
@@ -105,11 +105,13 @@
if (isinstance(self.TYPE, lltype.ContainerType)
and self.TYPE._gckind == 'gc'):
assert self.repeat == 1
- p = lltype.malloc(self.TYPE, flavor='raw', zero=zero)
+ p = lltype.malloc(self.TYPE, flavor='raw', zero=zero,
+ track_allocation=False)
return cast_ptr_to_adr(p)
else:
T = lltype.FixedSizeArray(self.TYPE, self.repeat)
- p = lltype.malloc(T, flavor='raw', zero=zero)
+ p = lltype.malloc(T, flavor='raw', zero=zero,
+ track_allocation=False)
array_adr = cast_ptr_to_adr(p)
return array_adr + ArrayItemsOffset(T)
@@ -288,7 +290,8 @@
count = 0
p = lltype.malloc(parenttype or self.TYPE, count,
immortal = self.TYPE._gckind == 'raw',
- zero = zero)
+ zero = zero,
+ track_allocation = False)
return cast_ptr_to_adr(p)
def raw_memcopy(self, srcadr, dstadr):
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lloperation.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lloperation.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lloperation.py Sat Oct 30 16:41:17 2010
@@ -403,6 +403,8 @@
'raw_load': LLOp(sideeffects=False),
'raw_store': LLOp(),
'stack_malloc': LLOp(), # mmh
+ 'track_alloc_start': LLOp(),
+ 'track_alloc_stop': LLOp(),
'adr_add': LLOp(canfold=True),
'adr_sub': LLOp(canfold=True),
'adr_delta': LLOp(canfold=True),
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lltype.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lltype.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lltype.py Sat Oct 30 16:41:17 2010
@@ -1,7 +1,3 @@
-import StringIO
-import traceback
-import sys
-
import py
from pypy.rlib.rarithmetic import (r_int, r_uint, intmask, r_singlefloat,
r_ulonglong, r_longlong, base_int,
@@ -10,25 +6,13 @@
from pypy.tool.uid import Hashable
from pypy.tool.tls import tlsobject
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
from types import NoneType
from sys import maxint
import weakref
TLS = tlsobject()
-# Track allocations to detect memory leaks
-# Don't track 'gc' and immortal mallocs
-TRACK_ALLOCATIONS = False
-ALLOCATED = identity_dict()
-
-def start_tracking_allocations():
- global TRACK_ALLOCATIONS
- TRACK_ALLOCATIONS = True
- ALLOCATED.clear()
-
-def stop_tracking_allocations():
- global TRACK_ALLOCATIONS
- TRACK_ALLOCATIONS = False
class _uninitialized(object):
def __init__(self, TYPE):
@@ -794,6 +778,8 @@
return llmemory.cast_adr_to_ptr(value, TGT)
elif TGT == llmemory.Address and isinstance(ORIG, Ptr):
return llmemory.cast_ptr_to_adr(value)
+ elif TGT == Signed and isinstance(ORIG, Ptr) and ORIG.TO._gckind == 'raw':
+ return llmemory.cast_adr_to_int(llmemory.cast_ptr_to_adr(value), 'symbolic')
raise TypeError("don't know how to cast from %r to %r" % (ORIG, TGT))
@@ -1380,41 +1366,21 @@
__slots__ = ('_TYPE',
'_parent_type', '_parent_index', '_keepparent',
'_wrparent',
- '__weakref__', '_traceback',
- '__storage')
+ '__weakref__',
+ '_storage')
- def __init__(self, TYPE, track_allocation=None):
+ def __init__(self, TYPE):
self._wrparent = None
self._TYPE = TYPE
self._storage = True # means "use default storage", as opposed to:
# None - container was freed
# - using ctypes
# (see ll2ctypes.py)
- if track_allocation is not False and TRACK_ALLOCATIONS:
- self._traceback = self._get_traceback()
- ALLOCATED[self] = None
- else:
- self._traceback = None
-
- def _get_traceback(self):
- frame = sys._getframe().f_back.f_back.f_back.f_back
- sio = StringIO.StringIO()
- traceback.print_stack(frame, file=sio)
- return sio.getvalue()
def _free(self):
self._check() # no double-frees
self._storage = None
- def _storage_get(self):
- return self.__storage
-
- def _storage_set(self, value):
- self.__storage = value
- if value is not True and self in ALLOCATED:
- del ALLOCATED[self]
- _storage = property(_storage_get, _storage_set)
-
def _was_freed(self):
if self._storage is None:
return True
@@ -1493,12 +1459,12 @@
__slots__ = ('_hash_cache_', '_compilation_info')
- def __new__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None, track_allocation=None):
+ def __new__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None):
my_variety = _struct_variety(TYPE._names)
return object.__new__(my_variety)
- def __init__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None, track_allocation=None):
- _parentable.__init__(self, TYPE, track_allocation)
+ def __init__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None):
+ _parentable.__init__(self, TYPE)
if n is not None and TYPE._arrayfld is None:
raise TypeError("%r is not variable-sized" % (TYPE,))
if n is None and TYPE._arrayfld is not None:
@@ -1506,8 +1472,7 @@
first, FIRSTTYPE = TYPE._first_struct()
for fld, typ in TYPE._flds.items():
if fld == TYPE._arrayfld:
- value = _array(typ, n, initialization=initialization, parent=self, parentindex=fld,
- track_allocation=track_allocation)
+ value = _array(typ, n, initialization=initialization, parent=self, parentindex=fld)
else:
value = typ._allocate(initialization=initialization, parent=self, parentindex=fld)
setattr(self, fld, value)
@@ -1568,12 +1533,12 @@
__slots__ = ('items',)
- def __init__(self, TYPE, n, initialization=None, parent=None, parentindex=None, track_allocation=None):
+ def __init__(self, TYPE, n, initialization=None, parent=None, parentindex=None):
if not isinstance(n, int):
raise TypeError, "array length must be an int"
if n < 0:
raise ValueError, "negative array length"
- _parentable.__init__(self, TYPE, track_allocation)
+ _parentable.__init__(self, TYPE)
try:
myrange = range(n)
except OverflowError:
@@ -1640,7 +1605,7 @@
_cache = weakref.WeakKeyDictionary() # parentarray -> {subarrays}
def __init__(self, TYPE, parent, baseoffset_or_fieldname):
- _parentable.__init__(self, TYPE, track_allocation=False)
+ _parentable.__init__(self, TYPE)
self._setparentstructure(parent, baseoffset_or_fieldname)
# Keep the parent array alive, we share the same allocation.
# Don't do it if we are inside a GC object, though -- it's someone
@@ -1648,6 +1613,13 @@
if typeOf(top_container(parent))._gckind == 'raw':
self._keepparent = parent
+ def __str__(self):
+ parent = self._wrparent()
+ if parent is None:
+ return '_subarray at %s in already freed' % (self._parent_index,)
+ return '_subarray at %r in %s' % (self._parent_index,
+ parent._TYPE)
+
def __repr__(self):
parent = self._wrparent()
if parent is None:
@@ -1861,8 +1833,9 @@
return id(self.value)
-def malloc(T, n=None, flavor='gc', immortal=False, zero=False):
- assert flavor != 'cpy'
+def malloc(T, n=None, flavor='gc', immortal=False, zero=False,
+ track_allocation=True):
+ assert flavor in ('gc', 'raw')
if zero or immortal:
initialization = 'example'
elif flavor == 'raw':
@@ -1870,9 +1843,9 @@
else:
initialization = 'malloc'
if isinstance(T, Struct):
- o = _struct(T, n, initialization=initialization, track_allocation=flavor == "raw" and not immortal)
+ o = _struct(T, n, initialization=initialization)
elif isinstance(T, Array):
- o = _array(T, n, initialization=initialization, track_allocation=flavor == "raw" and not immortal)
+ o = _array(T, n, initialization=initialization)
elif isinstance(T, OpaqueType):
assert n is None
o = _opaque(T, initialization=initialization)
@@ -1880,17 +1853,50 @@
raise TypeError, "malloc for Structs and Arrays only"
if T._gckind != 'gc' and not immortal and flavor.startswith('gc'):
raise TypeError, "gc flavor malloc of a non-GC non-immortal structure"
+ if flavor == "raw" and not immortal and track_allocation:
+ leakfinder.remember_malloc(o, framedepth=2)
solid = immortal or not flavor.startswith('gc') # immortal or non-gc case
return _ptr(Ptr(T), o, solid)
-def free(p, flavor):
+def free(p, flavor, track_allocation=True):
if flavor.startswith('gc'):
raise TypeError, "gc flavor free"
T = typeOf(p)
if not isinstance(T, Ptr) or p._togckind() != 'raw':
raise TypeError, "free(): only for pointers to non-gc containers"
+ if track_allocation:
+ leakfinder.remember_free(p._obj0)
p._obj0._free()
+def _make_scoped_allocator(T):
+ class ScopedAlloc:
+ def __init__(self, n=None, zero=False):
+ if n is None:
+ self.buf = malloc(T, flavor='raw', zero=zero)
+ else:
+ self.buf = malloc(T, n, flavor='raw', zero=zero)
+
+ def __enter__(self):
+ return self.buf
+
+ def __exit__(self, *args):
+ free(self.buf, flavor='raw')
+
+ ScopedAlloc.__name__ = 'ScopedAlloc_%s' % (T,)
+ return ScopedAlloc
+_make_scoped_allocator._annspecialcase_ = 'specialize:memo'
+
+def scoped_alloc(T, n=None, zero=False):
+ """Returns a context manager which handles allocation and
+ deallocation of temporary memory. Use it in a with statement::
+
+ with scoped_alloc(Array(Signed), 1) as array:
+ ...use array...
+ ...it's freed now.
+ """
+ return _make_scoped_allocator(T)(n=n, zero=zero)
+scoped_alloc._annspecialcase_ = 'specialize:arg(0)'
+
def functionptr(TYPE, name, **attrs):
if not isinstance(TYPE, FuncType):
raise TypeError, "functionptr() for FuncTypes only"
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rbuilder.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rbuilder.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rbuilder.py Sat Oct 30 16:41:17 2010
@@ -100,6 +100,10 @@
ll_builder.used = used
@staticmethod
+ def ll_getlength(ll_builder):
+ return ll_builder.used
+
+ @staticmethod
def ll_build(ll_builder):
final_size = ll_builder.used
assert final_size >= 0
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rclass.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rclass.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rclass.py Sat Oct 30 16:41:17 2010
@@ -329,16 +329,33 @@
fields['__class__'] = 'typeptr', get_type_repr(self.rtyper)
else:
# instance attributes
- if llfields is None:
- llfields = []
attrs = self.classdef.attrs.items()
attrs.sort()
+ myllfields = []
for name, attrdef in attrs:
if not attrdef.readonly:
r = self.rtyper.getrepr(attrdef.s_value)
mangled_name = 'inst_' + name
fields[name] = mangled_name, r
- llfields.append((mangled_name, r.lowleveltype))
+ myllfields.append((mangled_name, r.lowleveltype))
+
+ # Sort the instance attributes by decreasing "likely size",
+ # as reported by rffi.sizeof(), to minimize padding holes in C.
+ # Fields of the same size are sorted by name (by attrs.sort()
+ # above) just to minimize randomness.
+ def keysize((_, T)):
+ if T is lltype.Void:
+ return None
+ from pypy.rpython.lltypesystem.rffi import sizeof
+ try:
+ return -sizeof(T)
+ except StandardError:
+ return None
+ myllfields.sort(key = keysize)
+ if llfields is None:
+ llfields = myllfields
+ else:
+ llfields = llfields + myllfields
self.rbase = getinstancerepr(self.rtyper, self.classdef.basedef,
self.gcflavor)
@@ -403,7 +420,7 @@
return cast_pointer(self.lowleveltype, result)
def create_instance(self):
- return malloc(self.object_type, flavor=self.gcflavor)
+ return malloc(self.object_type, flavor=self.gcflavor, immortal=True)
def initialize_prebuilt_data(self, value, classdef, result):
if self.classdef is not None:
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rdict.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rdict.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rdict.py Sat Oct 30 16:41:17 2010
@@ -306,6 +306,13 @@
hop.exception_cannot_occur()
return hop.gendirectcall(ll_clear, v_dict)
+ def rtype_method_popitem(self, hop):
+ v_dict, = hop.inputargs(self)
+ r_tuple = hop.r_result
+ cTUPLE = hop.inputconst(lltype.Void, r_tuple.lowleveltype)
+ hop.exception_is_here()
+ return hop.gendirectcall(ll_popitem, cTUPLE, v_dict)
+
class __extend__(pairtype(DictRepr, rmodel.Repr)):
def rtype_getitem((r_dict, r_key), hop):
@@ -465,6 +472,10 @@
i = ll_dict_lookup(d, key, d.keyhash(key))
if not d.entries.valid(i):
raise KeyError
+ _ll_dict_del(d, i)
+ll_dict_delitem.oopspec = 'dict.delitem(d, key)'
+
+def _ll_dict_del(d, i):
d.entries.mark_deleted(i)
d.num_items -= 1
# clear the key and the value if they are GC pointers
@@ -481,7 +492,6 @@
num_entries = len(d.entries)
if num_entries > DICT_INITSIZE and d.num_items < num_entries / 4:
ll_dict_resize(d)
-ll_dict_delitem.oopspec = 'dict.delitem(d, key)'
def ll_dict_resize(d):
old_entries = d.entries
@@ -810,3 +820,26 @@
i = ll_dict_lookup(d, key, d.keyhash(key))
return d.entries.valid(i)
ll_contains.oopspec = 'dict.contains(d, key)'
+
+POPITEMINDEX = lltype.Struct('PopItemIndex', ('nextindex', lltype.Signed))
+global_popitem_index = lltype.malloc(POPITEMINDEX, zero=True, immortal=True)
+
+def ll_popitem(ELEM, dic):
+ entries = dic.entries
+ dmask = len(entries) - 1
+ base = global_popitem_index.nextindex
+ counter = 0
+ while counter <= dmask:
+ i = (base + counter) & dmask
+ counter += 1
+ if entries.valid(i):
+ break
+ else:
+ raise KeyError
+ global_popitem_index.nextindex += counter
+ entry = entries[i]
+ r = lltype.malloc(ELEM.TO)
+ r.item0 = recast(ELEM.TO.item0, entry.key)
+ r.item1 = recast(ELEM.TO.item1, entry.value)
+ _ll_dict_del(dic, i)
+ return r
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rffi.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rffi.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rffi.py Sat Oct 30 16:41:17 2010
@@ -607,6 +607,15 @@
return array
str2charp._annenforceargs_ = [strtype]
+ def str2charp_immortal(s):
+ "NOT_RPYTHON"
+ array = lltype.malloc(TYPEP.TO, len(s) + 1, flavor='raw',
+ immortal=True)
+ for i in range(len(s)):
+ array[i] = s[i]
+ array[len(s)] = lastchar
+ return array
+
def free_charp(cp):
lltype.free(cp, flavor='raw')
@@ -644,10 +653,18 @@
"""
Either free a non-moving buffer or keep the original storage alive.
"""
- if rgc.can_move(data):
+ # We cannot rely on rgc.can_move(data) here, because its result
+ # might have changed since get_nonmovingbuffer(). Instead we check
+ # if 'buf' points inside 'data'. This is only possible if we
+ # followed the 2nd case in get_nonmovingbuffer(); in the first case,
+ # 'buf' points to its own raw-malloced memory.
+ data = llstrtype(data)
+ data_start = cast_ptr_to_adr(data) + \
+ offsetof(STRTYPE, 'chars') + itemoffsetof(STRTYPE.chars, 0)
+ followed_2nd_path = (buf == cast(TYPEP, data_start))
+ keepalive_until_here(data)
+ if not followed_2nd_path:
lltype.free(buf, flavor='raw')
- else:
- keepalive_until_here(data)
# int -> (char*, str)
def alloc_buffer(count):
@@ -717,19 +734,19 @@
l = [cp[i] for i in range(size)]
return emptystr.join(l)
- return (str2charp, free_charp, charp2str,
+ return (str2charp, str2charp_immortal, free_charp, charp2str,
get_nonmovingbuffer, free_nonmovingbuffer,
alloc_buffer, str_from_buffer, keep_buffer_alive_until_here,
charp2strn, charpsize2str,
)
-(str2charp, free_charp, charp2str,
+(str2charp, str2charp_immortal, free_charp, charp2str,
get_nonmovingbuffer, free_nonmovingbuffer,
alloc_buffer, str_from_buffer, keep_buffer_alive_until_here,
charp2strn, charpsize2str,
) = make_string_mappings(str)
-(unicode2wcharp, free_wcharp, wcharp2unicode,
+(unicode2wcharp, unicode2wcharp_immortal, free_wcharp, wcharp2unicode,
get_nonmoving_unicodebuffer, free_nonmoving_unicodebuffer,
alloc_unicodebuffer, unicode_from_buffer, keep_unicodebuffer_alive_until_here,
wcharp2unicoden, wcharpsize2unicode,
@@ -916,3 +933,11 @@
"""
return cast(lltype.Signed, getattr(pdst, fieldname))
getintfield._annspecialcase_ = 'specialize:ll_and_arg(1)'
+
+class scoped_str2charp:
+ def __init__(self, value):
+ self.buf = str2charp(value)
+ def __enter__(self):
+ return self.buf
+ def __exit__(self, *args):
+ free_charp(self.buf)
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_ll2ctypes.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_ll2ctypes.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_ll2ctypes.py Sat Oct 30 16:41:17 2010
@@ -765,6 +765,7 @@
assert abs(float(b[1]) - 1.1) < 1E-6
assert isinstance(b[2], rffi.r_singlefloat)
assert abs(float(b[2]) - 2.2) < 1E-6
+ lltype.free(a, flavor='raw')
def test_different_signatures(self):
if sys.platform=='win32':
@@ -879,6 +880,7 @@
qsort(rffi.cast(rffi.VOIDP, a), 5, rffi.sizeof(rffi.INT), compare)
for i in range(5):
assert a[i] == i + 1
+ lltype.free(a, flavor='raw')
def test_array_type_bug(self):
A = lltype.Array(lltype.Signed)
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_llmemory.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_llmemory.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_llmemory.py Sat Oct 30 16:41:17 2010
@@ -324,12 +324,14 @@
p_t = lltype.malloc(T)
assert p_t.s == lltype.nullptr(S)
# raw malloc does not
- p_raw_t = lltype.malloc(T, flavor="raw")
- py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.s")
+ U = lltype.Struct("U", ('x', lltype.Signed))
+ p_raw_t = lltype.malloc(U, flavor="raw")
+ py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.x")
+ lltype.free(p_raw_t, flavor="raw")
# this sort of raw_malloc too
- p_raw_t = cast_adr_to_ptr(raw_malloc(sizeof(T)), lltype.Ptr(T))
- py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.s")
-
+ p_raw_t = cast_adr_to_ptr(raw_malloc(sizeof(U)), lltype.Ptr(U))
+ py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.x")
+
def test_raw_malloc_signed_bunch():
adr = raw_malloc(sizeof(lltype.Signed) * 50)
@@ -601,7 +603,8 @@
a = lltype.malloc(A, flavor='raw')
src = cast_ptr_to_adr(a) + itemoffsetof(A, 0)
raw_memclear(src, sizeof(lltype.Signed) * 0)
-
+ lltype.free(a, flavor="raw")
+
def test_nonneg():
S1 = lltype.GcStruct('S1', ('x', lltype.Float))
A1 = lltype.GcArray(lltype.Float)
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_lltype.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_lltype.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_lltype.py Sat Oct 30 16:41:17 2010
@@ -1,7 +1,9 @@
+from __future__ import with_statement
import py
from pypy.rpython.lltypesystem.lltype import *
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.tool.identity_dict import identity_dict
+from pypy.tool import leakfinder
def isweak(p, T):
try:
@@ -804,22 +806,20 @@
class TestTrackAllocation:
- def setup_method(self, func):
- start_tracking_allocations()
-
- def teardown_method(self, func):
- assert not lltype.ALLOCATED, "Memory was not correctly freed"
- stop_tracking_allocations()
+ def test_automatic_tracking(self):
+ # calls to start_tracking_allocations/stop_tracking_allocations
+ # should occur automatically from pypy/conftest.py. Check that.
+ assert leakfinder.TRACK_ALLOCATIONS
def test_track_allocation(self):
"""A malloc'd buffer fills the ALLOCATED dictionary"""
- assert lltype.TRACK_ALLOCATIONS
- assert not lltype.ALLOCATED
+ assert leakfinder.TRACK_ALLOCATIONS
+ assert not leakfinder.ALLOCATED
buf = malloc(Array(Signed), 1, flavor="raw")
- assert len(lltype.ALLOCATED) == 1
- assert lltype.ALLOCATED.keys() == [buf._obj]
+ assert len(leakfinder.ALLOCATED) == 1
+ assert leakfinder.ALLOCATED.keys() == [buf._obj]
free(buf, flavor="raw")
- assert not lltype.ALLOCATED
+ assert not leakfinder.ALLOCATED
def test_str_from_buffer(self):
"""gc-managed memory does not need to be freed"""
@@ -828,16 +828,28 @@
for i in range(size): raw_buf[i] = 'a'
rstr = rffi.str_from_buffer(raw_buf, gc_buf, size, size)
rffi.keep_buffer_alive_until_here(raw_buf, gc_buf)
- assert not lltype.ALLOCATED
+ assert not leakfinder.ALLOCATED
def test_leak_traceback(self):
"""Test info stored for allocated items"""
buf = malloc(Array(Signed), 1, flavor="raw")
- traceback = lltype.ALLOCATED.keys()[0]._traceback
+ traceback = leakfinder.ALLOCATED.values()[0]
lines = traceback.splitlines()
assert 'malloc(' in lines[-1] and 'flavor="raw")' in lines[-1]
- # XXX The traceback should not be too long
+ # The traceback should not be too long
print traceback
free(buf, flavor="raw")
+
+ def test_no_tracking(self):
+ p1 = malloc(Array(Signed), 1, flavor='raw', track_allocation=False)
+ p2 = malloc(Array(Signed), 1, flavor='raw', track_allocation=False)
+ free(p2, flavor='raw', track_allocation=False)
+ # p1 is not freed
+
+ def test_scoped_allocator(self):
+ with scoped_alloc(Array(Signed), 1) as array:
+ array[0] = -42
+ x = array[0]
+ assert x == -42
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_rffi.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_rffi.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/test/test_rffi.py Sat Oct 30 16:41:17 2010
@@ -9,7 +9,7 @@
from pypy.rpython.lltypesystem.rstr import STR
from pypy.rpython.lltypesystem import lltype
from pypy.tool.udir import udir
-from pypy.rpython.test.test_llinterp import interpret, MallocMismatch
+from pypy.rpython.test.test_llinterp import interpret
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.annotation.annrpython import RPythonAnnotator
from pypy.rpython.rtyper import RPythonTyper
@@ -787,3 +787,10 @@
mixann.getgraph(f2, [], s_None)
mixann.finish()
+def test_force_cast_unichar():
+ x = cast(lltype.UniChar, -1)
+ assert isinstance(x, unicode)
+ if sys.maxunicode == 65535:
+ assert cast(LONG, x) == 65535
+ else:
+ assert cast(LONG, cast(INT, x)) == -1
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/base.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/base.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/base.py Sat Oct 30 16:41:17 2010
@@ -20,12 +20,15 @@
prebuilt_gc_objects_are_static_roots = True
object_minimal_size = 0
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE):
+ def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
+ translated_to_c=True):
self.gcheaderbuilder = GCHeaderBuilder(self.HDR)
self.AddressStack = get_address_stack(chunk_size)
self.AddressDeque = get_address_deque(chunk_size)
self.AddressDict = AddressDict
self.config = config
+ assert isinstance(translated_to_c, bool)
+ self.translated_to_c = translated_to_c
def setup(self):
# all runtime mutable values' setup should happen here
@@ -79,7 +82,7 @@
def set_root_walker(self, root_walker):
self.root_walker = root_walker
- def write_barrier(self, addr_struct):
+ def write_barrier(self, newvalue, addr_struct):
pass
def statistics(self, index):
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/generation.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/generation.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/generation.py Sat Oct 30 16:41:17 2010
@@ -5,7 +5,6 @@
from pypy.rpython.memory.gc.base import read_from_env
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
from pypy.rpython.lltypesystem import lltype, llmemory, llarena
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rlib.objectmodel import free_non_gc_object
from pypy.rlib.debug import ll_assert
from pypy.rlib.debug import debug_print, debug_start, debug_stop
@@ -49,15 +48,17 @@
nursery_hash_base = -1
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
+ def __init__(self, config,
nursery_size=32*WORD,
min_nursery_size=32*WORD,
auto_nursery_size=False,
space_size=1024*WORD,
- max_space_size=sys.maxint//2+1):
- SemiSpaceGC.__init__(self, config, chunk_size = chunk_size,
+ max_space_size=sys.maxint//2+1,
+ **kwds):
+ SemiSpaceGC.__init__(self, config,
space_size = space_size,
- max_space_size = max_space_size)
+ max_space_size = max_space_size,
+ **kwds)
assert min_nursery_size <= nursery_size <= space_size // 2
self.initial_nursery_size = nursery_size
self.auto_nursery_size = auto_nursery_size
@@ -157,6 +158,14 @@
"odd-valued (i.e. tagged) pointer unexpected here")
return self.nursery <= addr < self.nursery_top
+ def appears_to_be_in_nursery(self, addr):
+ # same as is_in_nursery(), but may return True accidentally if
+ # 'addr' is a tagged pointer with just the wrong value.
+ if not self.translated_to_c:
+ if not self.is_valid_gc_object(addr):
+ return False
+ return self.nursery <= addr < self.nursery_top
+
def malloc_fixedsize_clear(self, typeid, size, can_collect,
has_finalizer=False, contains_weakptr=False):
if (has_finalizer or not can_collect or
@@ -326,7 +335,7 @@
addr = pointer.address[0]
newaddr = self.copy(addr)
pointer.address[0] = newaddr
- self.write_into_last_generation_obj(obj)
+ self.write_into_last_generation_obj(obj, newaddr)
# ____________________________________________________________
# Implementation of nursery-only collections
@@ -457,9 +466,9 @@
# "if addr_struct.int0 & JIT_WB_IF_FLAG: remember_young_pointer()")
JIT_WB_IF_FLAG = GCFLAG_NO_YOUNG_PTRS
- def write_barrier(self, addr_struct):
- if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS:
- self.remember_young_pointer(addr_struct)
+ def write_barrier(self, newvalue, addr_struct):
+ if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS:
+ self.remember_young_pointer(addr_struct, newvalue)
def _setup_wb(self):
DEBUG = self.DEBUG
@@ -470,23 +479,33 @@
# For x86, there is also an extra requirement: when the JIT calls
# remember_young_pointer(), it assumes that it will not touch the SSE
# registers, so it does not save and restore them (that's a *hack*!).
- def remember_young_pointer(addr_struct):
+ def remember_young_pointer(addr_struct, addr):
#llop.debug_print(lltype.Void, "\tremember_young_pointer",
# addr_struct, "<-", addr)
if DEBUG:
ll_assert(not self.is_in_nursery(addr_struct),
"nursery object with GCFLAG_NO_YOUNG_PTRS")
- self.old_objects_pointing_to_young.append(addr_struct)
- self.header(addr_struct).tid &= ~GCFLAG_NO_YOUNG_PTRS
- self.write_into_last_generation_obj(addr_struct)
+ #
+ # What is important in this function is that it *must*
+ # clear the flag GCFLAG_NO_YOUNG_PTRS from 'addr_struct'
+ # if 'addr' is in the nursery. It is ok if, accidentally,
+ # it also clears the flag in some more rare cases, like
+ # 'addr' being a tagged pointer whose value happens to be
+ # a large integer that fools is_in_nursery().
+ if self.appears_to_be_in_nursery(addr):
+ self.old_objects_pointing_to_young.append(addr_struct)
+ self.header(addr_struct).tid &= ~GCFLAG_NO_YOUNG_PTRS
+ self.write_into_last_generation_obj(addr_struct, addr)
remember_young_pointer._dont_inline_ = True
self.remember_young_pointer = remember_young_pointer
- def write_into_last_generation_obj(self, addr_struct):
+ def write_into_last_generation_obj(self, addr_struct, addr):
objhdr = self.header(addr_struct)
if objhdr.tid & GCFLAG_NO_HEAP_PTRS:
- objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
- self.last_generation_root_objects.append(addr_struct)
+ if (self.is_valid_gc_object(addr) and
+ not self.is_last_generation(addr)):
+ objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
+ self.last_generation_root_objects.append(addr_struct)
write_into_last_generation_obj._always_inline_ = True
def assume_young_pointers(self, addr_struct):
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/markcompact.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/markcompact.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/markcompact.py Sat Oct 30 16:41:17 2010
@@ -2,7 +2,6 @@
from pypy.rpython.memory.gc.base import MovingGCBase, read_from_env
from pypy.rlib.debug import ll_assert, have_debug_prints
from pypy.rlib.debug import debug_print, debug_start, debug_stop
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rpython.memory.support import get_address_stack, get_address_deque
from pypy.rpython.memory.support import AddressDict
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
@@ -86,9 +85,9 @@
free = NULL
next_collect_after = -1
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, space_size=4096,
- min_next_collect_after=128):
- MovingGCBase.__init__(self, config, chunk_size)
+ def __init__(self, config, space_size=4096,
+ min_next_collect_after=128, **kwds):
+ MovingGCBase.__init__(self, config, **kwds)
self.space_size = space_size
self.min_next_collect_after = min_next_collect_after
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/marksweep.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/marksweep.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/marksweep.py Sat Oct 30 16:41:17 2010
@@ -1,7 +1,6 @@
from pypy.rpython.lltypesystem.llmemory import raw_malloc, raw_free
from pypy.rpython.lltypesystem.llmemory import raw_memcopy, raw_memclear
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rpython.memory.support import get_address_stack
from pypy.rpython.memory.gcheader import GCHeaderBuilder
from pypy.rpython.lltypesystem import lltype, llmemory, rffi, llgroup
@@ -48,9 +47,9 @@
# translating to a real backend.
TRANSLATION_PARAMS = {'start_heap_size': 8*1024*1024} # XXX adjust
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, start_heap_size=4096):
+ def __init__(self, config, start_heap_size=4096, **kwds):
self.param_start_heap_size = start_heap_size
- GCBase.__init__(self, config, chunk_size)
+ GCBase.__init__(self, config, **kwds)
def setup(self):
GCBase.setup(self)
@@ -714,8 +713,8 @@
_alloc_flavor_ = "raw"
COLLECT_EVERY = 2000
- def __init__(self, chunk_size=DEFAULT_CHUNK_SIZE, start_heap_size=4096):
- MarkSweepGC.__init__(self, chunk_size, start_heap_size)
+ def __init__(self, config, **kwds):
+ MarkSweepGC.__init__(self, config, **kwds)
self.count_mallocs = 0
def maybe_collect(self):
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/minimark.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/minimark.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/minimark.py Sat Oct 30 16:41:17 2010
@@ -4,7 +4,6 @@
from pypy.rpython.lltypesystem.llmemory import raw_malloc_usage
from pypy.rpython.memory.gc.base import GCBase, MovingGCBase
from pypy.rpython.memory.gc import minimarkpage, base, generation
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask, r_uint
from pypy.rlib.rarithmetic import LONG_BIT_SHIFT
from pypy.rlib.debug import ll_assert, debug_print, debug_start, debug_stop
@@ -140,7 +139,7 @@
"large_object_gcptrs": 8250*WORD,
}
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE,
+ def __init__(self, config,
read_from_env=False,
nursery_size=32*WORD,
page_size=16*WORD,
@@ -150,8 +149,9 @@
card_page_indices=0,
large_object=8*WORD,
large_object_gcptrs=10*WORD,
- ArenaCollectionClass=None):
- MovingGCBase.__init__(self, config, chunk_size)
+ ArenaCollectionClass=None,
+ **kwds):
+ MovingGCBase.__init__(self, config, **kwds)
assert small_request_threshold % WORD == 0
self.read_from_env = read_from_env
self.nursery_size = nursery_size
@@ -636,6 +636,14 @@
"odd-valued (i.e. tagged) pointer unexpected here")
return self.nursery <= addr < self.nursery_top
+ def appears_to_be_in_nursery(self, addr):
+ # same as is_in_nursery(), but may return True accidentally if
+ # 'addr' is a tagged pointer with just the wrong value.
+ if not self.translated_to_c:
+ if not self.is_valid_gc_object(addr):
+ return False
+ return self.nursery <= addr < self.nursery_top
+
def is_forwarded(self, obj):
"""Returns True if the nursery obj is marked as forwarded.
Implemented a bit obscurely by checking an unrelated flag
@@ -726,16 +734,16 @@
def JIT_max_size_of_young_obj(cls):
return cls.TRANSLATION_PARAMS['large_object']
- def write_barrier(self, addr_struct):
+ def write_barrier(self, newvalue, addr_struct):
if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS:
- self.remember_young_pointer(addr_struct)
+ self.remember_young_pointer(addr_struct, newvalue)
- def write_barrier_from_array(self, addr_array, index):
+ def write_barrier_from_array(self, newvalue, addr_array, index):
if self.header(addr_array).tid & GCFLAG_NO_YOUNG_PTRS:
if self.card_page_indices > 0: # <- constant-folded
self.remember_young_pointer_from_array(addr_array, index)
else:
- self.remember_young_pointer(addr_array)
+ self.remember_young_pointer(addr_array, newvalue)
def _init_writebarrier_logic(self):
DEBUG = self.DEBUG
@@ -746,27 +754,28 @@
# For x86, there is also an extra requirement: when the JIT calls
# remember_young_pointer(), it assumes that it will not touch the SSE
# registers, so it does not save and restore them (that's a *hack*!).
- def remember_young_pointer(addr_struct):
+ def remember_young_pointer(addr_struct, newvalue):
# 'addr_struct' is the address of the object in which we write.
+ # 'newvalue' is the address that we are going to write in there.
if DEBUG:
ll_assert(not self.is_in_nursery(addr_struct),
"nursery object with GCFLAG_NO_YOUNG_PTRS")
#
- # We assume that what we are writing is a pointer to the nursery
- # (and don't care for the fact that this new pointer may not
- # actually point to the nursery, which seems ok). What we need is
+ # If it seems that what we are writing is a pointer to the nursery
+ # (as checked with appears_to_be_in_nursery()), then we need
# to remove the flag GCFLAG_NO_YOUNG_PTRS and add the old object
# to the list 'old_objects_pointing_to_young'. We know that
# 'addr_struct' cannot be in the nursery, because nursery objects
# never have the flag GCFLAG_NO_YOUNG_PTRS to start with.
- self.old_objects_pointing_to_young.append(addr_struct)
objhdr = self.header(addr_struct)
- objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
+ if self.appears_to_be_in_nursery(newvalue):
+ self.old_objects_pointing_to_young.append(addr_struct)
+ objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
#
# Second part: if 'addr_struct' is actually a prebuilt GC
# object and it's the first time we see a write to it, we
# add it to the list 'prebuilt_root_objects'. Note that we
- # do it even in the (rare?) case of 'addr' being another
+ # do it even in the (rare?) case of 'addr' being NULL or another
# prebuilt object, to simplify code.
if objhdr.tid & GCFLAG_NO_HEAP_PTRS:
objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
@@ -780,16 +789,24 @@
def _init_writebarrier_with_card_marker(self):
+ DEBUG = self.DEBUG
def remember_young_pointer_from_array(addr_array, index):
# 'addr_array' is the address of the object in which we write,
# which must have an array part; 'index' is the index of the
# item that is (or contains) the pointer that we write.
+ if DEBUG:
+ ll_assert(not self.is_in_nursery(addr_array),
+ "nursery array with GCFLAG_NO_YOUNG_PTRS")
objhdr = self.header(addr_array)
if objhdr.tid & GCFLAG_HAS_CARDS == 0:
#
- # no cards, use default logic. The 'nocard_logic()' is just
- # 'remember_young_pointer()', but forced to be inlined here.
- nocard_logic(addr_array)
+ # no cards, use default logic. Mostly copied from above.
+ self.old_objects_pointing_to_young.append(addr_array)
+ objhdr = self.header(addr_array)
+ objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS
+ if objhdr.tid & GCFLAG_NO_HEAP_PTRS:
+ objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS
+ self.prebuilt_root_objects.append(addr_array)
return
#
# 'addr_array' is a raw_malloc'ed array with card markers
@@ -807,17 +824,15 @@
return
#
# We set the flag (even if the newly written address does not
- # actually point to the nursery -- like remember_young_pointer()).
+ # actually point to the nursery, which seems to be ok -- actually
+ # it seems more important that remember_young_pointer_from_array()
+ # does not take 3 arguments).
addr_byte.char[0] = chr(byte | bitmask)
#
if objhdr.tid & GCFLAG_CARDS_SET == 0:
self.old_objects_with_cards_set.append(addr_array)
objhdr.tid |= GCFLAG_CARDS_SET
- nocard_logic = func_with_new_name(self.remember_young_pointer,
- 'remember_young_pointer_nocard')
- del nocard_logic._dont_inline_
- nocard_logic._always_inline_ = True
remember_young_pointer_from_array._dont_inline_ = True
self.remember_young_pointer_from_array = (
remember_young_pointer_from_array)
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/minimarkpage.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/minimarkpage.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/minimarkpage.py Sat Oct 30 16:41:17 2010
@@ -100,11 +100,14 @@
# allocation of the given size.
length = small_request_threshold / WORD + 1
self.page_for_size = lltype.malloc(rffi.CArray(PAGE_PTR), length,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
self.full_page_for_size = lltype.malloc(rffi.CArray(PAGE_PTR), length,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
self.nblocks_for_size = lltype.malloc(rffi.CArray(lltype.Signed),
- length, flavor='raw')
+ length, flavor='raw',
+ immortal=True)
self.hdrsize = llmemory.raw_malloc_usage(llmemory.sizeof(PAGE_HEADER))
assert page_size > self.hdrsize
self.nblocks_for_size[0] = 0 # unused
@@ -114,11 +117,13 @@
self.max_pages_per_arena = arena_size // page_size
self.arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
self.max_pages_per_arena,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
# this is used in mass_free() only
self.old_arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR),
self.max_pages_per_arena,
- flavor='raw', zero=True)
+ flavor='raw', zero=True,
+ immortal=True)
#
# the arena currently consumed; it must have at least one page
# available, or be NULL. The arena object that we point to is
@@ -281,7 +286,7 @@
npages = (arena_end - firstpage) // self.page_size
#
# Allocate an ARENA object and initialize it
- arena = lltype.malloc(ARENA, flavor='raw')
+ arena = lltype.malloc(ARENA, flavor='raw', track_allocation=False)
arena.base = arena_base
arena.nfreepages = 0 # they are all uninitialized pages
arena.totalpages = npages
@@ -332,7 +337,7 @@
#
# The whole arena is empty. Free it.
llarena.arena_free(arena.base)
- lltype.free(arena, flavor='raw')
+ lltype.free(arena, flavor='raw', track_allocation=False)
#
else:
# Insert 'arena' in the correct arenas_lists[n]
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/semispace.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/semispace.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/semispace.py Sat Oct 30 16:41:17 2010
@@ -1,7 +1,6 @@
from pypy.rpython.lltypesystem.llmemory import raw_malloc, raw_free
from pypy.rpython.lltypesystem.llmemory import raw_memcopy, raw_memclear
from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage
-from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE
from pypy.rpython.memory.support import get_address_stack, get_address_deque
from pypy.rpython.memory.support import AddressDict
from pypy.rpython.lltypesystem import lltype, llmemory, llarena, rffi, llgroup
@@ -59,11 +58,11 @@
# translating to a real backend.
TRANSLATION_PARAMS = {'space_size': 8*1024*1024} # XXX adjust
- def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, space_size=4096,
- max_space_size=sys.maxint//2+1):
+ def __init__(self, config, space_size=4096, max_space_size=sys.maxint//2+1,
+ **kwds):
self.param_space_size = space_size
self.param_max_space_size = max_space_size
- MovingGCBase.__init__(self, config, chunk_size)
+ MovingGCBase.__init__(self, config, **kwds)
def setup(self):
#self.total_collection_time = 0.0
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/test/test_direct.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/test/test_direct.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/test/test_direct.py Sat Oct 30 16:41:17 2010
@@ -70,6 +70,7 @@
GC_PARAMS = self.GC_PARAMS.copy()
if hasattr(meth, 'GC_PARAMS'):
GC_PARAMS.update(meth.GC_PARAMS)
+ GC_PARAMS['translated_to_c'] = False
self.gc = self.GCClass(config, **GC_PARAMS)
self.gc.DEBUG = True
self.rootwalker = DirectRootWalker(self)
@@ -86,17 +87,19 @@
def write(self, p, fieldname, newvalue):
if self.gc.needs_write_barrier:
+ newaddr = llmemory.cast_ptr_to_adr(newvalue)
addr_struct = llmemory.cast_ptr_to_adr(p)
- self.gc.write_barrier(addr_struct)
+ self.gc.write_barrier(newaddr, addr_struct)
setattr(p, fieldname, newvalue)
def writearray(self, p, index, newvalue):
if self.gc.needs_write_barrier:
+ newaddr = llmemory.cast_ptr_to_adr(newvalue)
addr_struct = llmemory.cast_ptr_to_adr(p)
if hasattr(self.gc, 'write_barrier_from_array'):
- self.gc.write_barrier_from_array(addr_struct, index)
+ self.gc.write_barrier_from_array(newaddr, addr_struct, index)
else:
- self.gc.write_barrier(addr_struct)
+ self.gc.write_barrier(newaddr, addr_struct)
p[index] = newvalue
def malloc(self, TYPE, n=None):
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/asmgcroot.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/asmgcroot.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/asmgcroot.py Sat Oct 30 16:41:17 2010
@@ -75,7 +75,8 @@
key = (TYPE, num)
if key not in sradict:
CONTAINER = lltype.FixedSizeArray(TYPE, 1)
- p = lltype.malloc(CONTAINER, flavor='raw', zero=True)
+ p = lltype.malloc(CONTAINER, flavor='raw', zero=True,
+ immortal=True)
sradict[key] = Constant(p, lltype.Ptr(CONTAINER))
sra.append(sradict[key])
#
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/framework.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/framework.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/framework.py Sat Oct 30 16:41:17 2010
@@ -426,6 +426,7 @@
if GCClass.needs_write_barrier:
self.write_barrier_ptr = getfn(GCClass.write_barrier.im_func,
[s_gc,
+ annmodel.SomeAddress(),
annmodel.SomeAddress()],
annmodel.s_None,
inline=True)
@@ -434,13 +435,15 @@
# func should not be a bound method, but a real function
assert isinstance(func, types.FunctionType)
self.write_barrier_failing_case_ptr = getfn(func,
- [annmodel.SomeAddress()],
+ [annmodel.SomeAddress(),
+ annmodel.SomeAddress()],
annmodel.s_None)
func = getattr(GCClass, 'write_barrier_from_array', None)
if func is not None:
self.write_barrier_from_array_ptr = getfn(func.im_func,
[s_gc,
annmodel.SomeAddress(),
+ annmodel.SomeAddress(),
annmodel.SomeInteger()],
annmodel.s_None,
inline=True)
@@ -1021,6 +1024,8 @@
and not isinstance(v_newvalue, Constant)
and v_struct.concretetype.TO._gckind == "gc"
and hop.spaceop not in self.clean_sets):
+ v_newvalue = hop.genop("cast_ptr_to_adr", [v_newvalue],
+ resulttype = llmemory.Address)
v_structaddr = hop.genop("cast_ptr_to_adr", [v_struct],
resulttype = llmemory.Address)
if (self.write_barrier_from_array_ptr is not None and
@@ -1030,12 +1035,14 @@
assert v_index.concretetype == lltype.Signed
hop.genop("direct_call", [self.write_barrier_from_array_ptr,
self.c_const_gc,
+ v_newvalue,
v_structaddr,
v_index])
else:
self.write_barrier_calls += 1
hop.genop("direct_call", [self.write_barrier_ptr,
self.c_const_gc,
+ v_newvalue,
v_structaddr])
hop.rename('bare_' + opname)
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/transform.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/transform.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/transform.py Sat Oct 30 16:41:17 2010
@@ -430,7 +430,8 @@
return self.parenttransformer.gct_malloc_varsize(hop)
def gct_free(self, hop):
- flavor = hop.spaceop.args[1].value
+ flags = hop.spaceop.args[1].value
+ flavor = flags['flavor']
assert flavor == 'raw'
return self.parenttransformer.gct_free(hop)
@@ -532,6 +533,8 @@
resulttype=llmemory.Address)
if flags.get('zero'):
hop.genop("raw_memclear", [v_raw, c_size])
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_start", [v_raw])
return v_raw
def gct_fv_stack_malloc(self, hop, flags, TYPE, c_size):
@@ -602,15 +605,20 @@
[self.raw_malloc_varsize_ptr, v_length,
c_const_size, c_item_size, c_offset_to_length],
resulttype=llmemory.Address)
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_start", [v_raw])
return v_raw
def gct_free(self, hop):
op = hop.spaceop
- flavor = op.args[1].value
+ flags = op.args[1].value
+ flavor = flags['flavor']
v = op.args[0]
assert flavor != 'cpy', "cannot free CPython objects directly"
if flavor == 'raw':
v = hop.genop("cast_ptr_to_adr", [v], resulttype=llmemory.Address)
+ if flags.get('track_allocation', True):
+ hop.genop("track_alloc_stop", [v])
hop.genop('raw_free', [v])
else:
assert False, "%s has no support for free with flavor %r" % (self, flavor)
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gcwrapper.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gcwrapper.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gcwrapper.py Sat Oct 30 16:41:17 2010
@@ -9,7 +9,10 @@
def __init__(self, llinterp, flowgraphs, gc_class, GC_PARAMS={}):
translator = llinterp.typer.annotator.translator
config = translator.config.translation
- self.gc = gc_class(config, chunk_size = 10, **GC_PARAMS)
+ self.gc = gc_class(config,
+ chunk_size = 10,
+ translated_to_c = False,
+ **GC_PARAMS)
self.gc.set_root_walker(LLInterpRootWalker(self))
self.gc.DEBUG = True
self.llinterp = llinterp
@@ -39,7 +42,8 @@
#
# Interface for the llinterp
#
- def malloc(self, TYPE, n=None, flavor='gc', zero=False):
+ def malloc(self, TYPE, n=None, flavor='gc', zero=False,
+ track_allocation=True):
if flavor == 'gc':
typeid = self.get_type_id(TYPE)
addr = self.gc.malloc(typeid, n, zero=zero)
@@ -48,7 +52,8 @@
gctypelayout.zero_gc_pointers(result)
return result
else:
- return lltype.malloc(TYPE, n, flavor=flavor, zero=zero)
+ return lltype.malloc(TYPE, n, flavor=flavor, zero=zero,
+ track_allocation=track_allocation)
def malloc_nonmovable(self, TYPE, n=None, zero=False):
typeid = self.get_type_id(TYPE)
@@ -66,9 +71,10 @@
return self.gc.shrink_array(addr, smallersize)
return False
- def free(self, TYPE, flavor='gc'):
+ def free(self, TYPE, flavor='gc', track_allocation=True):
assert flavor != 'gc'
- return lltype.free(TYPE, flavor=flavor)
+ return lltype.free(TYPE, flavor=flavor,
+ track_allocation=track_allocation)
def setfield(self, obj, fieldname, fieldvalue):
STRUCT = lltype.typeOf(obj).TO
@@ -94,6 +100,7 @@
assert (type(index) is int # <- fast path
or lltype.typeOf(index) == lltype.Signed)
self.gc.write_barrier_from_array(
+ llmemory.cast_ptr_to_adr(newvalue),
llmemory.cast_ptr_to_adr(toplevelcontainer),
index)
wb = False
@@ -101,6 +108,7 @@
#
if wb:
self.gc.write_barrier(
+ llmemory.cast_ptr_to_adr(newvalue),
llmemory.cast_ptr_to_adr(toplevelcontainer))
llheap.setinterior(toplevelcontainer, inneraddr, INNERTYPE, newvalue)
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/support.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/memory/support.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/support.py Sat Oct 30 16:41:17 2010
@@ -30,7 +30,8 @@
# we zero-initialize the chunks to make the translation
# backends happy, but we don't need to do it at run-time.
zero = not we_are_translated()
- return lltype.malloc(CHUNK, flavor="raw", zero=zero)
+ return lltype.malloc(CHUNK, flavor="raw", zero=zero,
+ track_allocation=False)
result = self.free_list
self.free_list = result.next
@@ -44,7 +45,7 @@
# Don't cache the old chunks but free them immediately.
# Helps debugging, and avoids that old chunks full of
# addresses left behind by a test end up in genc...
- lltype.free(chunk, flavor="raw")
+ lltype.free(chunk, flavor="raw", track_allocation=False)
unused_chunks = FreeList()
cache[chunk_size] = unused_chunks, null_chunk
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/test/test_gc.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/memory/test/test_gc.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/test/test_gc.py Sat Oct 30 16:41:17 2010
@@ -368,6 +368,14 @@
res = self.interpret(f, [4, 42])
assert res == 12
+ def test_print_leak(self):
+ def f(n):
+ for i in range(n):
+ print i
+ return 42
+ res = self.interpret(f, [10])
+ assert res == 42
+
def test_weakref_across_minor_collection(self):
import weakref
class A:
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/test/test_transformed_gc.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/memory/test/test_transformed_gc.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/test/test_transformed_gc.py Sat Oct 30 16:41:17 2010
@@ -906,7 +906,8 @@
gcname = "marksweep"
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
- GC_PARAMS = {'start_heap_size': 1024*WORD }
+ GC_PARAMS = {'start_heap_size': 1024*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
@@ -1144,7 +1145,8 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.marksweep import PrintingMarkSweepGC as GCClass
- GC_PARAMS = {'start_heap_size': 1024*WORD }
+ GC_PARAMS = {'start_heap_size': 1024*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestSemiSpaceGC(GenericMovingGCTests):
@@ -1154,7 +1156,8 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.semispace import SemiSpaceGC as GCClass
- GC_PARAMS = {'space_size': 512*WORD}
+ GC_PARAMS = {'space_size': 512*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestMarkCompactGC(GenericMovingGCTests):
@@ -1163,7 +1166,8 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.markcompact import MarkCompactGC as GCClass
- GC_PARAMS = {'space_size': 4096*WORD}
+ GC_PARAMS = {'space_size': 4096*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestGenerationGC(GenericMovingGCTests):
@@ -1175,7 +1179,8 @@
from pypy.rpython.memory.gc.generation import GenerationGC as \
GCClass
GC_PARAMS = {'space_size': 512*WORD,
- 'nursery_size': 32*WORD}
+ 'nursery_size': 32*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
def define_weakref_across_minor_collection(cls):
@@ -1372,7 +1377,8 @@
GenerationGC._teardown(self)
GC_PARAMS = {'space_size': 512*WORD,
- 'nursery_size': 128*WORD}
+ 'nursery_size': 128*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
def define_working_nursery(cls):
@@ -1404,7 +1410,8 @@
from pypy.rpython.memory.gc.hybrid import HybridGC as GCClass
GC_PARAMS = {'space_size': 512*WORD,
'nursery_size': 32*WORD,
- 'large_object': 8*WORD}
+ 'large_object': 8*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
def define_ref_from_rawmalloced_to_regular(cls):
@@ -1477,6 +1484,7 @@
'large_object': 8*WORD,
'large_object_gcptrs': 10*WORD,
'card_page_indices': 4,
+ 'translated_to_c': False,
}
root_stack_depth = 200
@@ -1585,7 +1593,8 @@
gcname = "marksweep"
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
- GC_PARAMS = {'start_heap_size': 1024*WORD }
+ GC_PARAMS = {'start_heap_size': 1024*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestHybridTaggedPointerGC(TaggedPointerGCTests):
@@ -1596,7 +1605,8 @@
from pypy.rpython.memory.gc.generation import GenerationGC as \
GCClass
GC_PARAMS = {'space_size': 512*WORD,
- 'nursery_size': 32*WORD}
+ 'nursery_size': 32*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
class TestMarkCompactTaggedpointerGC(TaggedPointerGCTests):
@@ -1605,5 +1615,6 @@
class gcpolicy(gc.FrameworkGcPolicy):
class transformerclass(framework.FrameworkGCTransformer):
from pypy.rpython.memory.gc.markcompact import MarkCompactGC as GCClass
- GC_PARAMS = {'space_size': 4096*WORD}
+ GC_PARAMS = {'space_size': 4096*WORD,
+ 'translated_to_c': False}
root_stack_depth = 200
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/module/ll_time.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/module/ll_time.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/module/ll_time.py Sat Oct 30 16:41:17 2010
@@ -108,7 +108,7 @@
errcode = -1
if self.GETTIMEOFDAY_NO_TZ:
- errcode = g_gettimeofday(t)
+ errcode = c_gettimeofday(t)
else:
errcode = c_gettimeofday(t, void)
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/rbuilder.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/rbuilder.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/rbuilder.py Sat Oct 30 16:41:17 2010
@@ -36,8 +36,12 @@
hop.exception_cannot_occur()
return hop.gendirectcall(self.ll_append_multiple_char, *vlist)
+ def rtype_method_getlength(self, hop):
+ vlist = hop.inputargs(self)
+ hop.exception_cannot_occur()
+ return hop.gendirectcall(self.ll_getlength, *vlist)
+
def rtype_method_build(self, hop):
vlist = hop.inputargs(self)
hop.exception_cannot_occur()
return hop.gendirectcall(self.ll_build, *vlist)
-
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/rbuiltin.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/rbuiltin.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/rbuiltin.py Sat Oct 30 16:41:17 2010
@@ -345,17 +345,22 @@
BUILTIN_TYPER[object.__init__] = rtype_object__init__
# annotation of low-level types
-def rtype_malloc(hop, i_flavor=None, i_zero=None):
+def rtype_malloc(hop, i_flavor=None, i_zero=None, i_track_allocation=None):
assert hop.args_s[0].is_constant()
vlist = [hop.inputarg(lltype.Void, arg=0)]
opname = 'malloc'
- v_flavor, v_zero = parse_kwds(hop, (i_flavor, lltype.Void), (i_zero, None))
+ v_flavor, v_zero, v_track_allocation = parse_kwds(hop,
+ (i_flavor, lltype.Void),
+ (i_zero, None),
+ (i_track_allocation, None))
flags = {'flavor': 'gc'}
if v_flavor is not None:
flags['flavor'] = v_flavor.value
if i_zero is not None:
flags['zero'] = v_zero.value
+ if i_track_allocation is not None:
+ flags['track_allocation'] = v_track_allocation.value
vlist.append(hop.inputconst(lltype.Void, flags))
if hop.nb_args == 2:
@@ -366,10 +371,19 @@
hop.exception_is_here()
return hop.genop(opname, vlist, resulttype = hop.r_result.lowleveltype)
-def rtype_free(hop, i_flavor):
- assert i_flavor == 1
+def rtype_free(hop, i_flavor, i_track_allocation=None):
+ vlist = [hop.inputarg(hop.args_r[0], arg=0)]
+ v_flavor, v_track_allocation = parse_kwds(hop,
+ (i_flavor, lltype.Void),
+ (i_track_allocation, None))
+ #
+ assert v_flavor is not None and v_flavor.value == 'raw'
+ flags = {'flavor': 'raw'}
+ if i_track_allocation is not None:
+ flags['track_allocation'] = v_track_allocation.value
+ vlist.append(hop.inputconst(lltype.Void, flags))
+ #
hop.exception_cannot_occur()
- vlist = hop.inputargs(hop.args_r[0], lltype.Void)
hop.genop('free', vlist)
def rtype_const_result(hop):
@@ -584,8 +598,9 @@
vinst, = hop.inputargs(hop.args_r[0])
flavor = hop.args_r[0].gcflavor
assert flavor != 'gc'
- cflavor = hop.inputconst(lltype.Void, flavor)
- return hop.genop('free', [vinst, cflavor])
+ flags = {'flavor': flavor}
+ cflags = hop.inputconst(lltype.Void, flags)
+ return hop.genop('free', [vinst, cflags])
BUILTIN_TYPER[objectmodel.free_non_gc_object] = rtype_free_non_gc_object
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/rpbc.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/rpbc.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/rpbc.py Sat Oct 30 16:41:17 2010
@@ -256,6 +256,8 @@
def convert_const(self, value):
if isinstance(value, types.MethodType) and value.im_self is None:
value = value.im_func # unbound method -> bare function
+ elif isinstance(value, staticmethod):
+ value = value.__get__(42) # hackish, get the function wrapped by staticmethod
if self.lowleveltype is Void:
return None
if value is None:
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/rtyper.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/rtyper.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/rtyper.py Sat Oct 30 16:41:17 2010
@@ -421,7 +421,7 @@
assert noexclink.exitcase is None
if pos == "removed":
# the exception cannot actually occur at all.
- # See for example rspecialcase.rtype_call_specialcase().
+ # This is set by calling exception_cannot_occur().
# We just remove all exception links.
block.exitswitch = None
block.exits = block.exits[:1]
@@ -1019,7 +1019,7 @@
from pypy.rpython import rint, rbool, rfloat
from pypy.rpython import rrange
from pypy.rpython import rstr, rdict, rlist
-from pypy.rpython import rclass, rbuiltin, rpbc, rspecialcase
+from pypy.rpython import rclass, rbuiltin, rpbc
from pypy.rpython import rexternalobj
from pypy.rpython import rptr
from pypy.rpython import rgeneric
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/test/test_llinterp.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/test/test_llinterp.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/test/test_llinterp.py Sat Oct 30 16:41:17 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
import py
import sys
from pypy.rpython.lltypesystem.lltype import typeOf, pyobjectptr, Ptr,\
@@ -5,6 +6,7 @@
from pypy.rpython.lltypesystem.lloperation import llop
from pypy.rpython.llinterp import LLInterpreter, LLException, log
from pypy.rpython.rmodel import inputconst
+from pypy.rpython.annlowlevel import hlstr
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.rint import signed_repr
from pypy.rpython.lltypesystem import rstr, lltype
@@ -12,13 +14,11 @@
from pypy.annotation.model import lltype_to_annotation
from pypy.rlib.rarithmetic import r_uint, ovfcheck
from pypy.rpython.ootypesystem import ootype
+from pypy.tool import leakfinder
from pypy import conftest
# switch on logging of interp to show more info on failing tests
-class MallocMismatch(Exception):
- pass
-
def setup_module(mod):
mod.logstate = py.log._getstate()
py.log.setconsumer("llinterp", py.log.STDOUT)
@@ -72,7 +72,7 @@
def get_interpreter(func, values, view='auto', viewbefore='auto', policy=None,
someobjects=False, type_system="lltype", backendopt=False,
- config=None, malloc_check=True, **extraconfigopts):
+ config=None, **extraconfigopts):
extra_key = [(key, value) for key, value in extraconfigopts.iteritems()]
extra_key.sort()
extra_key = tuple(extra_key)
@@ -97,7 +97,7 @@
viewbefore, policy, type_system=type_system,
backendopt=backendopt, config=config,
**extraconfigopts)
- interp = LLInterpreter(typer, malloc_check=malloc_check)
+ interp = LLInterpreter(typer)
_tcache[key] = (t, interp, graph)
# keep the cache small
_lastinterpreted.append(key)
@@ -115,10 +115,17 @@
interp, graph = get_interpreter(func, values, view, viewbefore, policy,
someobjects, type_system=type_system,
backendopt=backendopt, config=config,
- malloc_check=malloc_check, **kwargs)
- result = interp.eval_graph(graph, values)
- if malloc_check and interp.mallocs:
- raise MallocMismatch(interp.mallocs)
+ **kwargs)
+ if not malloc_check:
+ result = interp.eval_graph(graph, values)
+ else:
+ prev = leakfinder.start_tracking_allocations()
+ try:
+ result = interp.eval_graph(graph, values)
+ finally:
+ leaks = leakfinder.stop_tracking_allocations(False, prev)
+ if leaks:
+ raise leakfinder.MallocMismatch(leaks)
return result
def interpret_raises(exc, func, values, view='auto', viewbefore='auto',
@@ -418,6 +425,7 @@
assert result
def test_stack_malloc():
+ py.test.skip("stack-flavored mallocs no longer supported")
class A(object):
pass
def f():
@@ -430,6 +438,7 @@
assert result == 1
def test_invalid_stack_access():
+ py.test.skip("stack-flavored mallocs no longer supported")
class A(object):
pass
globala = A()
@@ -605,7 +614,7 @@
if x:
free(t, flavor='raw')
interpret(f, [1])
- py.test.raises(MallocMismatch, "interpret(f, [0])")
+ py.test.raises(leakfinder.MallocMismatch, "interpret(f, [0])")
def f():
t1 = malloc(T, flavor='raw')
@@ -615,3 +624,37 @@
interpret(f, [])
+def test_context_manager():
+ state = []
+ class C:
+ def __enter__(self):
+ state.append('acquire')
+ return self
+ def __exit__(self, *args):
+ if args[1] is not None:
+ state.append('raised')
+ state.append('release')
+ def f():
+ try:
+ with C() as c:
+ state.append('use')
+ raise ValueError
+ except ValueError:
+ pass
+ return ', '.join(state)
+ res = interpret(f, [])
+ assert hlstr(res) == 'acquire, use, raised, release'
+
+
+def test_scoped_allocator():
+ from pypy.rpython.lltypesystem.lltype import scoped_alloc, Array, Signed
+ T = Array(Signed)
+
+ def f():
+ x = 0
+ with scoped_alloc(T, 1) as array:
+ array[0] = -42
+ x = array[0]
+ assert x == -42
+
+ res = interpret(f, [])
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/test/test_nongc.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/test/test_nongc.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/test/test_nongc.py Sat Oct 30 16:41:17 2010
@@ -79,7 +79,7 @@
py.test.raises(TypeError,rtyper.specialize) # results in an invalid cast
def test_isinstance():
- class A:
+ class A(object):
_alloc_flavor_ = "raw"
class B(A):
pass
@@ -95,7 +95,24 @@
o = B()
else:
o = C()
- return 100*isinstance(o, A)+10*isinstance(o, B)+1*isinstance(o ,C)
+ res = 100*isinstance(o, A) + 10*isinstance(o, B) + 1*isinstance(o, C)
+ if i == 0:
+ pass
+ elif i == 1:
+ assert isinstance(o, A)
+ free_non_gc_object(o)
+ elif i == 2:
+ assert isinstance(o, B)
+ free_non_gc_object(o)
+ else:
+ assert isinstance(o, C)
+ free_non_gc_object(o)
+ return res
+
+ assert f(1) == 100
+ assert f(2) == 110
+ assert f(3) == 111
+ assert f(0) == 0
a = RPythonAnnotator()
#does not raise:
@@ -131,10 +148,14 @@
d = b
elif i == 2:
e = c
- return (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
+ res = (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
0x0008*(a is e) | 0x0010*(b is c) | 0x0020*(b is d) |
0x0040*(b is e) | 0x0080*(c is d) | 0x0100*(c is e) |
0x0200*(d is e))
+ free_non_gc_object(a)
+ free_non_gc_object(b)
+ free_non_gc_object(c)
+ return res
a = RPythonAnnotator()
#does not raise:
s = a.build_types(f, [int])
@@ -169,10 +190,13 @@
d = b
elif i == 2:
e = c
- return (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
+ res = (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) |
0x0008*(a is e) | 0x0010*(b is c) | 0x0020*(b is d) |
0x0040*(b is e) | 0x0080*(c is d) | 0x0100*(c is e) |
0x0200*(d is e))
+ free_non_gc_object(a)
+ free_non_gc_object(b)
+ return res
a = RPythonAnnotator()
#does not raise:
s = a.build_types(f, [int])
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rbuilder.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rbuilder.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rbuilder.py Sat Oct 30 16:41:17 2010
@@ -1,4 +1,4 @@
-
+import py
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.rpython.lltypesystem.rbuilder import *
from pypy.rpython.annlowlevel import llstr, hlstr
@@ -55,8 +55,29 @@
assert res == 'aabcabcdefbuuuu'
assert isinstance(res, unicode)
+ def test_string_getlength(self):
+ def func():
+ s = StringBuilder()
+ s.append("a")
+ s.append("abc")
+ return s.getlength()
+ res = self.interpret(func, [])
+ assert res == 4
+
+ def test_unicode_getlength(self):
+ def func():
+ s = UnicodeBuilder()
+ s.append(u"a")
+ s.append(u"abc")
+ return s.getlength()
+ res = self.interpret(func, [])
+ assert res == 4
+
class TestLLtype(BaseTestStringBuilder, LLRtypeMixin):
pass
class TestOOtype(BaseTestStringBuilder, OORtypeMixin):
- pass
+ def test_string_getlength(self):
+ py.test.skip("getlength(): not implemented on ootype")
+ def test_unicode_getlength(self):
+ py.test.skip("getlength(): not implemented on ootype")
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rclass.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rclass.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rclass.py Sat Oct 30 16:41:17 2010
@@ -3,7 +3,7 @@
from pypy.translator.translator import TranslationContext, graphof
from pypy.rpython.lltypesystem.lltype import *
from pypy.rpython.ootypesystem import ootype
-from pypy.rlib.rarithmetic import intmask
+from pypy.rlib.rarithmetic import intmask, r_longlong
from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin
from pypy.objspace.flow.model import summary
@@ -319,6 +319,17 @@
res = self.interpret(f, [])
assert res == 42
+ def test_staticmethod2(self):
+ class A(object):
+ f = staticmethod(lambda x, y: x*y)
+ class B(A):
+ f = staticmethod(lambda x, y: x+y)
+ def f():
+ b = B()
+ return b.f(6, 7)
+ res = self.interpret(f, [])
+ assert res == 13
+
def test_is(self):
class A: pass
class B(A): pass
@@ -1001,6 +1012,40 @@
res = self.interpret(f, [5])
assert res == 0
+ def test_order_of_fields(self):
+ class A(object):
+ pass
+ def f(n):
+ a = A()
+ a.as_int = n
+ a.as_char = chr(n)
+ a.as_unichar = unichr(n)
+ a.as_double = n + 0.5
+ a.as_bool = bool(n)
+ a.as_void = None
+ a.as_longlong = r_longlong(n)
+ a.as_reference = A()
+ return a
+
+ res = self.interpret(f, [5])
+ names = list(typeOf(res).TO._names)
+ i = names.index('inst_as_int')
+ c = names.index('inst_as_char')
+ u = names.index('inst_as_unichar')
+ d = names.index('inst_as_double')
+ b = names.index('inst_as_bool')
+ v = names.index('inst_as_void')
+ l = names.index('inst_as_longlong')
+ r = names.index('inst_as_reference')
+ assert v == 1 # void fields are first
+ assert sorted([c, b]) == [7, 8]
+ if sys.maxint == 2147483647:
+ assert sorted([u, i, r]) == [4, 5, 6] # 32-bit types
+ assert sorted([d, l]) == [2, 3] # 64-bit types
+ else:
+ assert sorted([u]) == [6] # 32-bit types
+ assert sorted([i, r, d, l]) == [2, 3, 4, 5] # 64-bit types
+
class TestOOtype(BaseTestRclass, OORtypeMixin):
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rdict.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rdict.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rdict.py Sat Oct 30 16:41:17 2010
@@ -682,6 +682,26 @@
# if it does not crash, we are fine. It crashes if you forget the hash field.
self.interpret(func, [])
+ def test_dict_popitem(self):
+ def func():
+ d = {}
+ d[5] = 2
+ d[6] = 3
+ k1, v1 = d.popitem()
+ assert len(d) == 1
+ k2, v2 = d.popitem()
+ try:
+ d.popitem()
+ except KeyError:
+ pass
+ else:
+ assert 0, "should have raised KeyError"
+ assert len(d) == 0
+ return k1*1000 + v1*100 + k2*10 + v2
+
+ res = self.interpret(func, [])
+ assert res in [5263, 6352]
+
# ____________________________________________________________
def test_opt_nullkeymarker(self):
Modified: pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rptr.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rptr.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rptr.py Sat Oct 30 16:41:17 2010
@@ -212,10 +212,31 @@
S = Struct('S', ('x', Signed))
def fn(n):
- p = malloc(S, flavor='whatever')
+ p = malloc(S, flavor='raw')
p.x = n
result = p.x
- free(p, flavor='whatever')
+ free(p, flavor='raw')
+ return n
+
+ res = interpret(fn, [23])
+ assert res == 23
+
+ S = Struct('S', ('x', Signed))
+ def fn(n):
+ p = malloc(S, flavor='raw', track_allocation=False)
+ p.x = n
+ result = p.x
+ return n
+
+ res = interpret(fn, [23])
+ assert res == 23
+
+ S = Struct('S', ('x', Signed))
+ def fn(n):
+ p = malloc(S, flavor='raw', track_allocation=False)
+ p.x = n
+ result = p.x
+ free(p, flavor='raw', track_allocation=False)
return n
res = interpret(fn, [23])
Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/funcgen.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/c/funcgen.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/c/funcgen.py Sat Oct 30 16:41:17 2010
@@ -427,7 +427,7 @@
r = self.expr(op.result)
return 'OP_CALL_ARGS((%s), %s);' % (', '.join(args), r)
- def generic_call(self, FUNC, fnexpr, args_v, v_result):
+ def generic_call(self, FUNC, fnexpr, args_v, v_result, targets=None):
args = []
assert len(args_v) == len(FUNC.TO.ARGS)
for v, ARGTYPE in zip(args_v, FUNC.TO.ARGS):
@@ -444,17 +444,26 @@
# skip assignment of 'void' return value
r = self.expr(v_result)
line = '%s = %s' % (r, line)
+ if targets:
+ for graph in targets:
+ if getattr(graph, 'inhibit_tail_call', False):
+ line += '\nPYPY_INHIBIT_TAIL_CALL();'
+ break
return line
def OP_DIRECT_CALL(self, op):
fn = op.args[0]
+ try:
+ targets = [fn.value._obj.graph]
+ except AttributeError:
+ targets = None
return self.generic_call(fn.concretetype, self.expr(fn),
- op.args[1:], op.result)
+ op.args[1:], op.result, targets)
def OP_INDIRECT_CALL(self, op):
fn = op.args[0]
return self.generic_call(fn.concretetype, self.expr(fn),
- op.args[1:-1], op.result)
+ op.args[1:-1], op.result, op.args[-1].value)
def OP_ADR_CALL(self, op):
ARGTYPES = [v.concretetype for v in op.args[1:]]
Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/instruction.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/instruction.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/instruction.py Sat Oct 30 16:41:17 2010
@@ -82,6 +82,11 @@
def all_sources_of(self, localvar):
return [localvar]
+class InsnCondJump(Insn): # only for debugging; not used internally
+ _args_ = ['label']
+ def __init__(self, label):
+ self.label = label
+
class Label(Insn):
_args_ = ['label', 'lineno']
def __init__(self, label, lineno):
@@ -170,9 +175,12 @@
self.delta = -7 # use an odd value as marker
class InsnStop(Insn):
- pass
+ _args_ = ['reason']
+ def __init__(self, reason='?'):
+ self.reason = reason
class InsnRet(InsnStop):
+ _args_ = []
framesize = 0
def __init__(self, registers):
self.registers = registers
Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/test/elf/track5.s
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/test/elf/track5.s (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/test/elf/track5.s Sat Oct 30 16:41:17 2010
@@ -44,7 +44,7 @@
addl %eax, %ebx
jmp .L1221
.L1227:
- call RPyAbort
+ ;;call RPyAbort
cmpl 12(%esi), %ebx
jb .L1229
addl $20, %esp
Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/test/test_trackgcroot.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/test/test_trackgcroot.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/test/test_trackgcroot.py Sat Oct 30 16:41:17 2010
@@ -109,7 +109,7 @@
def test_computegcmaptable():
tests = []
- for format in ('elf', 'darwin', 'msvc', 'elf64'):
+ for format in ('elf', 'elf64', 'darwin', 'darwin64', 'msvc'):
for path in this_dir.join(format).listdir("track*.s"):
n = path.purebasename[5:]
try:
Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/trackgcroot.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/trackgcroot.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/trackgcroot.py Sat Oct 30 16:41:17 2010
@@ -6,7 +6,7 @@
from pypy.translator.c.gcc.instruction import InsnFunctionStart, InsnStop
from pypy.translator.c.gcc.instruction import InsnSetLocal, InsnCopyLocal
from pypy.translator.c.gcc.instruction import InsnPrologue, InsnEpilogue
-from pypy.translator.c.gcc.instruction import InsnGCROOT
+from pypy.translator.c.gcc.instruction import InsnGCROOT, InsnCondJump
from pypy.translator.c.gcc.instruction import InsnStackAdjust
from pypy.translator.c.gcc.instruction import InsnCannotFollowEsp
from pypy.translator.c.gcc.instruction import LocalVar, somenewvalue
@@ -46,6 +46,7 @@
self.findlabels()
self.parse_instructions()
try:
+ self.trim_unreachable_instructions()
self.find_noncollecting_calls()
if not self.list_collecting_call_insns():
return []
@@ -122,19 +123,36 @@
assert label not in self.labels, "duplicate label: %s" % label
self.labels[label] = Label(label, lineno)
+ def trim_unreachable_instructions(self):
+ reached = set([self.insns[0]])
+ prevlen = 0
+ while len(reached) > prevlen:
+ prevlen = len(reached)
+ for insn in self.insns:
+ if insn not in reached:
+ for previnsn in insn.previous_insns:
+ if previnsn in reached:
+ # this instruction is reachable too
+ reached.add(insn)
+ break
+ # now kill all unreachable instructions
+ i = 0
+ while i < len(self.insns):
+ if self.insns[i] in reached:
+ i += 1
+ else:
+ del self.insns[i]
+
def find_noncollecting_calls(self):
- cannot_collect = self.CANNOT_COLLECT.copy()
+ cannot_collect = {}
for line in self.lines:
match = self.r_gcnocollect_marker.search(line)
if match:
name = match.group(1)
cannot_collect[name] = True
#
- if self.format in ('darwin', 'mingw32', 'msvc'):
- self.cannot_collect = dict.fromkeys(
- ['_' + name for name in cannot_collect])
- else:
- self.cannot_collect = cannot_collect
+ self.cannot_collect = dict.fromkeys(
+ [self.function_names_prefix + name for name in cannot_collect])
def append_instruction(self, insn):
# Add the instruction to the list, and link it to the previous one.
@@ -410,7 +428,8 @@
return result
# ____________________________________________________________
- CANNOT_COLLECT = { # some of the most used functions that cannot collect
+ BASE_FUNCTIONS_NOT_RETURNING = {
+ 'abort': None,
'pypy_debug_catch_fatal_exception': None,
'RPyAbort': None,
'RPyAssertFailed': None,
@@ -427,6 +446,7 @@
IGNORE_OPS_WITH_PREFIXES = dict.fromkeys([
'cmp', 'test', 'set', 'sahf', 'lahf', 'cltd', 'cld', 'std',
'rep', 'movs', 'lods', 'stos', 'scas', 'cwtl', 'cwde', 'prefetch',
+ 'pslld',
# floating-point operations cannot produce GC pointers
'f',
'cvt', 'ucomi', 'comi', 'subs', 'subp' , 'adds', 'addp', 'xorp',
@@ -438,7 +458,7 @@
'bswap', 'bt', 'rdtsc',
'punpck', 'pshufd',
# zero-extending moves should not produce GC pointers
- 'movz',
+ 'movz',
])
visit_movb = visit_nop
@@ -644,7 +664,7 @@
if label != '0':
self.register_jump_to(label)
tablelin += 1
- return InsnStop()
+ return InsnStop("jump table")
if self.r_unaryinsn_star.match(line):
# that looks like an indirect tail-call.
# tail-calls are equivalent to RET for us
@@ -658,7 +678,7 @@
assert not target.startswith('.')
# tail-calls are equivalent to RET for us
return InsnRet(self.CALLEE_SAVE_REGISTERS)
- return InsnStop()
+ return InsnStop("jump")
def register_jump_to(self, label):
if not isinstance(self.insns[-1], InsnStop):
@@ -682,7 +702,7 @@
else:
label = match.group(1)
self.register_jump_to(label)
- return []
+ return [InsnCondJump(label)]
visit_jmpl = visit_jmp
visit_je = conditional_jump
@@ -754,7 +774,7 @@
target, = sources
if target in self.FUNCTIONS_NOT_RETURNING:
- return [InsnStop(), InsnCannotFollowEsp()]
+ return [InsnStop(target)]
if self.format == 'mingw32' and target == '__alloca':
# in functions with large stack requirements, windows
# needs a call to _alloca(), to turn reserved pages
@@ -885,7 +905,7 @@
# statically known pointer to a register
# %eax -> %rax
- new_line = re.sub(r"%e(ax|bx|cx|dx|di|si)$", r"%r\1", line)
+ new_line = re.sub(r"%e(ax|bx|cx|dx|di|si|bp)$", r"%r\1", line)
# %r10d -> %r10
new_line = re.sub(r"%r(\d+)d$", r"%r\1", new_line)
return func(self, new_line)
@@ -951,6 +971,7 @@
class ElfFunctionGcRootTracker32(FunctionGcRootTracker32):
format = 'elf'
+ function_names_prefix = ''
ESP = '%esp'
EBP = '%ebp'
@@ -984,13 +1005,14 @@
r_bottom_marker = re.compile(r"\t/[*] GC_STACK_BOTTOM [*]/")
FUNCTIONS_NOT_RETURNING = {
- 'abort': None,
'_exit': None,
'__assert_fail': None,
'___assert_rtn': None,
'L___assert_rtn$stub': None,
'L___eprintf$stub': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING[_name] = None
def __init__(self, lines, filetag=0):
match = self.r_functionstart.match(lines[0])
@@ -1010,6 +1032,8 @@
class ElfFunctionGcRootTracker64(FunctionGcRootTracker64):
format = 'elf64'
+ function_names_prefix = ''
+
ESP = '%rsp'
EBP = '%rbp'
EAX = '%rax'
@@ -1042,13 +1066,14 @@
r_bottom_marker = re.compile(r"\t/[*] GC_STACK_BOTTOM [*]/")
FUNCTIONS_NOT_RETURNING = {
- 'abort': None,
'_exit': None,
'__assert_fail': None,
'___assert_rtn': None,
'L___assert_rtn$stub': None,
'L___eprintf$stub': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING[_name] = None
def __init__(self, lines, filetag=0):
match = self.r_functionstart.match(lines[0])
@@ -1066,8 +1091,9 @@
ElfFunctionGcRootTracker64.init_regexp()
-class DarwinFunctionGcRootTracker(ElfFunctionGcRootTracker32):
+class DarwinFunctionGcRootTracker32(ElfFunctionGcRootTracker32):
format = 'darwin'
+ function_names_prefix = '_'
r_functionstart = re.compile(r"_(\w+):\s*$")
OFFSET_LABELS = 0
@@ -1077,17 +1103,36 @@
funcname = '_' + match.group(1)
FunctionGcRootTracker32.__init__(self, funcname, lines, filetag)
-class Mingw32FunctionGcRootTracker(DarwinFunctionGcRootTracker):
+class DarwinFunctionGcRootTracker64(ElfFunctionGcRootTracker64):
+ format = 'darwin64'
+ function_names_prefix = '_'
+
+ LABEL = ElfFunctionGcRootTracker32.LABEL
+ r_jmptable_item = re.compile(r"\t.(?:long|quad)\t"+LABEL+"(-\"?[A-Za-z0-9$]+\"?)?\s*$")
+
+ r_functionstart = re.compile(r"_(\w+):\s*$")
+ OFFSET_LABELS = 0
+
+ def __init__(self, lines, filetag=0):
+ match = self.r_functionstart.match(lines[0])
+ funcname = '_' + match.group(1)
+ FunctionGcRootTracker64.__init__(self, funcname, lines, filetag)
+
+class Mingw32FunctionGcRootTracker(DarwinFunctionGcRootTracker32):
format = 'mingw32'
+ function_names_prefix = '_'
FUNCTIONS_NOT_RETURNING = {
- '_abort': None,
'_exit': None,
'__assert': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING['_' + _name] = None
class MsvcFunctionGcRootTracker(FunctionGcRootTracker32):
format = 'msvc'
+ function_names_prefix = '_'
+
ESP = 'esp'
EBP = 'ebp'
EAX = 'eax'
@@ -1127,7 +1172,6 @@
r_bottom_marker = re.compile(r"; .+\tpypy_asm_stack_bottom\(\);")
FUNCTIONS_NOT_RETURNING = {
- '_abort': None,
'__exit': None,
'__assert': None,
'__wassert': None,
@@ -1136,6 +1180,8 @@
'DWORD PTR __imp__abort': None,
'DWORD PTR __imp___wassert': None,
}
+ for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING:
+ FUNCTIONS_NOT_RETURNING['_' + _name] = None
@classmethod
def init_regexp(cls):
@@ -1343,7 +1389,7 @@
class DarwinAssemblerParser(AssemblerParser):
format = "darwin"
- FunctionGcRootTracker = DarwinFunctionGcRootTracker
+ FunctionGcRootTracker = DarwinFunctionGcRootTracker32
r_textstart = re.compile(r"\t.text\s*$")
@@ -1389,6 +1435,10 @@
return super(DarwinAssemblerParser, self).process_function(
lines, entrypoint, filename)
+class DarwinAssemblerParser64(DarwinAssemblerParser):
+ format = "darwin64"
+ FunctionGcRootTracker = DarwinFunctionGcRootTracker64
+
class Mingw32AssemblerParser(DarwinAssemblerParser):
format = "mingw32"
FunctionGcRootTracker = Mingw32FunctionGcRootTracker
@@ -1512,6 +1562,7 @@
'elf': ElfAssemblerParser,
'elf64': ElfAssemblerParser64,
'darwin': DarwinAssemblerParser,
+ 'darwin64': DarwinAssemblerParser64,
'mingw32': Mingw32AssemblerParser,
'msvc': MsvcAssemblerParser,
}
@@ -1543,15 +1594,13 @@
assert self.seen_main
def _globalname(name, disp=""):
- if self.format in ('darwin', 'mingw32', 'msvc'):
- name = '_' + name
- return name
+ return tracker_cls.function_names_prefix + name
def _variant(**kwargs):
txt = kwargs[self.format]
print >> output, "\t%s" % txt
- if self.format == 'elf64':
+ if self.format in ('elf64', 'darwin64'):
word_decl = '.quad'
else:
word_decl = '.long'
@@ -1604,10 +1653,11 @@
}
}
"""
- elif self.format == 'elf64':
+ elif self.format in ('elf64', 'darwin64'):
print >> output, "\t.text"
print >> output, "\t.globl %s" % _globalname('pypy_asm_stackwalk')
- print >> output, "\t.type pypy_asm_stackwalk, @function"
+ _variant(elf64='.type pypy_asm_stackwalk, @function',
+ darwin64='')
print >> output, "%s:" % _globalname('pypy_asm_stackwalk')
print >> output, """\
@@ -1652,8 +1702,9 @@
/* the return value is the one of the 'call' above, */
/* because %rax (and possibly %rdx) are unmodified */
ret
- .size pypy_asm_stackwalk, .-pypy_asm_stackwalk
"""
+ _variant(elf64='.size pypy_asm_stackwalk, .-pypy_asm_stackwalk',
+ darwin64='')
else:
print >> output, "\t.text"
print >> output, "\t.globl %s" % _globalname('pypy_asm_stackwalk')
@@ -1780,6 +1831,7 @@
_variant(elf='.section\t.rodata',
elf64='.section\t.rodata',
darwin='.const',
+ darwin64='.const',
mingw32='')
print >> output, """\
@@ -1857,7 +1909,10 @@
shuffle = False
output_raw_table = False
if sys.platform == 'darwin':
- format = 'darwin'
+ if sys.maxint > 2147483647:
+ format = 'darwin64'
+ else:
+ format = 'darwin'
elif sys.platform == 'win32':
format = 'mingw32'
else:
Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/src/g_include.h
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/c/src/g_include.h (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/c/src/g_include.h Sat Oct 30 16:41:17 2010
@@ -53,6 +53,7 @@
# include "src/rtyper.h"
# include "src/debug_print.h"
# include "src/debug_traceback.h"
+# include "src/debug_alloc.h"
#ifndef AVR
# include "src/ll_os.h"
# include "src/ll_strtod.h"
Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/src/main.h
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/c/src/main.h (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/c/src/main.h Sat Oct 30 16:41:17 2010
@@ -53,10 +53,16 @@
}
exitcode = STANDALONE_ENTRY_POINT(list);
+
+#ifdef RPY_ASSERT
+ pypy_debug_alloc_results();
+#endif
+
if (RPyExceptionOccurred()) {
/* print the RPython traceback */
pypy_debug_catch_fatal_exception();
}
+
return exitcode;
memory_out:
Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/src/signals.h
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/c/src/signals.h (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/c/src/signals.h Sat Oct 30 16:41:17 2010
@@ -6,20 +6,6 @@
#include
-#ifndef LONG_MAX
-#if SIZEOF_LONG == 4
-#define LONG_MAX 0X7FFFFFFFL
-#elif SIZEOF_LONG == 8
-#define LONG_MAX 0X7FFFFFFFFFFFFFFFL
-#else
-#error "could not set LONG_MAX in pyport.h"
-#endif
-#endif
-
-#ifndef LONG_MIN
-#define LONG_MIN (-LONG_MAX-1)
-#endif
-
#include
#ifdef MS_WINDOWS
@@ -28,10 +14,6 @@
#include
-#ifndef SIG_ERR
-#define SIG_ERR ((PyOS_sighandler_t)(-1))
-#endif
-
#if defined(PYOS_OS2) && !defined(PYCC_GCC)
#define NSIG 12
#include
@@ -65,11 +47,11 @@
/* utility to poll for signals that arrived */
int pypysig_poll(void); /* => signum or -1 */
-/* When a signal is received, the high bit of pypysig_occurred is set.
- After all signals are processed by pypysig_poll(), the high bit is
+/* When a signal is received, the bit 30 of pypysig_occurred is set.
+ After all signals are processed by pypysig_poll(), the bit 30 is
cleared again. The variable is exposed and RPython code is free to
use the other bits in any way. */
-#define PENDING_SIGNAL_BIT (LONG_MIN) /* high bit */
+#define PENDING_SIGNAL_BIT (1 << 30)
/* This is a struct for the JIT. See interp_signal.py. */
struct pypysig_long_struct {
long value;
Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/src/stack.h
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/c/src/stack.h (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/c/src/stack.h Sat Oct 30 16:41:17 2010
@@ -33,6 +33,12 @@
&& LL_stack_too_big_slowpath());
}
+#ifdef __GNUC__
+# define PYPY_INHIBIT_TAIL_CALL() asm("/* inhibit_tail_call */")
+#else
+# define PYPY_INHIBIT_TAIL_CALL() /* add hints for other compilers here */
+#endif
+
#ifndef PYPY_NOT_MAIN_FILE
#include
Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_genc.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_genc.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_genc.py Sat Oct 30 16:41:17 2010
@@ -426,6 +426,7 @@
if py.test.config.option.view:
t.view()
assert ' BarStruct ' in t.driver.cbuilder.c_source_filename.read()
+ free(foo, flavor="raw")
def test_recursive_llhelper():
from pypy.rpython.annlowlevel import llhelper
@@ -473,7 +474,27 @@
return f(s)
a_f = A(f, "f")
a_g = A(g, "g")
- t = lltype.malloc(STRUCT, flavor="raw")
+ t = lltype.malloc(STRUCT, flavor="raw", immortal=True)
t.bar = llhelper(FTPTR, a_f.make_func())
fn = compile(chooser, [bool])
assert fn(True)
+
+def test_inhibit_tail_call():
+ from pypy.rpython.lltypesystem import lltype
+ def foobar_fn(n):
+ return 42
+ foobar_fn._dont_inline_ = True
+ def main(n):
+ return foobar_fn(n)
+ #
+ t = Translation(main, [int], backend="c")
+ t.rtype()
+ t.context._graphof(foobar_fn).inhibit_tail_call = True
+ t.source_c()
+ lines = t.driver.cbuilder.c_source_filename.readlines()
+ for i, line in enumerate(lines):
+ if '= pypy_g_foobar_fn' in line:
+ break
+ else:
+ assert 0, "the call was not found in the C source"
+ assert 'PYPY_INHIBIT_TAIL_CALL();' in lines[i+1]
Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_lltyped.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_lltyped.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_lltyped.py Sat Oct 30 16:41:17 2010
@@ -401,6 +401,7 @@
for i in range(n):
p = malloc(S, flavor='raw', zero=True)
if p.x != 0 or p.y != 0:
+ free(p, flavor='raw')
return -1
p.x = i
p.y = i
@@ -418,14 +419,16 @@
def f(n):
for length in range(n-1, -1, -1):
p = malloc(S, length, flavor='raw', zero=True)
- if p.x != 0:
- return -1
- p.x = n
- for j in range(length):
- if p.y[j] != 0:
- return -3
- p.y[j] = n^j
- free(p, flavor='raw')
+ try:
+ if p.x != 0:
+ return -1
+ p.x = n
+ for j in range(length):
+ if p.y[j] != 0:
+ return -3
+ p.y[j] = n^j
+ finally:
+ free(p, flavor='raw')
return 42
fn = self.getcompiled(f, [int])
@@ -655,7 +658,7 @@
def test_prebuilt_ll2ctypes_array(self):
from pypy.rpython.lltypesystem import rffi, ll2ctypes
A = rffi.CArray(Char)
- a = malloc(A, 6, flavor='raw')
+ a = malloc(A, 6, flavor='raw', immortal=True)
a[0] = 'a'
a[1] = 'b'
a[2] = 'c'
@@ -676,7 +679,7 @@
def test_ll2ctypes_array_from_c(self):
from pypy.rpython.lltypesystem import rffi, ll2ctypes
A = rffi.CArray(Char)
- a = malloc(A, 6, flavor='raw')
+ a = malloc(A, 6, flavor='raw', immortal=True)
a[0] = 'a'
a[1] = 'b'
a[2] = 'c'
Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_newgc.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_newgc.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_newgc.py Sat Oct 30 16:41:17 2010
@@ -624,13 +624,13 @@
os.unlink(self.filename)
def define_callback_with_collect(cls):
- from pypy.rlib.libffi import ffi_type_pointer, cast_type_to_ffitype,\
+ from pypy.rlib.clibffi import ffi_type_pointer, cast_type_to_ffitype,\
CDLL, ffi_type_void, CallbackFuncPtr, ffi_type_sint
from pypy.rpython.lltypesystem import rffi, ll2ctypes
import gc
ffi_size_t = cast_type_to_ffitype(rffi.SIZE_T)
- from pypy.rlib.libffi import get_libc_name
+ from pypy.rlib.clibffi import get_libc_name
def callback(ll_args, ll_res, stuff):
gc.collect()
Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_standalone.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_standalone.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_standalone.py Sat Oct 30 16:41:17 2010
@@ -16,11 +16,16 @@
class StandaloneTests(object):
config = None
- def compile(self, entry_point, debug=True, shared=False):
+ def compile(self, entry_point, debug=True, shared=False,
+ stackcheck=False):
t = TranslationContext(self.config)
t.buildannotator().build_types(entry_point, [s_list_of_strings])
t.buildrtyper().specialize()
+ if stackcheck:
+ from pypy.translator.transform import insert_ll_stackcheck
+ insert_ll_stackcheck(t)
+
t.config.translation.shared = shared
cbuilder = CStandaloneBuilder(t, entry_point, t.config)
@@ -630,6 +635,22 @@
else:
os.environ['CC'] = old_cc
+ def test_inhibit_tail_call(self):
+ # the point is to check that the f()->f() recursion stops
+ from pypy.rlib.rstackovf import StackOverflow
+ def f(n):
+ if n <= 0:
+ return 42
+ return f(n+1)
+ def entry_point(argv):
+ try:
+ return f(1)
+ except StackOverflow:
+ print 'hi!'
+ return 0
+ t, cbuilder = self.compile(entry_point, stackcheck=True)
+ out = cbuilder.cmdexec("")
+ assert out.strip() == "hi!"
class TestMaemo(TestStandalone):
def setup_class(cls):
Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_typed.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_typed.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_typed.py Sat Oct 30 16:41:17 2010
@@ -1,3 +1,4 @@
+from __future__ import with_statement
import autopath
import sys
import math
@@ -823,3 +824,41 @@
while int(x + frac) >= -sys.maxint-1:
x -= 1
assert f(x + frac) == -666
+
+ def test_context_manager(self):
+ state = []
+ class C:
+ def __init__(self, name):
+ self.name = name
+ def __enter__(self):
+ state.append('acquire')
+ return self
+ def __exit__(self, typ, value, tb):
+ if typ is not None:
+ if value is None:
+ raise RuntimeError('test failed')
+ state.append('raised')
+ else:
+ if value is not None:
+ raise RuntimeError('test failed')
+ state.append('release')
+
+ def func(n):
+ del state[:]
+ try:
+ with C('hello') as c:
+ state.append(c.name)
+ if n == 1:
+ raise ValueError
+ elif n == 2:
+ raise TypeError
+ except (ValueError, TypeError):
+ pass
+ return ', '.join(state)
+ f = self.getcompiled(func, [int])
+ res = f(0)
+ assert res == 'acquire, hello, release'
+ res = f(1)
+ assert res == 'acquire, hello, raised, release'
+ res = f(2)
+ assert res == 'acquire, hello, raised, release'
Modified: pypy/branch/jit-unroll-loops/pypy/translator/driver.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/driver.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/driver.py Sat Oct 30 16:41:17 2010
@@ -426,6 +426,22 @@
[OOTYPE],
"JIT compiler generation")
+ def task_jittest_lltype(self):
+ """ Run with the JIT on top of the llgraph backend
+ """
+ # parent process loop: spawn a child, wait for the child to finish,
+ # print a message, and restart
+ from pypy.translator.goal import unixcheckpoint
+ unixcheckpoint.restartable_point(auto='run')
+ # load the module pypy/jit/tl/jittest.py, which you can hack at
+ # and restart without needing to restart the whole translation process
+ from pypy.jit.tl import jittest
+ jittest.jittest(self)
+ #
+ task_jittest_lltype = taskdef(task_jittest_lltype,
+ [RTYPE],
+ "test of the JIT on the llgraph backend")
+
def task_backendopt_lltype(self):
""" Run all backend optimizations - lltype version
"""
@@ -433,7 +449,8 @@
backend_optimizations(self.translator)
#
task_backendopt_lltype = taskdef(task_backendopt_lltype,
- [RTYPE, '??pyjitpl_lltype'],
+ [RTYPE, '??pyjitpl_lltype',
+ '??jittest_lltype'],
"lltype back-end optimisations")
BACKENDOPT = 'backendopt_lltype'
Modified: pypy/branch/jit-unroll-loops/pypy/translator/goal/ann_override.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/goal/ann_override.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/goal/ann_override.py Sat Oct 30 16:41:17 2010
@@ -27,21 +27,6 @@
pol.pypytypes = {}
pol.single_space = single_space
- #def override__wrap_exception_cls(pol, space, x):
- # import pypy.objspace.std.typeobject as typeobject
- # clsdef = getbookkeeper().getuniqueclassdef(typeobject.W_TypeObject)
- # return annmodel.SomeInstance(clsdef, can_be_None=True)
- #
- #def override__fake_object(pol, space, x):
- # from pypy.interpreter import typedef
- # clsdef = getbookkeeper().getuniqueclassdef(typedef.W_Root)
- # return annmodel.SomeInstance(clsdef)
- #
- #def override__cpy_compile(pol, self, source, filename, mode, flags):
- # from pypy.interpreter import pycode
- # clsdef = getbookkeeper().getuniqueclassdef(pycode.PyCode)
- # return annmodel.SomeInstance(clsdef)
-
def specialize__wrap(pol, funcdesc, args_s):
from pypy.interpreter.baseobjspace import Wrappable
from pypy.annotation.classdef import ClassDef
Modified: pypy/branch/jit-unroll-loops/pypy/translator/goal/app_main.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/goal/app_main.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/goal/app_main.py Sat Oct 30 16:41:17 2010
@@ -255,7 +255,7 @@
break
elif arg == '-u':
unbuffered = True
- elif arg == '-O':
+ elif arg == '-O' or arg == '-OO':
pass
elif arg == '--version' or arg == '-V':
print "Python", sys.version
@@ -385,9 +385,12 @@
python_startup = os.getenv('PYTHONSTARTUP')
if python_startup:
try:
- startup = open(python_startup).read()
- except IOError:
- pass
+ f = open(python_startup)
+ startup = f.read()
+ f.close()
+ except IOError, e:
+ print >> sys.stderr, "Could not open PYTHONSTARTUP"
+ print >> sys.stderr, "IOError:", e
else:
def run_it():
co_python_startup = compile(startup,
Modified: pypy/branch/jit-unroll-loops/pypy/translator/goal/translate.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/goal/translate.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/goal/translate.py Sat Oct 30 16:41:17 2010
@@ -27,6 +27,7 @@
("annotate", "do type inference", "-a --annotate", ""),
("rtype", "do rtyping", "-t --rtype", ""),
("pyjitpl", "JIT generation step", "--pyjitpl", ""),
+ ("jittest", "JIT test with llgraph backend", "--jittest", ""),
("backendopt", "do backend optimizations", "--backendopt", ""),
("source", "create source", "-s --source", ""),
("compile", "compile", "-c --compile", " (default goal)"),
Modified: pypy/branch/jit-unroll-loops/pypy/translator/jvm/test/test_class.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/jvm/test/test_class.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/jvm/test/test_class.py Sat Oct 30 16:41:17 2010
@@ -1,6 +1,6 @@
import py
from pypy.translator.jvm.test.runtest import JvmTest
-from pypy.translator.oosupport.test_template.class_ import BaseTestClass, BaseTestSpecialcase
+from pypy.translator.oosupport.test_template.class_ import BaseTestClass
class TestJvmClass(JvmTest, BaseTestClass):
def test_overridden_classattr_as_defaults(self):
@@ -26,6 +26,3 @@
def test_specialize_methods(self):
py.test.skip('ABSTRACT METHOD FIX: RE-TEST AFTER MERGE')
-
-class TestJvmSpecialCase(JvmTest, BaseTestSpecialcase):
- pass
Modified: pypy/branch/jit-unroll-loops/pypy/translator/oosupport/test_template/class_.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/oosupport/test_template/class_.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/oosupport/test_template/class_.py Sat Oct 30 16:41:17 2010
@@ -1,6 +1,5 @@
import py
from pypy.rpython.test import test_rclass
-from pypy.rpython.test.test_rspecialcase import BaseTestRspecialcase
class BaseTestClass(test_rclass.TestOOtype):
def test_abstract_method(self):
@@ -66,6 +65,3 @@
def test_cast_object_mix_null(self):
py.test.skip('cannot return ootype.NULL from translated functions')
-
-class BaseTestSpecialcase(BaseTestRspecialcase):
- pass
Modified: pypy/branch/jit-unroll-loops/pypy/translator/platform/darwin.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/platform/darwin.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/platform/darwin.py Sat Oct 30 16:41:17 2010
@@ -14,7 +14,10 @@
def __init__(self, cc=None):
if cc is None:
- cc = 'gcc'
+ try:
+ cc = os.environ['CC']
+ except KeyError:
+ cc = 'gcc'
self.cc = cc
def _args_for_shared(self, args):
Modified: pypy/branch/jit-unroll-loops/pypy/translator/transform.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/translator/transform.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/translator/transform.py Sat Oct 30 16:41:17 2010
@@ -221,15 +221,19 @@
stack_check_ptr_const = Constant(stack_check_ptr, lltype.typeOf(stack_check_ptr))
edges = set()
insert_in = set()
+ block2graph = {}
for caller in translator.graphs:
for block, callee in find_calls_from(translator, caller):
if getattr(getattr(callee, 'func', None),
'insert_stack_check_here', False):
insert_in.add(callee.startblock)
+ block2graph[callee.startblock] = callee
continue
if block is not caller.startblock:
edges.add((caller.startblock, block))
+ block2graph[caller.startblock] = caller
edges.add((block, callee.startblock))
+ block2graph[block] = caller
edgelist = [Edge(block1, block2) for (block1, block2) in edges]
edgedict = make_edge_dict(edgelist)
@@ -241,6 +245,10 @@
v.concretetype = lltype.Void
unwind_op = SpaceOperation('direct_call', [stack_check_ptr_const], v)
block.operations.insert(0, unwind_op)
+ # prevents cycles of tail calls from occurring -- such cycles would
+ # not consume any stack, so would turn into potentially infinite loops
+ graph = block2graph[block]
+ graph.inhibit_tail_call = True
return len(insert_in)
From hakanardo at codespeak.net Sat Oct 30 16:53:10 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 16:53:10 +0200 (CEST)
Subject: [pypy-svn] r78531 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030145310.BFC9D282BF2@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 16:53:09 2010
New Revision: 78531
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
Log:
(hakanardo, cfbolz, arigo) fixed merge
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py Sat Oct 30 16:53:09 2010
@@ -11,7 +11,7 @@
from pypy.jit.metainterp.jitprof import EmptyProfiler
from pypy.jit.metainterp import executor, compile, resume, history
from pypy.jit.metainterp.resoperation import rop, opname, ResOperation
-from pypy.jit.metainterp.test.oparser import pure_parse
+from pypy.jit.tool.oparser import pure_parse
##class FakeFrame(object):
## parent_resumedata_snapshot = None
From hakanardo at codespeak.net Sat Oct 30 16:53:26 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 16:53:26 +0200 (CEST)
Subject: [pypy-svn] r78532 - pypy/branch/jit-unroll-loops/pypy/jit/tool
Message-ID: <20101030145326.8852A282BF4@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 16:53:25 2010
New Revision: 78532
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/tool/oparser.py
Log:
(hakanardo, cfbolz, arigo) fixed merge
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tool/oparser.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tool/oparser.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tool/oparser.py Sat Oct 30 16:53:25 2010
@@ -31,6 +31,9 @@
def getopnum(self):
return self.OPNUM
+ def clone(self):
+ return ESCAPE_OP(self.OPNUM, self.getarglist()[:], self.result, self.getdescr())
+
class ExtendedTreeLoop(TreeLoop):
def getboxes(self):
From cfbolz at codespeak.net Sat Oct 30 16:59:02 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 16:59:02 +0200 (CEST)
Subject: [pypy-svn] r78533 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: . optimizeopt test
Message-ID: <20101030145902.69068282B90@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 16:58:59 2010
New Revision: 78533
Added:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeutil.py
- copied, changed from r78527, pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizefindnode.py
Removed:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizefindnode.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/specnode.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizefindnode.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_specnode.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/viewnode.py
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/history.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualizable.py
Log:
kill perfect specialization and specnodes
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py Sat Oct 30 16:58:59 2010
@@ -11,7 +11,6 @@
from pypy.jit.metainterp.history import AbstractFailDescr, BoxInt
from pypy.jit.metainterp.history import BoxPtr, BoxObj, BoxFloat, Const
from pypy.jit.metainterp import history
-from pypy.jit.metainterp.specnode import NotSpecNode, more_general_specnodes
from pypy.jit.metainterp.typesystem import llhelper, oohelper
from pypy.jit.metainterp.optimizeutil import InvalidLoop
from pypy.jit.codewriter import heaptracker
@@ -42,7 +41,6 @@
def make_loop_token(nb_args, jitdriver_sd):
loop_token = LoopToken()
- loop_token.specnodes = [prebuiltNotSpecNode] * nb_args
loop_token.outermost_jitdriver_sd = jitdriver_sd
return loop_token
@@ -82,8 +80,6 @@
return old_loop_token
if loop.preamble.operations:
- loop.token.specnodes = [prebuiltNotSpecNode] * len(loop.inputargs) # FIXME
- loop.preamble.token.specnodes = [prebuiltNotSpecNode] * len(loop.preamble.inputargs) # FIXME
send_loop_to_backend(metainterp_sd, loop, "loop")
send_loop_to_backend(metainterp_sd, loop.preamble, "entry bridge")
insert_loop_token(old_loop_tokens, loop.preamble.token)
@@ -100,13 +96,8 @@
# The following algo means "as late as possible, but before another
# loop token that would be more general and so completely mask off
# the new loop_token".
- for i in range(len(old_loop_tokens)):
- if more_general_specnodes(old_loop_tokens[i].specnodes,
- loop_token.specnodes):
- old_loop_tokens.insert(i, loop_token)
- break
- else:
- old_loop_tokens.append(loop_token)
+ # XXX do we still need a list?
+ old_loop_tokens.append(loop_token)
def send_loop_to_backend(metainterp_sd, loop, type):
globaldata = metainterp_sd.globaldata
@@ -189,13 +180,10 @@
raise metainterp_sd.ExitFrameWithExceptionRef(cpu, value)
-prebuiltNotSpecNode = NotSpecNode()
-
class TerminatingLoopToken(LoopToken):
terminating = True
def __init__(self, nargs, finishdescr):
- self.specnodes = [prebuiltNotSpecNode]*nargs
self.finishdescr = finishdescr
def make_done_loop_tokens():
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/history.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/history.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/history.py Sat Oct 30 16:58:59 2010
@@ -728,7 +728,6 @@
"""
terminating = False # see TerminatingLoopToken in compile.py
outermost_jitdriver_sd = None
- # specnodes = ...
# and more data specified by the backend when the loop is compiled
number = 0
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize.py Sat Oct 30 16:58:59 2010
@@ -2,9 +2,7 @@
# ____________________________________________________________
-from pypy.jit.metainterp.optimizefindnode import PerfectSpecializationFinder
from pypy.jit.metainterp.optimizeopt import optimize_loop_1
-from pypy.jit.metainterp.specnode import equals_specnodes
def optimize_loop(metainterp_sd, old_loop_tokens, loop):
debug_start("jit-optimize")
@@ -16,19 +14,14 @@
def _optimize_loop(metainterp_sd, old_loop_tokens, loop):
cpu = metainterp_sd.cpu
metainterp_sd.logger_noopt.log_loop(loop.inputargs, loop.operations)
- finder = PerfectSpecializationFinder(cpu)
- finder.find_nodes_loop(loop)
- for old_loop_token in old_loop_tokens:
- if equals_specnodes(old_loop_token.specnodes, loop.token.specnodes):
- return old_loop_token
+ # XXX do we really still need a list?
+ if old_loop_tokens:
+ return old_loop_tokens[0]
optimize_loop_1(metainterp_sd, loop)
return None
# ____________________________________________________________
-from pypy.jit.metainterp.optimizefindnode import BridgeSpecializationFinder
-from pypy.jit.metainterp.optimizeopt import optimize_bridge_1
-
def optimize_bridge(metainterp_sd, old_loop_tokens, bridge):
debug_start("jit-optimize")
try:
@@ -37,15 +30,10 @@
debug_stop("jit-optimize")
def _optimize_bridge(metainterp_sd, old_loop_tokens, bridge):
- cpu = metainterp_sd.cpu
+ cpu = metainterp_sd.cpu
metainterp_sd.logger_noopt.log_loop(bridge.inputargs, bridge.operations)
- finder = BridgeSpecializationFinder(cpu)
- finder.find_nodes_bridge(bridge)
- for old_loop_token in old_loop_tokens:
- if finder.bridge_matches(old_loop_token.specnodes):
- bridge.operations[-1].setdescr(old_loop_token) # patch jump target
- optimize_bridge_1(metainterp_sd, bridge)
- return old_loop_token
+ if old_loop_tokens:
+ return old_loop_tokens[0]
return None
# ____________________________________________________________
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py Sat Oct 30 16:58:59 2010
@@ -8,10 +8,7 @@
from pypy.jit.metainterp.optimizeopt.unroll import OptUnroll
def optimize_loop_1(metainterp_sd, loop, not_a_bridge=True):
- """Optimize loop.operations to make it match the input of loop.specnodes
- and to remove internal overheadish operations. Note that loop.specnodes
- must be applicable to the loop; you will probably get an AssertionError
- if not.
+ """Optimize loop.operations to remove internal overheadish operations.
"""
optimizations = [OptIntBounds(),
OptRewrite(),
@@ -26,7 +23,5 @@
optimizer.propagate_all_forward()
def optimize_bridge_1(metainterp_sd, bridge):
- """The same, but for a bridge. The only difference is that we don't
- expect 'specnodes' on the bridge.
- """
+ """The same, but for a bridge. """
optimize_loop_1(metainterp_sd, bridge, False)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py Sat Oct 30 16:58:59 2010
@@ -207,9 +207,6 @@
assert isinstance(expectedclassbox, Const)
realclassbox = value.get_constant_class(self.optimizer.cpu)
if realclassbox is not None:
- # the following assert should always be true for now,
- # because invalid loops that would fail it are detected
- # earlier, in optimizefindnode.py.
if realclassbox.same_constant(expectedclassbox):
return
raise InvalidLoop
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Sat Oct 30 16:58:59 2010
@@ -1,9 +1,4 @@
from pypy.jit.metainterp.history import Const, ConstInt
-from pypy.jit.metainterp.specnode import SpecNode, NotSpecNode, ConstantSpecNode
-from pypy.jit.metainterp.specnode import AbstractVirtualStructSpecNode
-from pypy.jit.metainterp.specnode import VirtualInstanceSpecNode
-from pypy.jit.metainterp.specnode import VirtualArraySpecNode
-from pypy.jit.metainterp.specnode import VirtualStructSpecNode
from pypy.jit.metainterp.resoperation import rop, ResOperation
from pypy.jit.metainterp.optimizeutil import _findall, sort_descrs
from pypy.jit.metainterp.optimizeutil import descrlist_dict
@@ -228,79 +223,9 @@
boxes.append(self.box)
-class __extend__(SpecNode):
- def setup_virtual_node(self, optimizer, box, newinputargs):
- raise NotImplementedError
- def teardown_virtual_node(self, optimizer, value, newexitargs):
- raise NotImplementedError
-
-class __extend__(NotSpecNode):
- def setup_virtual_node(self, optimizer, box, newinputargs):
- newinputargs.append(box)
- def teardown_virtual_node(self, optimizer, value, newexitargs):
- newexitargs.append(value.force_box())
-
-class __extend__(ConstantSpecNode):
- def setup_virtual_node(self, optimizer, box, newinputargs):
- optimizer.make_constant(box, self.constbox)
- def teardown_virtual_node(self, optimizer, value, newexitargs):
- pass
-
-class __extend__(AbstractVirtualStructSpecNode):
- def setup_virtual_node(self, optimizer, box, newinputargs):
- vvalue = self._setup_virtual_node_1(optimizer, box)
- for ofs, subspecnode in self.fields:
- subbox = optimizer.new_box(ofs)
- subspecnode.setup_virtual_node(optimizer, subbox, newinputargs)
- vvaluefield = optimizer.getvalue(subbox)
- vvalue.setfield(ofs, vvaluefield)
- def _setup_virtual_node_1(self, optimizer, box):
- raise NotImplementedError
- def teardown_virtual_node(self, optimizer, value, newexitargs):
- assert value.is_virtual()
- for ofs, subspecnode in self.fields:
- subvalue = value.getfield(ofs, optimizer.new_const(ofs))
- subspecnode.teardown_virtual_node(optimizer, subvalue, newexitargs)
-
-class __extend__(VirtualInstanceSpecNode):
- def _setup_virtual_node_1(self, optimizer, box):
- return optimizer.make_virtual(self.known_class, box)
-
-class __extend__(VirtualStructSpecNode):
- def _setup_virtual_node_1(self, optimizer, box):
- return optimizer.make_vstruct(self.typedescr, box)
-
-class __extend__(VirtualArraySpecNode):
- def setup_virtual_node(self, optimizer, box, newinputargs):
- vvalue = optimizer.make_varray(self.arraydescr, len(self.items), box)
- for index in range(len(self.items)):
- subbox = optimizer.new_box_item(self.arraydescr)
- subspecnode = self.items[index]
- subspecnode.setup_virtual_node(optimizer, subbox, newinputargs)
- vvalueitem = optimizer.getvalue(subbox)
- vvalue.setitem(index, vvalueitem)
- def teardown_virtual_node(self, optimizer, value, newexitargs):
- assert value.is_virtual()
- for index in range(len(self.items)):
- subvalue = value.getitem(index)
- subspecnode = self.items[index]
- subspecnode.teardown_virtual_node(optimizer, subvalue, newexitargs)
-
class OptVirtualize(optimizer.Optimization):
"Virtualize objects until they escape."
- def setup(self, not_a_bridge):
- if not not_a_bridge:
- return
-
- inputargs = self.optimizer.loop.inputargs
- specnodes = self.optimizer.loop.token.specnodes
- assert len(inputargs) == len(specnodes)
- newinputargs = []
- for i in range(len(inputargs)):
- specnodes[i].setup_virtual_node(self, inputargs[i], newinputargs)
- self.optimizer.loop.inputargs = newinputargs
-
def make_virtual(self, known_class, box, source_op=None):
vvalue = VirtualValue(self.optimizer, known_class, box, source_op)
self.make_equal_to(box, vvalue)
@@ -316,21 +241,6 @@
self.make_equal_to(box, vvalue)
return vvalue
- def optimize_JUMP(self, op):
- self.emit_operation(op) # FIXME
- return
- orgop = self.optimizer.loop.operations[-1]
- exitargs = []
- target_loop_token = orgop.getdescr()
- assert isinstance(target_loop_token, LoopToken)
- specnodes = target_loop_token.specnodes
- assert op.numargs() == len(specnodes)
- for i in range(len(specnodes)):
- value = self.getvalue(op.getarg(i))
- specnodes[i].teardown_virtual_node(self, value, exitargs)
- op = op.copy_and_change(op.getopnum(), args=exitargs[:])
- self.emit_operation(op)
-
def optimize_VIRTUAL_REF(self, op):
indexbox = op.getarg(1)
#
@@ -377,7 +287,6 @@
def optimize_GETFIELD_GC(self, op):
value = self.getvalue(op.getarg(0))
if value.is_virtual():
- # optimizefindnode should ensure that fieldvalue is found
assert isinstance(value, AbstractVirtualValue)
fieldvalue = value.getfield(op.getdescr(), None)
if fieldvalue is None:
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py Sat Oct 30 16:58:59 2010
@@ -1746,8 +1746,7 @@
def designate_target_loop(self, gmp):
loop_token = gmp.target_loop_token
num_green_args = self.jitdriver_sd.num_green_args
- residual_args = self.get_residual_args(loop_token.specnodes,
- gmp.argboxes[num_green_args:])
+ residual_args = gmp.argboxes[num_green_args:]
history.set_future_values(self.cpu, residual_args)
return loop_token
@@ -1854,16 +1853,6 @@
if target_loop_token is not loop_tokens[0]:
compile.giveup()
- def get_residual_args(self, specnodes, args):
- if specnodes is None: # it is None only for tests
- return args
- assert len(specnodes) == len(args)
- expanded_args = []
- for i in range(len(specnodes)):
- specnode = specnodes[i]
- specnode.extract_runtime_data(self.cpu, args[i], expanded_args)
- return expanded_args
-
@specialize.arg(1)
def initialize_original_boxes(self, jitdriver_sd, *args):
original_boxes = []
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py Sat Oct 30 16:58:59 2010
@@ -1,32 +1,29 @@
from pypy.jit.metainterp.history import LoopToken, ConstInt, History, Stats
from pypy.jit.metainterp.history import BoxInt, INT
-from pypy.jit.metainterp.specnode import NotSpecNode, ConstantSpecNode
from pypy.jit.metainterp.compile import insert_loop_token, compile_new_loop
from pypy.jit.metainterp.compile import ResumeGuardDescr
from pypy.jit.metainterp.compile import ResumeGuardCountersInt
from pypy.jit.metainterp.compile import compile_tmp_callback
from pypy.jit.metainterp import optimize_nopspec, jitprof, typesystem, compile
-from pypy.jit.metainterp.test.test_optimizefindnode import LLtypeMixin
+from pypy.jit.metainterp.test.test_optimizeutil import LLtypeMixin
from pypy.jit.tool.oparser import parse
def test_insert_loop_token():
+ # XXX this test is a bit useless now that there are no specnodes
lst = []
#
tok1 = LoopToken()
- tok1.specnodes = [NotSpecNode()]
insert_loop_token(lst, tok1)
assert lst == [tok1]
#
tok2 = LoopToken()
- tok2.specnodes = [ConstantSpecNode(ConstInt(8))]
insert_loop_token(lst, tok2)
- assert lst == [tok2, tok1]
+ assert lst == [tok1, tok2]
#
tok3 = LoopToken()
- tok3.specnodes = [ConstantSpecNode(ConstInt(-13))]
insert_loop_token(lst, tok3)
- assert lst == [tok2, tok3, tok1]
+ assert lst == [tok1, tok2, tok3]
class FakeCPU:
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py Sat Oct 30 16:58:59 2010
@@ -1,9 +1,8 @@
import py
from pypy.rlib.objectmodel import instantiate
-from pypy.jit.metainterp.test.test_optimizefindnode import (LLtypeMixin,
- #OOtypeMixin,
- BaseTest)
-from pypy.jit.metainterp.optimizefindnode import PerfectSpecializationFinder
+from pypy.jit.metainterp.test.test_optimizeutil import (LLtypeMixin,
+ #OOtypeMixin,
+ BaseTest)
import pypy.jit.metainterp.optimizeopt.optimizer as optimizeopt
import pypy.jit.metainterp.optimizeopt.virtualize as virtualize
from pypy.jit.metainterp.optimizeutil import InvalidLoop
@@ -244,9 +243,8 @@
assert equaloplists(optimized.operations,
expected.operations, False, remap)
- def optimize_loop(self, ops, spectext, optops, checkspecnodes=True):
+ def optimize_loop(self, ops, spectext, optops):
loop = self.parse(ops)
- loop.token.specnodes = self.unpack_specnodes(spectext)
#
self.loop = loop
metainterp_sd = FakeMetaInterpStaticData(self.cpu)
@@ -858,7 +856,7 @@
"""
py.test.skip("XXX")
self.optimize_loop(ops, 'Not, Virtual(node_vtable, floatdescr=Not)',
- expected, checkspecnodes=False)
+ expected)
def test_virtual_2(self):
ops = """
@@ -913,7 +911,7 @@
self.optimize_loop(ops, '''Virtual(node_vtable),
Virtual(node_vtable),
Not''',
- expected, checkspecnodes=False)
+ expected)
#
# to be complete, we also check the no-opt case where most comparisons
# are not removed. The exact set of comparisons removed depends on
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 16:58:59 2010
@@ -1,9 +1,8 @@
import py
from pypy.rlib.objectmodel import instantiate
-from pypy.jit.metainterp.test.test_optimizefindnode import (LLtypeMixin,
- #OOtypeMixin,
- BaseTest)
-from pypy.jit.metainterp.optimizefindnode import PerfectSpecializationFinder
+from pypy.jit.metainterp.test.test_optimizeutil import (LLtypeMixin,
+ #OOtypeMixin,
+ BaseTest)
import pypy.jit.metainterp.optimizeopt.optimizer as optimizeopt
import pypy.jit.metainterp.optimizeopt.virtualize as virtualize
from pypy.jit.metainterp.optimizeopt import optimize_loop_1
@@ -67,7 +66,6 @@
def optimize_loop(self, ops, spectext, optops, expected_preamble=None):
loop = self.parse(ops)
- loop.token.specnodes = self.unpack_specnodes(spectext)
#
self.loop = loop
loop.preamble = TreeLoop('preamble')
Copied: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeutil.py (from r78527, pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizefindnode.py)
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizefindnode.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeutil.py Sat Oct 30 16:58:59 2010
@@ -8,17 +8,10 @@
from pypy.jit.metainterp.history import (BoxInt, BoxPtr, ConstInt, ConstPtr,
Const, TreeLoop, BoxObj,
ConstObj, AbstractDescr)
-from pypy.jit.metainterp.optimizefindnode import PerfectSpecializationFinder
-from pypy.jit.metainterp.optimizefindnode import BridgeSpecializationFinder
from pypy.jit.metainterp.optimizeutil import sort_descrs, InvalidLoop
-from pypy.jit.metainterp.specnode import NotSpecNode, prebuiltNotSpecNode
-from pypy.jit.metainterp.specnode import VirtualInstanceSpecNode
-from pypy.jit.metainterp.specnode import VirtualArraySpecNode
-from pypy.jit.metainterp.specnode import VirtualStructSpecNode
-from pypy.jit.metainterp.specnode import ConstantSpecNode
from pypy.jit.codewriter.effectinfo import EffectInfo
from pypy.jit.codewriter.heaptracker import register_known_gctype, adr2int
-from pypy.jit.metainterp.test.oparser import parse
+from pypy.jit.tool.oparser import parse
def test_sort_descrs():
class PseudoDescr(AbstractDescr):
@@ -245,956 +238,5 @@
boxkinds=boxkinds,
invent_fail_descr=self.invent_fail_descr)
- def unpack_specnodes(self, text):
- #
- def constclass(cls_vtable):
- if self.type_system == 'lltype':
- return ConstInt(adr2int(llmemory.cast_ptr_to_adr(cls_vtable)))
- else:
- return ConstObj(ootype.cast_to_object(cls_vtable))
- def constant(value):
- if isinstance(lltype.typeOf(value), lltype.Ptr):
- return ConstPtr(value)
- elif isinstance(ootype.typeOf(value), ootype.OOType):
- return ConstObj(ootype.cast_to_object(value))
- else:
- return ConstInt(value)
-
- def parsefields(kwds_fields):
- fields = []
- for key, value in kwds_fields.items():
- fields.append((self.namespace[key], value))
- fields.sort(key = lambda (x, _): x.sort_key())
- return fields
- def makeConstant(value):
- return ConstantSpecNode(constant(value))
- def makeVirtual(cls_vtable, **kwds_fields):
- fields = parsefields(kwds_fields)
- return VirtualInstanceSpecNode(constclass(cls_vtable), fields)
- def makeVirtualArray(arraydescr, *items):
- return VirtualArraySpecNode(arraydescr, items)
- def makeVirtualStruct(typedescr, **kwds_fields):
- fields = parsefields(kwds_fields)
- return VirtualStructSpecNode(typedescr, fields)
- #
- context = {'Not': prebuiltNotSpecNode,
- 'Constant': makeConstant,
- 'Virtual': makeVirtual,
- 'VArray': makeVirtualArray,
- 'VStruct': makeVirtualStruct}
- lst = eval('[' + text + ']', self.namespace, context)
- return lst
-
- def check_specnodes(self, specnodes, text):
- lst = self.unpack_specnodes(text)
- assert len(specnodes) == len(lst)
- for x, y in zip(specnodes, lst):
- assert x.equals(y, ge=False)
- return True
-
# ____________________________________________________________
-class BaseTestOptimizeFindNode(BaseTest):
-
- def find_nodes(self, ops, spectext, boxkinds=None):
- assert boxkinds is None or isinstance(boxkinds, dict)
- loop = self.parse(ops, boxkinds=boxkinds)
- perfect_specialization_finder = PerfectSpecializationFinder(self.cpu)
- perfect_specialization_finder.find_nodes_loop(loop)
- self.check_specnodes(loop.token.specnodes, spectext)
- return (loop.getboxes(), perfect_specialization_finder.getnode)
-
- def test_find_nodes_simple(self):
- ops = """
- [i]
- i0 = int_sub(i, 1)
- guard_value(i0, 0) [i0]
- jump(i0)
- """
- boxes, getnode = self.find_nodes(ops, 'Not')
- assert getnode(boxes.i).fromstart
- assert not getnode(boxes.i0).fromstart
-
- def test_find_nodes_non_escape(self):
- ops = """
- [p0]
- p1 = getfield_gc(p0, descr=nextdescr)
- i0 = getfield_gc(p1, descr=valuedescr)
- i1 = int_sub(i0, 1)
- p2 = getfield_gc(p0, descr=nextdescr)
- setfield_gc(p2, i1, descr=valuedescr)
- p3 = new_with_vtable(ConstClass(node_vtable))
- jump(p3)
- """
- boxes, getnode = self.find_nodes(ops,
- 'Virtual(node_vtable, nextdescr=Not)')
- assert not getnode(boxes.p0).escaped
- assert getnode(boxes.p1).escaped
- assert getnode(boxes.p2).escaped
- assert getnode(boxes.p0).fromstart
- assert getnode(boxes.p1).fromstart
- assert getnode(boxes.p2).fromstart
-
- def test_find_nodes_escape(self):
- ops = """
- [p0]
- p1 = getfield_gc(p0, descr=nextdescr)
- p2 = getfield_gc(p1, descr=nextdescr)
- i0 = getfield_gc(p2, descr=valuedescr)
- i1 = int_sub(i0, 1)
- escape(p1)
- p3 = getfield_gc(p0, descr=nextdescr)
- setfield_gc(p3, i1, descr=valuedescr)
- p4 = getfield_gc(p1, descr=nextdescr)
- setfield_gc(p4, i1, descr=valuedescr)
- p5 = new_with_vtable(ConstClass(node_vtable))
- jump(p5)
- """
- boxes, getnode = self.find_nodes(ops,
- 'Virtual(node_vtable, nextdescr=Not)')
- assert not getnode(boxes.p0).escaped
- assert getnode(boxes.p1).escaped
- assert getnode(boxes.p2).escaped # forced by p1
- assert getnode(boxes.p3).escaped # forced because p3 == p1
- assert getnode(boxes.p4).escaped # forced by p1
- assert getnode(boxes.p0).fromstart
- assert getnode(boxes.p1).fromstart
- assert getnode(boxes.p2).fromstart
- assert getnode(boxes.p3).fromstart
- assert not getnode(boxes.p4).fromstart
-
- def test_find_nodes_new_1(self):
- ops = """
- [p1]
- p2 = new_with_vtable(ConstClass(node_vtable))
- jump(p2)
- """
- boxes, getnode = self.find_nodes(ops, 'Virtual(node_vtable)')
-
- boxp1 = getnode(boxes.p1)
- boxp2 = getnode(boxes.p2)
- assert not boxp1.escaped
- assert not boxp2.escaped
-
- assert not boxp1.origfields
- assert not boxp1.curfields
- assert not boxp2.origfields
- assert not boxp2.curfields
-
- assert boxp1.fromstart
- assert not boxp2.fromstart
-
- assert boxp1.knownclsbox is None
- assert boxp2.knownclsbox.getaddr() == self.node_vtable_adr
-
- def test_find_nodes_new_2(self):
- ops = """
- [i1, p1]
- p2 = new_with_vtable(ConstClass(node_vtable))
- p3 = new_with_vtable(ConstClass(node_vtable2))
- setfield_gc(p2, p3, descr=nextdescr)
- setfield_gc(p3, i1, descr=valuedescr)
- jump(i1, p2)
- """
- self.find_nodes(ops,
- '''Not,
- Virtual(node_vtable,
- nextdescr=Virtual(node_vtable2,
- valuedescr=Not))''')
-
- def test_find_nodes_new_3(self):
- ops = """
- [sum, p1]
- guard_class(p1, ConstClass(node_vtable)) []
- i1 = getfield_gc(p1, descr=valuedescr)
- i2 = int_sub(i1, 1)
- sum2 = int_add(sum, i1)
- p2 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p2, i2, descr=valuedescr)
- p3 = new_with_vtable(ConstClass(node_vtable2))
- setfield_gc(p2, p3, descr=nextdescr)
- jump(sum2, p2)
- """
- boxes, getnode = self.find_nodes(
- ops,
- '''Not,
- Virtual(node_vtable,
- valuedescr=Not,
- nextdescr=Virtual(node_vtable2))''',
- boxkinds={'sum': BoxInt, 'sum2': BoxInt})
- assert getnode(boxes.sum) is not getnode(boxes.sum2)
- assert getnode(boxes.p1) is not getnode(boxes.p2)
-
- boxp1 = getnode(boxes.p1)
- boxp2 = getnode(boxes.p2)
- boxp3 = getnode(boxes.p3)
- assert not boxp1.escaped
- assert not boxp2.escaped
- assert not boxp3.escaped
-
- assert not boxp1.curfields
- assert boxp1.origfields[self.valuedescr] is getnode(boxes.i1)
- assert not boxp2.origfields
- assert boxp2.curfields[self.nextdescr] is boxp3
-
- assert boxp1.fromstart
- assert not boxp2.fromstart
- assert not boxp3.fromstart
-
- assert boxp2.knownclsbox.getaddr() == self.node_vtable_adr
- assert boxp3.knownclsbox.getaddr() == self.node_vtable_adr2
-
- def test_find_nodes_new_aliasing_0(self):
- ops = """
- [p1, p2]
- p3 = new_with_vtable(ConstClass(node_vtable))
- jump(p3, p3)
- """
- # both p1 and p2 must be NotSpecNodes; it's not possible to pass
- # the same Virtual both in p1 and p2 (at least so far).
- self.find_nodes(ops, 'Not, Not')
-
- def test_find_nodes_new_aliasing_1(self):
- ops = """
- [sum, p1]
- guard_class(p1, ConstClass(node_vtable)) []
- p3 = getfield_gc(p1, descr=nextdescr)
- guard_class(p3, ConstClass(node_vtable)) []
- i1 = getfield_gc(p1, descr=valuedescr)
- i2 = int_sub(i1, 1)
- sum2 = int_add(sum, i1)
- p2 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p2, i2, descr=valuedescr)
- setfield_gc(p2, p2, descr=nextdescr)
- jump(sum2, p2)
- """
- # the issue is the cycle "p2->p2", which cannot be represented
- # with SpecNodes so far
- self.find_nodes(ops, 'Not, Not',
- boxkinds={'sum': BoxInt, 'sum2': BoxInt})
-
- def test_find_nodes_new_aliasing_2(self):
- ops = """
- [p1, p2]
- escape(p2)
- p3 = new_with_vtable(ConstClass(node_vtable))
- jump(p3, p3)
- """
- # both p1 and p2 must be NotSpecNodes; it's not possible to pass
- # in p1 a Virtual and not in p2, as they both come from the same p3.
- self.find_nodes(ops, 'Not, Not')
-
- def test_find_nodes_new_mismatch(self):
- ops = """
- [p1]
- guard_class(p1, ConstClass(node_vtable)) []
- p2 = new_with_vtable(ConstClass(node_vtable2))
- jump(p2)
- """
- # this is not a valid loop at all, because of the mismatch
- # between the produced and the consumed class.
- py.test.raises(InvalidLoop, self.find_nodes, ops, None)
-
- def test_find_nodes_new_aliasing_mismatch(self):
- ops = """
- [p0, p1]
- guard_class(p0, ConstClass(node_vtable)) []
- guard_class(p1, ConstClass(node_vtable2)) []
- p2 = new_with_vtable(ConstClass(node_vtable2))
- jump(p2, p2)
- """
- # this is also not really a valid loop, but it's not detected
- # because p2 is passed more than once in the jump().
- self.find_nodes(ops, 'Not, Not')
-
- def test_find_nodes_new_escapes(self):
- ops = """
- [p0]
- escape(p0)
- p1 = new_with_vtable(ConstClass(node_vtable))
- jump(p1)
- """
- self.find_nodes(ops, 'Not')
-
- def test_find_nodes_new_unused(self):
- ops = """
- [p0]
- p1 = new_with_vtable(ConstClass(node_vtable))
- p2 = new_with_vtable(ConstClass(node_vtable))
- p3 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, p2, descr=nextdescr)
- setfield_gc(p2, p3, descr=nextdescr)
- jump(p1)
- """
- self.find_nodes(ops, '''
- Virtual(node_vtable,
- nextdescr=Virtual(node_vtable,
- nextdescr=Virtual(node_vtable)))''')
-
- def test_find_nodes_ptr_eq(self):
- ops = """
- [p3, p4, p2]
- p0 = new_with_vtable(ConstClass(node_vtable))
- p1 = new_with_vtable(ConstClass(node_vtable))
- guard_nonnull(p0) []
- i3 = ptr_ne(p0, NULL)
- guard_true(i3) []
- i4 = ptr_eq(p0, NULL)
- guard_false(i4) []
- i5 = ptr_ne(NULL, p0)
- guard_true(i5) []
- i6 = ptr_eq(NULL, p0)
- guard_false(i6) []
- i7 = ptr_ne(p0, p1)
- guard_true(i7) []
- i8 = ptr_eq(p0, p1)
- guard_false(i8) []
- i9 = ptr_ne(p0, p2)
- guard_true(i9) []
- i10 = ptr_eq(p0, p2)
- guard_false(i10) []
- i11 = ptr_ne(p2, p1)
- guard_true(i11) []
- i12 = ptr_eq(p2, p1)
- guard_false(i12) []
- jump(p0, p1, p2)
- """
- self.find_nodes(ops, '''Virtual(node_vtable),
- Virtual(node_vtable),
- Not''')
-
- def test_find_nodes_call(self):
- ops = """
- [i0, p2]
- p0 = new_with_vtable(ConstClass(node_vtable))
- i1 = call_pure(i0, p0) # forces p0 to not be virtual
- jump(i1, p0)
- """
- self.find_nodes(ops, 'Not, Not')
-
- def test_find_nodes_default_field(self):
- ops = """
- [p0]
- i0 = getfield_gc(p0, descr=valuedescr)
- guard_value(i0, 5) []
- p1 = new_with_vtable(ConstClass(node_vtable))
- # the field 'value' has its default value of 0
- jump(p1)
- """
- # The answer must contain the 'value' field, because otherwise
- # we might get incorrect results: when tracing, i0 was 5.
- self.find_nodes(ops, 'Virtual(node_vtable, valuedescr=Not)')
-
- def test_find_nodes_nonvirtual_guard_class(self):
- ops = """
- [p1]
- guard_class(p1, ConstClass(node_vtable)) [p1]
- jump(p1)
- """
- self.find_nodes(ops, 'Not')
-
- def test_find_nodes_p12_simple(self):
- ops = """
- [p1]
- i3 = getfield_gc(p1, descr=valuedescr)
- escape(i3)
- jump(p1)
- """
- self.find_nodes(ops, 'Not')
-
- def test_find_nodes_p123_simple(self):
- ops = """
- [i1, p2, p3]
- i3 = getfield_gc(p3, descr=valuedescr)
- escape(i3)
- p1 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, i1, descr=valuedescr)
- jump(i1, p1, p2)
- """
- # We cannot track virtuals that survive for more than two iterations.
- self.find_nodes(ops, 'Not, Not, Not')
-
- def test_find_nodes_p1234_simple(self):
- ops = """
- [i1, p2, p3, p4]
- i4 = getfield_gc(p4, descr=valuedescr)
- escape(i4)
- p1 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, i1, descr=valuedescr)
- jump(i1, p1, p2, p3)
- """
- # We cannot track virtuals that survive for more than two iterations.
- self.find_nodes(ops, 'Not, Not, Not, Not')
-
- def test_find_nodes_p123_guard_class(self):
- ops = """
- [i1, p2, p3]
- guard_class(p3, ConstClass(node_vtable)) [i1, p2, p3]
- i3 = getfield_gc(p3, descr=valuedescr)
- escape(i3)
- p1 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, i1, descr=valuedescr)
- jump(i1, p1, p2)
- """
- # We cannot track virtuals that survive for more than two iterations.
- self.find_nodes(ops, 'Not, Not, Not')
-
- def test_find_nodes_p123_rec(self):
- ops = """
- [i1, p2, p0d]
- p3 = getfield_gc(p0d, descr=nextdescr)
- i3 = getfield_gc(p3, descr=valuedescr)
- escape(i3)
- p1 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, i1, descr=valuedescr)
- p0c = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p0c, p2, descr=nextdescr)
- jump(i1, p1, p0c)
- """
- # We cannot track virtuals that survive for more than two iterations.
- self.find_nodes(ops, '''Not,
- Not,
- Virtual(node_vtable, nextdescr=Not)''')
-
- def test_find_nodes_setfield_bug(self):
- ops = """
- [p1, p2]
- escape(p1)
- setfield_gc(p1, p2, descr=nextdescr)
- p3 = new_with_vtable(ConstClass(node_vtable))
- jump(p1, p3)
- """
- self.find_nodes(ops, 'Not, Not')
-
- def test_find_nodes_array_virtual_1(self):
- ops = """
- [i1, p2]
- i2 = getarrayitem_gc(p2, 1, descr=arraydescr)
- escape(i2)
- p3 = new_array(3, descr=arraydescr)
- setarrayitem_gc(p3, 1, i1, descr=arraydescr)
- jump(i1, p3)
- """
- self.find_nodes(ops, 'Not, VArray(arraydescr, Not, Not, Not)')
-
- def test_find_nodes_array_virtual_2(self):
- ops = """
- [i1, p2]
- i2 = arraylen_gc(p2, descr=arraydescr)
- escape(i2)
- p3 = new_array(3, descr=arraydescr)
- setarrayitem_gc(p3, 1, i1, descr=arraydescr)
- jump(i1, p3)
- """
- self.find_nodes(ops, 'Not, VArray(arraydescr, Not, Not, Not)')
-
- def test_find_nodes_array_virtual_3(self):
- ops = """
- [pvalue1, p2]
- pvalue2 = new_with_vtable(ConstClass(node_vtable2))
- ps2 = getarrayitem_gc(p2, 1, descr=arraydescr)
- setfield_gc(ps2, pvalue2, descr=nextdescr)
- ps3 = getarrayitem_gc(p2, 1, descr=arraydescr)
- pvalue3 = getfield_gc(ps3, descr=nextdescr)
- ps1 = new_with_vtable(ConstClass(node_vtable))
- p3 = new_array(3, descr=arraydescr)
- setarrayitem_gc(p3, 1, ps1, descr=arraydescr)
- jump(pvalue3, p3)
- """
- self.find_nodes(ops, 'Virtual(node_vtable2), VArray(arraydescr, Not, Virtual(node_vtable), Not)')
-
- def test_find_nodes_array_virtual_empty(self):
- ops = """
- [i1, p2]
- p3 = new_array(3, descr=arraydescr)
- jump(i1, p3)
- """
- self.find_nodes(ops, 'Not, VArray(arraydescr, Not, Not, Not)')
-
- def test_find_nodes_array_nonvirtual_1(self):
- ops = """
- [i1, p2]
- i2 = getarrayitem_gc(p2, i1, descr=arraydescr)
- escape(i2)
- p3 = new_array(4, descr=arraydescr)
- setarrayitem_gc(p3, i1, i2, descr=arraydescr)
- jump(i1, p3)
- """
- # Does not work because of the variable index, 'i1'.
- self.find_nodes(ops, 'Not, Not')
-
- def test_find_nodes_array_forced_1(self):
- ops = """
- [p1, i1]
- p2 = new_array(1, descr=arraydescr)
- setarrayitem_gc(p2, 0, p1, descr=arraydescr)
- p3 = getarrayitem_gc(p2, i1, descr=arraydescr)
- p4 = new_with_vtable(ConstClass(node_vtable))
- jump(p4, i1)
- """
- # escapes because getarrayitem_gc uses a non-constant index
- self.find_nodes(ops, 'Not, Not')
-
- def test_find_nodes_arrayitem_forced(self):
- ops = """
- [p1]
- p2 = new_array(1, descr=arraydescr)
- escape(p2)
- p4 = new_with_vtable(ConstClass(node_vtable))
- setarrayitem_gc(p2, 0, p4, descr=arraydescr)
- jump(p4)
- """
- self.find_nodes(ops, 'Not')
-
- def test_find_nodes_struct_virtual_1(self):
- ops = """
- [i1, p2]
- i2 = getfield_gc(p2, descr=adescr)
- escape(i2)
- p3 = new(descr=ssize)
- setfield_gc(p3, i1, descr=adescr)
- jump(i1, p3)
- """
- self.find_nodes(ops, 'Not, VStruct(ssize, adescr=Not)')
-
- def test_find_nodes_struct_nonvirtual_1(self):
- ops = """
- [i1, p2]
- i2 = getfield_gc(p2, descr=adescr)
- escape(p2)
- p3 = new(descr=ssize)
- setfield_gc(p3, i1, descr=adescr)
- jump(i1, p3)
- """
- self.find_nodes(ops, 'Not, Not')
-
- def test_find_nodes_guard_value_constant(self):
- ops = """
- [p1]
- guard_value(p1, ConstPtr(myptr)) []
- jump(ConstPtr(myptr))
- """
- self.find_nodes(ops, 'Constant(myptr)')
-
- def test_find_nodes_guard_value_constant_mismatch(self):
- ops = """
- [p1]
- guard_value(p1, ConstPtr(myptr2)) []
- jump(ConstPtr(myptr))
- """
- py.test.raises(InvalidLoop, self.find_nodes, ops, None)
-
- def test_find_nodes_guard_value_escaping_constant(self):
- ops = """
- [p1]
- escape(p1)
- guard_value(p1, ConstPtr(myptr)) []
- jump(ConstPtr(myptr))
- """
- self.find_nodes(ops, 'Constant(myptr)')
-
- def test_find_nodes_guard_value_same_as_constant(self):
- ops = """
- [p1]
- guard_value(p1, ConstPtr(myptr)) []
- p2 = same_as(ConstPtr(myptr))
- jump(p2)
- """
- self.find_nodes(ops, 'Constant(myptr)')
-
- def test_find_nodes_store_into_loop_constant_1(self):
- ops = """
- [i0, p1, p4]
- p2 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, p2, descr=nextdescr)
- jump(i0, p1, p2)
- """
- self.find_nodes(ops, 'Not, Not, Not')
-
- def test_find_nodes_store_into_loop_constant_2(self):
- ops = """
- [i0, p4, p1]
- p2 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, p2, descr=nextdescr)
- jump(i0, p2, p1)
- """
- self.find_nodes(ops, 'Not, Not, Not')
-
- def test_find_nodes_store_into_loop_constant_3(self):
- ops = """
- [i0, p1]
- p2 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, p2, descr=nextdescr)
- call(i0)
- jump(i0, p1)
- """
- self.find_nodes(ops, 'Not, Not')
-
- def test_find_nodes_arithmetic_propagation_bug_0(self):
- ops = """
- [p1]
- i1 = getarrayitem_gc(p1, 0, descr=arraydescr)
- escape(i1)
- i2 = int_add(0, 1)
- p2 = new_array(i2, descr=arraydescr)
- i3 = escape()
- setarrayitem_gc(p2, 0, i3, descr=arraydescr)
- jump(p2)
- """
- self.find_nodes(ops, 'VArray(arraydescr, Not)')
-
- def test_find_nodes_arithmetic_propagation_bug_1(self):
- ops = """
- [p1]
- i1 = getarrayitem_gc(p1, 0, descr=arraydescr)
- escape(i1)
- i2 = same_as(1)
- p2 = new_array(i2, descr=arraydescr)
- setarrayitem_gc(p2, 0, 5)
- jump(p2)
- """
- self.find_nodes(ops, 'VArray(arraydescr, Not)')
-
- def test_find_nodes_arithmetic_propagation_bug_2(self):
- ops = """
- [p1]
- i0 = int_sub(17, 17)
- i1 = getarrayitem_gc(p1, i0, descr=arraydescr)
- escape(i1)
- i2 = int_add(0, 1)
- p2 = new_array(i2, descr=arraydescr)
- i3 = escape()
- setarrayitem_gc(p2, i0, i3, descr=arraydescr)
- jump(p2)
- """
- self.find_nodes(ops, 'VArray(arraydescr, Not)')
-
- def test_find_nodes_arithmetic_propagation_bug_3(self):
- ops = """
- [p1]
- i1 = getarrayitem_gc(p1, 0, descr=arraydescr)
- escape(i1)
- p3 = new_array(1, descr=arraydescr)
- i2 = arraylen_gc(p3, descr=arraydescr)
- p2 = new_array(i2, descr=arraydescr)
- i3 = escape()
- setarrayitem_gc(p2, 0, i3, descr=arraydescr)
- jump(p2)
- """
- self.find_nodes(ops, 'VArray(arraydescr, Not)')
-
- def test_find_nodes_bug_1(self):
- ops = """
- [p12]
- guard_nonnull(p12) []
- guard_class(p12, ConstClass(node_vtable)) []
- guard_class(p12, ConstClass(node_vtable)) []
- i22 = getfield_gc_pure(p12, descr=valuedescr)
- escape(i22)
- guard_nonnull(p12) []
- guard_class(p12, ConstClass(node_vtable)) []
- guard_class(p12, ConstClass(node_vtable)) []
- i29 = getfield_gc_pure(p12, descr=valuedescr)
- i31 = int_add_ovf(i29, 1)
- guard_no_overflow() []
- p33 = new_with_vtable(ConstClass(node_vtable)) # NODE
- setfield_gc(p33, i31, descr=valuedescr)
- #
- p35 = new_array(1, descr=arraydescr3) # Array(NODE)
- setarrayitem_gc(p35, 0, p33, descr=arraydescr3)
- p38 = new_with_vtable(ConstClass(u_vtable)) # U
- setfield_gc(p38, p35, descr=onedescr)
- guard_nonnull(p38) []
- guard_nonnull(p38) []
- guard_class(p38, ConstClass(u_vtable)) []
- p42 = getfield_gc(p38, descr=onedescr) # Array(NODE)
- i43 = arraylen_gc(p42, descr=arraydescr3)
- i45 = int_sub(i43, 0)
- p46 = new(descr=tsize) # T
- setfield_gc(p46, i45, descr=cdescr)
- p47 = new_array(i45, descr=arraydescr3) # Array(NODE)
- setfield_gc(p46, p47, descr=ddescr)
- i48 = int_lt(0, i43)
- guard_true(i48) []
- p49 = getarrayitem_gc(p42, 0, descr=arraydescr3) # NODE
- p50 = getfield_gc(p46, descr=ddescr) # Array(NODE)
- setarrayitem_gc(p50, 0, p49, descr=arraydescr3)
- i52 = int_lt(1, i43)
- guard_false(i52) []
- i53 = getfield_gc(p46, descr=cdescr)
- i55 = int_ne(i53, 1)
- guard_false(i55) []
- p56 = getfield_gc(p46, descr=ddescr) # Array(NODE)
- p58 = getarrayitem_gc(p56, 0, descr=arraydescr3) # NODE
- guard_nonnull(p38) []
- jump(p58)
- """
- self.find_nodes(ops, 'Virtual(node_vtable, valuedescr=Not)')
-
- # ------------------------------
- # Bridge tests
-
- def find_bridge(self, ops, inputspectext, outputspectext, boxkinds=None,
- mismatch=False):
- assert boxkinds is None or isinstance(boxkinds, dict)
- inputspecnodes = self.unpack_specnodes(inputspectext)
- outputspecnodes = self.unpack_specnodes(outputspectext)
- bridge = self.parse(ops, boxkinds=boxkinds)
- bridge_specialization_finder = BridgeSpecializationFinder(self.cpu)
- bridge_specialization_finder.find_nodes_bridge(bridge, inputspecnodes)
- matches = bridge_specialization_finder.bridge_matches(outputspecnodes)
- if mismatch:
- assert not matches
- else:
- assert matches
-
- def test_bridge_simple(self):
- ops = """
- [i0]
- i1 = int_add(i0, 1)
- jump(i1)
- """
- self.find_bridge(ops, 'Not', 'Not')
- self.find_bridge(ops, 'Not', 'Virtual(node_vtable)', mismatch=True)
-
- def test_bridge_simple_known_class(self):
- ops = """
- [p0]
- setfield_gc(p0, 123, descr=valuedescr)
- jump(p0)
- """
- self.find_bridge(ops, 'Not', 'Not')
-
- def test_bridge_simple_constant(self):
- ops = """
- []
- jump(ConstPtr(myptr))
- """
- self.find_bridge(ops, '', 'Not')
- self.find_bridge(ops, '', 'Constant(myptr)')
- self.find_bridge(ops, '', 'Constant(myptr2)', mismatch=True)
-
- def test_bridge_simple_constant_mismatch(self):
- ops = """
- [p0]
- jump(p0)
- """
- self.find_bridge(ops, 'Not', 'Not')
- self.find_bridge(ops, 'Not', 'Constant(myptr)', mismatch=True)
-
- def test_bridge_simple_virtual_1(self):
- ops = """
- [i0]
- p0 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p0, i0, descr=valuedescr)
- jump(p0)
- """
- self.find_bridge(ops, 'Not', 'Not')
- self.find_bridge(ops, 'Not', 'Virtual(node_vtable, valuedescr=Not)')
- self.find_bridge(ops, 'Not',
- '''Virtual(node_vtable,
- valuedescr=Not,
- nextdescr=Not)''')
- #
- self.find_bridge(ops, 'Not', 'Virtual(node_vtable)',
- mismatch=True) # missing valuedescr
- self.find_bridge(ops, 'Not', 'Virtual(node_vtable, nextdescr=Not)',
- mismatch=True) # missing valuedescr
- self.find_bridge(ops, 'Not', 'Virtual(node_vtable2, valuedescr=Not)',
- mismatch=True) # bad class
-
- def test_bridge_simple_virtual_struct(self):
- ops = """
- [i0]
- p0 = new(descr=ssize)
- setfield_gc(p0, i0, descr=adescr)
- jump(p0)
- """
- self.find_bridge(ops, 'Not', 'Not')
- self.find_bridge(ops, 'Not', 'VStruct(ssize, adescr=Not)')
-
- def test_bridge_simple_virtual_struct_non_unique(self):
- ops = """
- [i0]
- p0 = new(descr=ssize)
- setfield_gc(p0, i0, descr=adescr)
- jump(p0, p0)
- """
- self.find_bridge(ops, 'Not', 'Not, Not')
- self.find_bridge(ops, 'Not', 'VStruct(ssize), VStruct(ssize)',
- mismatch=True)
-
-
- def test_bridge_simple_virtual_2(self):
- ops = """
- [p0]
- setfield_gc(p0, 123, descr=valuedescr)
- jump(p0)
- """
- self.find_bridge(ops, 'Virtual(node_vtable)', 'Not')
- self.find_bridge(ops, 'Virtual(node_vtable)',
- 'Virtual(node_vtable, valuedescr=Not)')
- self.find_bridge(ops, 'Virtual(node_vtable, valuedescr=Not)',
- 'Virtual(node_vtable, valuedescr=Not)')
- self.find_bridge(ops, 'Virtual(node_vtable, valuedescr=Not)',
- '''Virtual(node_vtable,
- valuedescr=Not,
- nextdescr=Not)''')
- self.find_bridge(ops, '''Virtual(node_vtable,
- valuedescr=Not,
- nextdescr=Not)''',
- '''Virtual(node_vtable,
- valuedescr=Not,
- nextdescr=Not)''')
- #
- self.find_bridge(ops, 'Virtual(node_vtable)', 'Virtual(node_vtable)',
- mismatch=True) # because of missing valuedescr
- self.find_bridge(ops, 'Virtual(node_vtable)',
- 'Virtual(node_vtable2, valuedescr=Not)',
- mismatch=True) # bad class
-
- def test_bridge_virtual_mismatch_1(self):
- ops = """
- [i0]
- p0 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p0, i0, descr=valuedescr)
- jump(p0, p0)
- """
- self.find_bridge(ops, 'Not', 'Not, Not')
- #
- self.find_bridge(ops, 'Not',
- '''Virtual(node_vtable, valuedescr=Not),
- Virtual(node_vtable, valuedescr=Not)''',
- mismatch=True) # duplicate p0
-
- def test_bridge_guard_class(self):
- ops = """
- [p1]
- p2 = getfield_gc(p1, descr=nextdescr)
- guard_class(p2, ConstClass(node_vtable)) []
- jump(p2)
- """
- self.find_bridge(ops, 'Not', 'Not')
- self.find_bridge(ops, 'Virtual(node_vtable2, nextdescr=Not)', 'Not')
- self.find_bridge(ops,
- '''Virtual(node_vtable,
- nextdescr=Virtual(node_vtable,
- nextdescr=Not))''',
- '''Virtual(node_vtable,
- nextdescr=Not)''')
- #
- self.find_bridge(ops, 'Not', 'Virtual(node_vtable)',
- mismatch=True)
-
- def test_bridge_unused(self):
- ops = """
- []
- p1 = new_with_vtable(ConstClass(node_vtable))
- p2 = new_with_vtable(ConstClass(node_vtable))
- p3 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, p2, descr=nextdescr)
- setfield_gc(p2, p3, descr=nextdescr)
- jump(p1)
- """
- self.find_bridge(ops, '',
- '''Not''')
- self.find_bridge(ops, '',
- '''Virtual(node_vtable,
- nextdescr=Not)''')
- self.find_bridge(ops, '',
- '''Virtual(node_vtable,
- nextdescr=Virtual(node_vtable,
- nextdescr=Not))''')
- self.find_bridge(ops, '',
- '''Virtual(node_vtable,
- nextdescr=Virtual(node_vtable,
- nextdescr=Virtual(node_vtable)))''')
- self.find_bridge(ops, '',
- '''Virtual(node_vtable,
- nextdescr=Virtual(node_vtable,
- nextdescr=Virtual(node_vtable,
- nextdescr=Not)))''')
-
- def test_bridge_to_finish(self):
- ops = """
- [i1]
- i2 = int_add(i1, 5)
- finish(i2)
- """
- self.find_bridge(ops, 'Not', 'Not')
-
- def test_bridge_virtual_to_finish(self):
- ops = """
- [i1]
- p1 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, i1, descr=valuedescr)
- finish(p1)
- """
- self.find_bridge(ops, 'Not', 'Not')
- self.find_bridge(ops, 'Not',
- 'Virtual(node_vtable, valuedescr=Not)',
- mismatch=True)
-
- def test_bridge_array_virtual_1(self):
- ops = """
- [i1]
- p1 = new_array(3, descr=arraydescr)
- setarrayitem_gc(p1, 0, i1, descr=arraydescr)
- jump(p1)
- """
- self.find_bridge(ops, 'Not', 'Not')
- self.find_bridge(ops, 'Not', 'VArray(arraydescr, Not, Not, Not)')
-
- def test_bridge_array_virtual_size_mismatch(self):
- ops = """
- [i1]
- p1 = new_array(5, descr=arraydescr)
- setarrayitem_gc(p1, 0, i1, descr=arraydescr)
- jump(p1)
- """
- self.find_bridge(ops, 'Not', 'Not')
- self.find_bridge(ops, 'Not', 'VArray(arraydescr, Not, Not, Not)',
- mismatch=True)
-
- def test_bridge_array_virtual_2(self):
- ops = """
- [i1]
- p1 = new_array(3, descr=arraydescr)
- setarrayitem_gc(p1, 0, i1, descr=arraydescr)
- escape(p1)
- jump(p1)
- """
- self.find_bridge(ops, 'Not', 'Not')
- self.find_bridge(ops, 'Not', 'VArray(arraydescr, Not, Not, Not)',
- mismatch=True)
-
- def test_bridge_nested_structs(self):
- ops = """
- []
- p1 = new_with_vtable(ConstClass(node_vtable))
- p2 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, p2, descr=nextdescr)
- jump(p1)
- """
- self.find_bridge(ops, '', 'Not')
- self.find_bridge(ops, '', 'Virtual(node_vtable, nextdescr=Not)')
- self.find_bridge(ops, '',
- 'Virtual(node_vtable, nextdescr=Virtual(node_vtable))')
- self.find_bridge(ops, '',
- 'Virtual(node_vtable, nextdescr=Virtual(node_vtable2))',
- mismatch=True)
-
-
-class TestLLtype(BaseTestOptimizeFindNode, LLtypeMixin):
- pass
-
-##class TestOOtype(BaseTestOptimizeFindNode, OOtypeMixin):
-## def test_find_nodes_instanceof(self):
-## ops = """
-## [i0]
-## p0 = new_with_vtable(ConstClass(node_vtable))
-## i1 = instanceof(p0, descr=nodesize)
-## jump(i1)
-## """
-## boxes, getnode = self.find_nodes(ops, 'Not')
-## assert not getnode(boxes.p0).escaped
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py Sat Oct 30 16:58:59 2010
@@ -5,7 +5,7 @@
from pypy.jit.metainterp.resume import *
from pypy.jit.metainterp.history import BoxInt, BoxPtr, ConstInt
from pypy.jit.metainterp.history import ConstPtr, ConstFloat
-from pypy.jit.metainterp.test.test_optimizefindnode import LLtypeMixin
+from pypy.jit.metainterp.test.test_optimizeutil import LLtypeMixin
from pypy.jit.metainterp import executor
from pypy.jit.codewriter import heaptracker
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualizable.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualizable.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualizable.py Sat Oct 30 16:58:59 2010
@@ -11,7 +11,7 @@
from pypy.rpython.rclass import FieldListAccessor
from pypy.jit.metainterp.warmspot import get_stats, get_translator
from pypy.jit.metainterp import history
-from pypy.jit.metainterp.test.test_optimizefindnode import LLtypeMixin
+from pypy.jit.metainterp.test.test_optimizeutil import LLtypeMixin
def promote_virtualizable(*args):
pass
From hakanardo at codespeak.net Sat Oct 30 17:11:33 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 17:11:33 +0200 (CEST)
Subject: [pypy-svn] r78534 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030151133.14442282B9D@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 17:11:31 2010
New Revision: 78534
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 17:11:31 2010
@@ -947,11 +947,15 @@
i1 = ptr_eq(p2, NULL)
jump(i1)
"""
- expected = """
+ preamble = """
[i0]
- jump(1)
+ jump()
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ []
+ jump()
+ """
+ self.optimize_loop(ops, 'Not', expected, preamble)
def test_virtual_constant_isnonnull(self):
@@ -963,9 +967,13 @@
i1 = ptr_eq(p2, NULL)
jump(i1)
"""
- expected = """
+ preamble = """
[i0]
- jump(0)
+ jump()
+ """
+ expected = """
+ []
+ jump()
"""
self.optimize_loop(ops, 'Not', expected)
@@ -1001,8 +1009,22 @@
setfield_gc(p1, i1, descr=valuedescr)
jump(i, p1)
"""
- expected = ops
- self.optimize_loop(ops, 'Not, Not', expected)
+ preamble = """
+ [i, p0]
+ i0 = getfield_gc(p0, descr=valuedescr)
+ escape(p0)
+ i1 = int_add(i0, i)
+ jump(i, i1)
+ """
+ expected = """
+ [i, i1]
+ p1 = new_with_vtable(ConstClass(node_vtable))
+ setfield_gc(p1, i1, descr=valuedescr)
+ escape(p1)
+ i2 = int_add(i1, i)
+ jump(i, i2)
+ """
+ self.optimize_loop(ops, 'Not, Not', expected, preamble)
def test_nonvirtual_later(self):
ops = """
@@ -1079,8 +1101,12 @@
i1 = getfield_gc(ConstPtr(myptr), descr=valuedescr)
jump(i1)
"""
- expected = ops
- self.optimize_loop(ops, 'Not', expected)
+ preamble = ops
+ expected = """
+ [i]
+ jump(i)
+ """
+ self.optimize_loop(ops, 'Not', expected, preamble)
def test_varray_1(self):
ops = """
@@ -1107,11 +1133,15 @@
i2 = getarrayitem_gc(p1, 1, descr=arraydescr)
jump(i2)
"""
- expected = """
+ preamble = """
[i1]
- jump(0)
+ jump()
"""
- self.optimize_loop(ops, 'Not', expected)
+ expected = """
+ []
+ jump()
+ """
+ self.optimize_loop(ops, 'Not', expected, preamble)
def test_varray_float(self):
ops = """
@@ -1176,13 +1206,21 @@
setarrayitem_gc(p2, 0, 20, descr=arraydescr)
jump(i0, p2)
"""
- expected = """
- [i0, i1, i2]
+ preamble = """
+ [i0, p1]
+ i1 = getarrayitem_gc(p1, 0, descr=arraydescr)
+ i2 = getarrayitem_gc(p1, 1, descr=arraydescr)
i3 = int_sub(i1, i2)
guard_value(i3, 15) []
- jump(i0, 20, i0)
+ jump(i0)
+ """
+ expected = """
+ [i0]
+ i3 = int_sub(20, i0)
+ guard_value(i3, 15) []
+ jump(5)
"""
- self.optimize_loop(ops, 'Not, VArray(arraydescr, Not, Not)', expected)
+ self.optimize_loop(ops, 'Not, Not', expected, preamble)
def test_p123_array(self):
ops = """
From hakanardo at codespeak.net Sat Oct 30 17:23:16 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 17:23:16 +0200 (CEST)
Subject: [pypy-svn] r78535 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030152316.37B3C282BF4@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 17:23:14 2010
New Revision: 78535
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 17:23:14 2010
@@ -1231,8 +1231,22 @@
setarrayitem_gc(p1, 0, i1, descr=arraydescr)
jump(i1, p1, p2)
"""
+ preamble = """
+ [i1, p2, p3]
+ i3 = getarrayitem_gc(p3, 0, descr=arraydescr)
+ escape(i3)
+ jump(i1, p2)
+ """
+ expected = """
+ [i1, p2]
+ i3 = getarrayitem_gc(p2, 0, descr=arraydescr)
+ escape(i3)
+ p1 = new_array(1, descr=arraydescr)
+ setarrayitem_gc(p1, 0, i1, descr=arraydescr)
+ jump(i1, p1)
+ """
# We cannot track virtuals that survive for more than two iterations.
- self.optimize_loop(ops, 'Not, Not, Not', ops)
+ self.optimize_loop(ops, 'Not, Not, Not', expected, preamble)
def test_varray_forced_1(self):
ops = """
@@ -1265,12 +1279,18 @@
setfield_gc(p3, i1, descr=adescr)
jump(i1, p3)
"""
- expected = """
- [i1, i2]
+ preamble = """
+ [i1, p2]
+ i2 = getfield_gc(p2, descr=adescr)
escape(i2)
- jump(i1, i1)
+ jump(i1)
+ """
+ expected = """
+ [i1]
+ escape(i1)
+ jump(i1)
"""
- self.optimize_loop(ops, 'Not, VStruct(ssize, adescr=Not)', expected)
+ self.optimize_loop(ops, 'Not, Not', expected, preamble)
def test_p123_vstruct(self):
ops = """
@@ -1281,8 +1301,22 @@
setfield_gc(p1, i1, descr=adescr)
jump(i1, p1, p2)
"""
+ preamble = """
+ [i1, p2, p3]
+ i3 = getfield_gc(p3, descr=adescr)
+ escape(i3)
+ jump(i1, p2)
+ """
+ expected = """
+ [i1, p2]
+ i3 = getfield_gc(p2, descr=adescr)
+ escape(i3)
+ p1 = new(descr=ssize)
+ setfield_gc(p1, i1, descr=adescr)
+ jump(i1, p1)
+ """
# We cannot track virtuals that survive for more than two iterations.
- self.optimize_loop(ops, 'Not, Not, Not', ops)
+ self.optimize_loop(ops, 'Not, Not, Not', expected, preamble)
def test_duplicate_getfield_1(self):
ops = """
From cfbolz at codespeak.net Sat Oct 30 17:28:19 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 17:28:19 +0200 (CEST)
Subject: [pypy-svn] r78536 - pypy/branch/jit-unroll-loops/pypy/jit/metainterp
Message-ID: <20101030152819.1CCBA282BE3@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 17:28:17 2010
New Revision: 78536
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize_nopspec.py
Log:
(arigo, cfbolz) kill findnode here
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize_nopspec.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize_nopspec.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize_nopspec.py Sat Oct 30 17:28:17 2010
@@ -1,8 +1,8 @@
from pypy.rlib.debug import debug_start, debug_stop
from pypy.jit.metainterp.optimizeopt import optimize_loop_1, optimize_bridge_1
-from pypy.jit.metainterp.optimizefindnode import PerfectSpecializationFinder
-from pypy.jit.metainterp.optimizefindnode import BridgeSpecializationFinder
+
+# XXX this file should probably disable unrolling instead
def optimize_loop(metainterp_sd, old_loop_tokens, loop):
debug_start("jit-optimize")
@@ -14,11 +14,6 @@
def _optimize_loop(metainterp_sd, old_loop_tokens, loop):
cpu = metainterp_sd.cpu
metainterp_sd.logger_noopt.log_loop(loop.inputargs, loop.operations)
- # XXX the following lines are probably still needed, to discard invalid
- # loops. bit silly to run a full perfect specialization and throw the
- # result away.
- finder = PerfectSpecializationFinder(cpu)
- finder.find_nodes_loop(loop, False)
if old_loop_tokens:
return old_loop_tokens[0]
optimize_loop_1(metainterp_sd, loop)
@@ -34,9 +29,6 @@
def _optimize_bridge(metainterp_sd, old_loop_tokens, bridge):
cpu = metainterp_sd.cpu
metainterp_sd.logger_noopt.log_loop(bridge.inputargs, bridge.operations)
- # XXX same comment as above applies
- finder = BridgeSpecializationFinder(cpu)
- finder.find_nodes_bridge(bridge)
if old_loop_tokens:
old_loop_token = old_loop_tokens[0]
bridge.operations[-1].setdescr(old_loop_token) # patch jump target
From hakanardo at codespeak.net Sat Oct 30 17:32:19 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 17:32:19 +0200 (CEST)
Subject: [pypy-svn] r78537 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030153219.B805D282BAD@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 17:32:17 2010
New Revision: 78537
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 17:32:17 2010
@@ -1569,7 +1569,7 @@
setfield_gc(p1, i2, descr=valuedescr)
jump(p1, i1, i2, p3)
"""
- expected = """
+ preamble = """
[p1, i1, i2, p3]
#
i3 = getarrayitem_gc_pure(p3, 1, descr=arraydescr)
@@ -1579,9 +1579,20 @@
#
setfield_gc(p1, i2, descr=valuedescr)
setfield_gc(p1, i4, descr=nextdescr)
- jump(p1, i1, i2, p3)
+ jump(p1, i1, i2, p3, i3)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ expected = """
+ [p1, i1, i2, p3, i3]
+ #
+ i4 = getarrayitem_gc(p3, i3, descr=arraydescr)
+ i5 = int_add(i3, i4)
+ setarrayitem_gc(p3, 0, i5, descr=arraydescr)
+ #
+ setfield_gc(p1, i2, descr=valuedescr)
+ setfield_gc(p1, i4, descr=nextdescr)
+ jump(p1, i1, i2, p3, i3)
+ """
+ self.optimize_loop(ops, 'Not, Not, Not, Not', expected, preamble)
def test_duplicate_setfield_5(self):
ops = """
From david at codespeak.net Sat Oct 30 17:53:49 2010
From: david at codespeak.net (david at codespeak.net)
Date: Sat, 30 Oct 2010 17:53:49 +0200 (CEST)
Subject: [pypy-svn] r78538 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101030155349.CA711282BAD@codespeak.net>
Author: david
Date: Sat Oct 30 17:53:48 2010
New Revision: 78538
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/arch.py
pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py
Log:
Implement uint_floordiv and some helper functions to generate operations
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/arch.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/arch.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/arch.py Sat Oct 30 17:53:48 2010
@@ -1,14 +1,18 @@
+from pypy.rlib.rarithmetic import r_uint
from pypy.rpython.lltypesystem import lltype
FUNC_ALIGN=8
WORD=4
arm_int_div_sign = lltype.Ptr(lltype.FuncType([lltype.Signed, lltype.Signed], lltype.Signed))
-arm_int_mod_sign = arm_int_div_sign
-
def arm_int_div(a, b):
return int(a/float(b))
+arm_uint_div_sign = lltype.Ptr(lltype.FuncType([lltype.Unsigned, lltype.Unsigned], lltype.Unsigned))
+def arm_uint_div(a, b):
+ return r_uint(a)/r_uint(b)
+
+arm_int_mod_sign = arm_int_div_sign
def arm_int_mod(a, b):
sign = 1
if a < 0:
@@ -18,3 +22,4 @@
b = -1 * b
res = a % b
return sign * res
+
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Sat Oct 30 17:53:48 2010
@@ -1,7 +1,7 @@
+from pypy.jit.backend.arm import arch
from pypy.jit.backend.arm import conditions as cond
from pypy.jit.backend.arm import registers as reg
-from pypy.jit.backend.arm.arch import (WORD, FUNC_ALIGN, arm_int_div,
- arm_int_div_sign, arm_int_mod_sign, arm_int_mod)
+from pypy.jit.backend.arm.arch import (WORD, FUNC_ALIGN)
from pypy.jit.backend.arm.instruction_builder import define_instructions
from pypy.rlib.rmmap import alloc, PTR
@@ -9,6 +9,20 @@
from pypy.rpython.lltypesystem import lltype, rffi
from pypy.jit.metainterp.history import ConstInt, BoxInt, Box, BasicFailDescr
+def binary_helper_call(name):
+ signature = getattr(arch, 'arm_%s_sign' % name)
+ function = getattr(arch, 'arm_%s' % name)
+ def f(self, cond=cond.AL):
+ """Generates a call to a helper function, takes its
+ arguments in r0 and r1, result is placed in r0"""
+ self.PUSH(range(2, 12), cond=cond)
+ addr = rffi.cast(lltype.Signed, llhelper(signature, function))
+ self.gen_load_int(reg.r2.value, addr, cond=cond)
+ self.gen_load_int(reg.lr.value, self.curraddr()+self.size_of_gen_load_int+WORD, cond=cond)
+ self.MOV_rr(reg.pc.value, reg.r2.value, cond=cond)
+ self.LDM(reg.sp.value, range(2, 12), w=1, cond=cond) # XXX Replace with POP instr. someday
+ return f
+
class AbstractARMv7Builder(object):
def _init(self, data, map_size):
self._data = data
@@ -43,25 +57,9 @@
def BKPT(self, cond=cond.AL):
self.write32(cond << 28 | 0x1200070)
- def DIV(self, cond=cond.AL):
- """Generates a call to a helper function used for division, takes its
- arguments in r0 and r1, result is placed in r0"""
- self.PUSH(range(2, 12), cond=cond)
- div_addr = rffi.cast(lltype.Signed, llhelper(arm_int_div_sign, arm_int_div))
- self.gen_load_int(reg.r2.value, div_addr, cond=cond)
- self.gen_load_int(reg.lr.value, self.curraddr()+self.size_of_gen_load_int+WORD, cond=cond)
- self.MOV_rr(reg.pc.value, reg.r2.value, cond=cond)
- self.LDM(reg.sp.value, range(2, 12), w=1, cond=cond) # XXX Replace with POP instr. someday
-
- def MOD(self, cond=cond.AL):
- """Generate a call to a helper function used for modulo, takes its
- arguments in r0 and r1, result is placed in r0"""
- self.PUSH(range(2, 12), cond=cond)
- mod_addr = rffi.cast(lltype.Signed, llhelper(arm_int_mod_sign, arm_int_mod))
- self.gen_load_int(reg.r2.value, mod_addr, cond=cond)
- self.gen_load_int(reg.lr.value, self.curraddr()+self.size_of_gen_load_int+WORD, cond=cond)
- self.MOV_rr(reg.pc.value, reg.r2.value, cond=cond)
- self.LDM(reg.sp.value, range(2, 12), w=1, cond=cond) # XXX Replace with POP instr. someday
+ DIV = binary_helper_call('int_div')
+ MOD = binary_helper_call('int_mod')
+ UDIV = binary_helper_call('uint_div')
def _encode_reg_list(self, instr, regs):
for reg in regs:
@@ -121,7 +119,7 @@
self.checks = True
self.n_data=0
- _space_for_jump = 10 * WORD
+ _space_for_jump = 9 * WORD
def writechar(self, char):
if self.checks and not self._pos < self._size - self._space_for_jump:
self.checks = False
@@ -133,13 +131,13 @@
def _add_more_mem(self):
new_mem = alloc(self._size)
new_mem_addr = rffi.cast(lltype.Signed, new_mem)
- self.PUSH([reg.r0.value, reg.ip.value])
- self.gen_load_int(reg.r0.value, new_mem_addr)
- self.MOV_rr(reg.pc.value, reg.r0.value)
+ self.PUSH([reg.ip.value, reg.lr.value])
+ self.gen_load_int(reg.lr.value, new_mem_addr)
+ self.MOV_rr(reg.pc.value, reg.lr.value)
self._dump_trace('data%d.asm' % self.n_data)
- self.n_data+=1
+ self.n_data += 1
self._data = new_mem
self._pos = 0
- self.LDM(reg.sp.value, [reg.r0.value, reg.ip.value], w=1) # XXX Replace with POP instr. someday
+ self.LDM(reg.sp.value, [reg.ip.value, reg.lr.value], w=1) # XXX Replace with POP instr. someday
define_instructions(AbstractARMv7Builder)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py Sat Oct 30 17:53:48 2010
@@ -175,6 +175,8 @@
return f
+def define_long_mult_div_instructions(name, table):
+ pass
def imm_operation(rt, rn, imm):
return ((rn & 0xFF) << 16
| (rt & 0xFF) << 12
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Sat Oct 30 17:53:48 2010
@@ -39,6 +39,19 @@
return fcond
return f
+def gen_emit_op_by_helper_call(opname):
+ def f(self, op, regalloc, fcond):
+ arg1 = regalloc.make_sure_var_in_reg(op.getarg(0), selected_reg=r.r0)
+ arg2 = regalloc.make_sure_var_in_reg(op.getarg(1), selected_reg=r.r1)
+ assert arg1 == r.r0
+ assert arg2 == r.r1
+ res = regalloc.try_allocate_reg(op.result)
+ getattr(self.mc, opname)(fcond)
+ self.mc.MOV_rr(res.value, r.r0.value, cond=fcond)
+ regalloc.possibly_free_vars_for_op(op)
+ return fcond
+ return f
+
class IntOpAsslember(object):
_mixin_ = True
@@ -104,28 +117,6 @@
regalloc.possibly_free_var(reg2)
return fcond
- def emit_op_int_floordiv(self, op, regalloc, fcond):
- arg1 = regalloc.make_sure_var_in_reg(op.getarg(0), selected_reg=r.r0)
- arg2 = regalloc.make_sure_var_in_reg(op.getarg(1), selected_reg=r.r1)
- assert arg1 == r.r0
- assert arg2 == r.r1
- res = regalloc.try_allocate_reg(op.result)
- self.mc.DIV(fcond)
- self.mc.MOV_rr(res.value, r.r0.value, cond=fcond)
- regalloc.possibly_free_vars_for_op(op)
- return fcond
-
- def emit_op_int_mod(self, op, regalloc, fcond):
- res = regalloc.force_allocate_reg(op.result)
- arg1 = regalloc.make_sure_var_in_reg(op.getarg(0), selected_reg=r.r0)
- arg2 = regalloc.make_sure_var_in_reg(op.getarg(1), selected_reg=r.r1)
- assert arg1 == r.r0
- assert arg2 == r.r1
- self.mc.MOD(fcond)
- self.mc.MOV_rr(res.value, r.r0.value, cond=fcond)
- regalloc.possibly_free_vars_for_op(op)
- return fcond
-
def _put_in_reg(self, box, regalloc):
if isinstance(box, ConstInt):
t = Box()
@@ -135,6 +126,10 @@
reg = regalloc.try_allocate_reg(box)
return reg
+ emit_op_int_floordiv = gen_emit_op_by_helper_call('DIV')
+ emit_op_int_mod = gen_emit_op_by_helper_call('MOD')
+ emit_op_uint_floordiv = gen_emit_op_by_helper_call('UDIV')
+
emit_op_int_and = gen_emit_op_ri('AND')
emit_op_int_or = gen_emit_op_ri('ORR')
emit_op_int_xor = gen_emit_op_ri('EOR')
From hakanardo at codespeak.net Sat Oct 30 17:58:35 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 17:58:35 +0200 (CEST)
Subject: [pypy-svn] r78539 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030155835.EAF50282C00@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 17:58:34 2010
New Revision: 78539
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 17:58:34 2010
@@ -1635,7 +1635,22 @@
setfield_gc(p1, i2, descr=valuedescr)
jump(p1, i1, i2, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', ops)
+ preamble = """
+ [p1, i1, i2, i3]
+ setfield_gc(p1, i1, descr=valuedescr)
+ guard_true(i3) []
+ i4 = int_neg(i2)
+ setfield_gc(p1, i2, descr=valuedescr)
+ jump(p1, i1, i2, i4)
+ """
+ expected = """
+ [p1, i1, i2, i4]
+ setfield_gc(p1, i1, descr=valuedescr)
+ guard_true(i4) []
+ setfield_gc(p1, i2, descr=valuedescr)
+ jump(p1, i1, i2, 1)
+ """
+ self.optimize_loop(ops, 'Not, Not, Not, Not', expected, preamble)
def test_duplicate_setfield_residual_guard_2(self):
# the difference with the previous test is that the field value is
@@ -1649,14 +1664,20 @@
setfield_gc(p1, NULL, descr=nextdescr)
jump(p1, i2, i4)
"""
- expected = """
+ preamble = """
[p1, i2, i3]
guard_true(i3) [p1]
i4 = int_neg(i2)
setfield_gc(p1, NULL, descr=nextdescr)
jump(p1, i2, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ expected = """
+ [p1, i2, i4]
+ guard_true(i4) [p1]
+ setfield_gc(p1, NULL, descr=nextdescr)
+ jump(p1, i2, 1)
+ """
+ self.optimize_loop(ops, 'Not, Not, Not', expected, preamble)
def test_duplicate_setfield_residual_guard_3(self):
ops = """
@@ -1669,13 +1690,19 @@
setfield_gc(p1, NULL, descr=nextdescr)
jump(p1, i2, i4)
"""
- expected = """
+ preamble = """
[p1, i2, i3]
guard_true(i3) [i2, p1]
i4 = int_neg(i2)
setfield_gc(p1, NULL, descr=nextdescr)
jump(p1, i2, i4)
"""
+ expected = """
+ [p1, i2, i4]
+ guard_true(i4) [i2, p1]
+ setfield_gc(p1, NULL, descr=nextdescr)
+ jump(p1, i2, 1)
+ """
self.optimize_loop(ops, 'Not, Not, Not', expected)
def test_duplicate_setfield_residual_guard_4(self):
@@ -1690,7 +1717,16 @@
setfield_gc(p1, i2, descr=valuedescr)
jump(p1, i1, i2, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', ops)
+ preamble = ops
+ expected = """
+ [p1, i1, i2, i4]
+ setfield_gc(p1, i1, descr=valuedescr)
+ i5 = int_eq(i4, 5)
+ guard_true(i5) []
+ setfield_gc(p1, i2, descr=valuedescr)
+ jump(p1, i1, i2, 5)
+ """
+ self.optimize_loop(ops, 'Not, Not, Not, Not', expected, preamble)
def test_duplicate_setfield_aliasing(self):
# a case where aliasing issues (and not enough cleverness) mean
From arigo at codespeak.net Sat Oct 30 18:03:10 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 30 Oct 2010 18:03:10 +0200 (CEST)
Subject: [pypy-svn] r78540 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030160310.71671282C01@codespeak.net>
Author: arigo
Date: Sat Oct 30 18:03:08 2010
New Revision: 78540
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
Log:
Fix.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py Sat Oct 30 18:03:08 2010
@@ -265,7 +265,7 @@
OptHeap(),
]
optimizer = Optimizer(metainterp_sd, loop, optimizations,
- virtuals=True)
+ not_a_bridge=True)
optimizer.propagate_all_forward()
#
expected = self.parse(optops)
From cfbolz at codespeak.net Sat Oct 30 18:06:52 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 18:06:52 +0200 (CEST)
Subject: [pypy-svn] r78541 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: . test
Message-ID: <20101030160652.6FB3D282C03@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 18:06:50 2010
New Revision: 78541
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(arigo, cfbolz): turn make_fail_descr into a setup_method and sanitize it a bit.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py Sat Oct 30 18:06:50 2010
@@ -299,6 +299,7 @@
storage = self.storage
# make sure that nobody attached resume data to this guard yet
assert storage.rd_numb is None
+ assert storage.rd_snapshot is not None # is that true?
numb, liveboxes_from_env, v = self.memo.number(values,
storage.rd_snapshot)
self.liveboxes_from_env = liveboxes_from_env
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 18:06:50 2010
@@ -66,6 +66,9 @@
def optimize_loop(self, ops, spectext, optops, expected_preamble=None):
loop = self.parse(ops)
+ expected = self.parse(optops)
+ if expected_preamble:
+ expected_preamble = self.parse(expected_preamble)
#
self.loop = loop
loop.preamble = TreeLoop('preamble')
@@ -75,7 +78,6 @@
metainterp_sd.virtualref_info = self.vrefinfo
optimize_loop_1(metainterp_sd, loop)
#
- expected = self.parse(optops)
print
print loop.preamble.inputargs
@@ -87,13 +89,36 @@
self.assert_equal(loop, expected)
if expected_preamble:
- expected = self.parse(expected_preamble)
- self.assert_equal(loop.preamble, expected)
+ self.assert_equal(loop.preamble, expected_preamble)
return loop
class OptimizeOptTest(BaseTestOptimizeOpt):
+ def setup_method(self, meth=None):
+ class FailDescr(compile.ResumeGuardDescr):
+ oparse = None
+ def _oparser_uses_descr_of_guard(self, oparse, fail_args):
+ # typically called twice, before and after optimization
+ if self.oparse is None:
+ self.rd_frame_info_list = resume.FrameInfo(None,
+ "code", 11)
+ self.rd_snapshot = resume.Snapshot(None, fail_args)
+ self.oparse = oparse
+ def _clone_if_mutable(self):
+ assert self is fdescr
+ return fdescr2
+ #
+ fdescr = instantiate(FailDescr)
+ self.namespace['fdescr'] = fdescr
+ fdescr2 = instantiate(FailDescr)
+ self.namespace['fdescr2'] = fdescr2
+
+ def teardown_method(self, meth):
+ self.namespace.pop('fdescr', None)
+ self.namespace.pop('fdescr2', None)
+
+
def test_simple(self):
ops = """
[]
@@ -2079,7 +2104,6 @@
self.optimize_loop(ops, "Not, Not, Not, Not, Not", expected)
def test_merge_guard_nonnull_guard_class(self):
- self.make_fail_descr()
ops = """
[p1, i0, i1, i2, p2]
guard_nonnull(p1, descr=fdescr) [i0]
@@ -2087,17 +2111,22 @@
guard_class(p1, ConstClass(node_vtable)) [i1]
jump(p2, i0, i1, i3, p2)
"""
- expected = """
+ preamble = """
[p1, i0, i1, i2, p2]
guard_nonnull_class(p1, ConstClass(node_vtable), descr=fdescr) [i0]
i3 = int_add(i1, i2)
- jump(p2, i0, i1, i3, p2)
+ jump(p2, i0, i1, i3)
"""
- self.optimize_loop(ops, "Not, Not, Not, Not, Not", expected)
+ expected = """
+ [p2, i0, i1, i2]
+ guard_nonnull_class(p2, ConstClass(node_vtable), descr=fdescr2) [i0]
+ i3 = int_add(i1, i2)
+ jump(p2, i0, i1, i3)
+ """
+ self.optimize_loop(ops, "Not, Not, Not, Not, Not", expected, preamble)
self.check_expanded_fail_descr("i0", rop.GUARD_NONNULL_CLASS)
def test_merge_guard_nonnull_guard_value(self):
- self.make_fail_descr()
ops = """
[p1, i0, i1, i2, p2]
guard_nonnull(p1, descr=fdescr) [i0]
@@ -2115,7 +2144,6 @@
self.check_expanded_fail_descr("i0", rop.GUARD_VALUE)
def test_merge_guard_nonnull_guard_class_guard_value(self):
- self.make_fail_descr()
ops = """
[p1, i0, i1, i2, p2]
guard_nonnull(p1, descr=fdescr) [i0]
@@ -2333,23 +2361,6 @@
# ----------
- def make_fail_descr(self):
- class FailDescr(compile.ResumeGuardDescr):
- oparse = None
- def _oparser_uses_descr_of_guard(self, oparse, fail_args):
- # typically called twice, before and after optimization
- if self.oparse is None:
- fdescr.rd_frame_info_list = resume.FrameInfo(None,
- "code", 11)
- fdescr.rd_snapshot = resume.Snapshot(None, fail_args)
- self.oparse = oparse
- #
- fdescr = instantiate(FailDescr)
- self.namespace['fdescr'] = fdescr
-
- def teardown_method(self, meth):
- self.namespace.pop('fdescr', None)
-
def _verify_fail_args(self, boxes, oparse, text):
import re
r = re.compile(r"\bwhere\s+(\w+)\s+is a\s+(\w+)")
@@ -2458,7 +2469,6 @@
self._verify_fail_args(boxes, fdescr.oparse, expectedtext)
def test_expand_fail_1(self):
- self.make_fail_descr()
ops = """
[i1, i3]
# first rename i3 into i4
@@ -2479,7 +2489,6 @@
self.check_expanded_fail_descr('15, i3', rop.GUARD_TRUE)
def test_expand_fail_2(self):
- self.make_fail_descr()
ops = """
[i1, i2]
p1 = new_with_vtable(ConstClass(node_vtable))
@@ -2499,7 +2508,6 @@
''', rop.GUARD_TRUE)
def test_expand_fail_3(self):
- self.make_fail_descr()
ops = """
[i1, i2, i3, p3]
p1 = new_with_vtable(ConstClass(node_vtable))
@@ -2525,7 +2533,7 @@
def test_expand_fail_4(self):
for arg in ['p1', 'i2,p1', 'p1,p2', 'p2,p1',
'i2,p1,p2', 'i2,p2,p1']:
- self.make_fail_descr()
+ self.setup_method() # humpf
ops = """
[i1, i2, i3]
p1 = new_with_vtable(ConstClass(node_vtable))
@@ -2550,7 +2558,6 @@
rop.GUARD_TRUE)
def test_expand_fail_5(self):
- self.make_fail_descr()
ops = """
[i1, i2, i3, i4]
p1 = new_with_vtable(ConstClass(node_vtable))
@@ -2574,7 +2581,6 @@
''', rop.GUARD_TRUE)
def test_expand_fail_6(self):
- self.make_fail_descr()
ops = """
[p0, i0, i1]
guard_true(i0, descr=fdescr) [p0]
@@ -2594,7 +2600,6 @@
''', rop.GUARD_TRUE)
def test_expand_fail_varray(self):
- self.make_fail_descr()
ops = """
[i1]
p1 = new_array(3, descr=arraydescr)
@@ -2615,7 +2620,6 @@
''', rop.GUARD_TRUE)
def test_expand_fail_vstruct(self):
- self.make_fail_descr()
ops = """
[i1, p1]
p2 = new(descr=ssize)
@@ -2637,7 +2641,6 @@
''', rop.GUARD_TRUE)
def test_expand_fail_v_all_1(self):
- self.make_fail_descr()
ops = """
[i1, p1a, i2]
p6s = getarrayitem_gc(p1a, 0, descr=arraydescr2)
@@ -2678,7 +2681,6 @@
''', rop.GUARD_TRUE)
def test_expand_fail_lazy_setfield_1(self):
- self.make_fail_descr()
ops = """
[p1, i2, i3]
p2 = new_with_vtable(ConstClass(node_vtable))
@@ -2704,7 +2706,6 @@
''', rop.GUARD_TRUE)
def test_expand_fail_lazy_setfield_2(self):
- self.make_fail_descr()
ops = """
[i2, i3]
p2 = new_with_vtable(ConstClass(node_vtable))
@@ -3022,7 +3023,6 @@
self.optimize_loop(ops, 'Not, Not', expected)
def test_vref_virtual_2(self):
- self.make_fail_descr()
ops = """
[p0, i1]
#
@@ -3069,7 +3069,6 @@
''', rop.GUARD_NOT_FORCED)
def test_vref_virtual_and_lazy_setfield(self):
- self.make_fail_descr()
ops = """
[p0, i1]
#
@@ -3108,7 +3107,6 @@
''', rop.GUARD_NO_EXCEPTION)
def test_vref_virtual_after_finish(self):
- self.make_fail_descr()
ops = """
[i1]
p1 = new_with_vtable(ConstClass(node_vtable))
@@ -3136,7 +3134,6 @@
self.optimize_loop(ops, 'Not', expected)
def test_vref_nonvirtual_and_lazy_setfield(self):
- self.make_fail_descr()
ops = """
[i1, p1]
p2 = virtual_ref(p1, 23)
From hakanardo at codespeak.net Sat Oct 30 18:11:57 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 18:11:57 +0200 (CEST)
Subject: [pypy-svn] r78542 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030161157.DAC59282C06@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 18:11:56 2010
New Revision: 78542
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 18:11:56 2010
@@ -1926,6 +1926,9 @@
[p1, i2, i3, p4]
guard_true(i3) [p1, p4]
i4 = int_neg(i2)
+ p2 = new_with_vtable(ConstClass(node_vtable))
+ setfield_gc(p2, p4, descr=nextdescr)
+ setfield_gc(p1, p2, descr=nextdescr)
jump(p1, i2, i4, p4)
"""
expected = """
@@ -2004,6 +2007,19 @@
setfield_gc(p1a, p3a, descr=otherdescr)
jump(p1a)
"""
+ preamble = """
+ [p1]
+ guard_nonnull(p1) []
+ guard_class(p1, ConstClass(node_vtable2)) []
+ p2 = getfield_gc(p1, descr=nextdescr)
+ guard_class(p2, ConstClass(node_vtable)) []
+ p3 = getfield_gc(p1, descr=otherdescr)
+ guard_class(p3, ConstClass(node_vtable)) []
+ p3a = new_with_vtable(ConstClass(node_vtable))
+ escape(p3a)
+ setfield_gc(p3, p2, descr=otherdescr)
+ jump(p2a, p3a)
+ """
expected = """
[p2, p3]
guard_class(p2, ConstClass(node_vtable)) []
From cfbolz at codespeak.net Sat Oct 30 18:21:49 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 18:21:49 +0200 (CEST)
Subject: [pypy-svn] r78543 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030162149.6AF19282BFE@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 18:21:45 2010
New Revision: 78543
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(arigo, cfbolz): remove spectext everywhere in test_optimizeopt
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 18:21:45 2010
@@ -64,7 +64,7 @@
assert equaloplists(optimized.operations,
expected.operations, False, remap)
- def optimize_loop(self, ops, spectext, optops, expected_preamble=None):
+ def optimize_loop(self, ops, optops, expected_preamble=None):
loop = self.parse(ops)
expected = self.parse(optops)
if expected_preamble:
@@ -128,7 +128,7 @@
escape(f)
jump()
"""
- self.optimize_loop(ops, '', ops)
+ self.optimize_loop(ops, ops)
def test_constant_propagate(self):
ops = """
@@ -145,7 +145,7 @@
[]
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_constant_propagate_ovf(self):
ops = """
@@ -163,7 +163,7 @@
[]
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_constfold_all(self):
from pypy.jit.backend.llgraph.llimpl import TYPES # xxx fish
@@ -197,7 +197,7 @@
escape(%d)
jump()
""" % expected_value
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
# ----------
@@ -217,7 +217,7 @@
[p0]
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected, expected_preamble=preamble)
+ self.optimize_loop(ops, expected, expected_preamble=preamble)
def test_remove_guard_class_2(self):
ops = """
@@ -233,7 +233,7 @@
escape(p0)
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_remove_guard_class_constant(self):
ops = """
@@ -246,7 +246,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_constant_boolrewrite_lt(self):
ops = """
@@ -267,7 +267,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected, expected_preamble=preamble)
+ self.optimize_loop(ops, expected, expected_preamble=preamble)
def test_constant_boolrewrite_gt(self):
ops = """
@@ -288,7 +288,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected, expected_preamble=preamble)
+ self.optimize_loop(ops, expected, expected_preamble=preamble)
def test_constant_boolrewrite_reflex(self):
ops = """
@@ -309,7 +309,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected, expected_preamble=preamble)
+ self.optimize_loop(ops, expected, expected_preamble=preamble)
def test_constant_boolrewrite_reflex_invers(self):
ops = """
@@ -330,7 +330,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected, expected_preamble=preamble)
+ self.optimize_loop(ops, expected, expected_preamble=preamble)
def test_remove_consecutive_guard_value_constfold(self):
ops = """
@@ -350,7 +350,7 @@
escape(3)
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_remove_guard_value_if_constant(self):
ops = """
@@ -362,7 +362,7 @@
[]
jump()
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_ooisnull_oononnull_1(self):
ops = """
@@ -380,7 +380,7 @@
[p0]
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_int_is_true_1(self):
ops = """
@@ -401,7 +401,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_int_is_true_is_zero(self):
py.test.skip("XXX implement me")
@@ -419,7 +419,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_ooisnull_oononnull_2(self):
ops = """
@@ -437,7 +437,7 @@
[p0]
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_ooisnull_on_null_ptr_1(self):
ops = """
@@ -453,7 +453,7 @@
guard_isnull(p0) []
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_ooisnull_oononnull_via_virtual(self):
ops = """
@@ -474,7 +474,7 @@
[p0]
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_oois_1(self):
ops = """
@@ -499,7 +499,7 @@
[p0]
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_nonnull_1(self):
ops = """
@@ -521,7 +521,7 @@
setfield_gc(p0, 5, descr=valuedescr)
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_const_guard_value(self):
ops = """
@@ -534,7 +534,7 @@
[]
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_constptr_guard_value(self):
ops = """
@@ -543,7 +543,7 @@
guard_value(p1, ConstPtr(myptr)) []
jump()
"""
- self.optimize_loop(ops, '', ops)
+ self.optimize_loop(ops, ops)
def test_guard_value_to_guard_true(self):
ops = """
@@ -562,7 +562,7 @@
[i]
jump(i)
"""
- self.optimize_loop(ops, 'Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_guard_value_to_guard_false(self):
ops = """
@@ -581,7 +581,7 @@
[i]
jump(i)
"""
- self.optimize_loop(ops, 'Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_guard_value_on_nonbool(self):
ops = """
@@ -600,7 +600,7 @@
[]
jump()
"""
- self.optimize_loop(ops, 'Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_int_is_true_of_bool(self):
ops = """
@@ -621,7 +621,7 @@
[i0, i1]
jump(i0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
@@ -650,7 +650,7 @@
jump(i1, p3)
"""
# We cannot track virtuals that survive for more than two iterations.
- self.optimize_loop(ops, 'Not, Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_p123_nested(self):
ops = """
@@ -683,7 +683,7 @@
setfield_gc(p4, p1sub, descr=nextdescr)
jump(i1, p4)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_p123_anti_nested(self):
ops = """
@@ -719,7 +719,7 @@
setfield_gc(p1, p3sub, descr=nextdescr)
jump(i1, p1, p3sub)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
# ----------
@@ -743,7 +743,7 @@
i3 = call(i2, descr=nonwritedescr)
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected, expected)
+ self.optimize_loop(ops, expected, expected)
# ----------
@@ -772,7 +772,7 @@
[i1]
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
# ----------
@@ -797,7 +797,7 @@
i1 = int_add(i2, i)
jump(i, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_virtual_float(self):
ops = """
@@ -819,7 +819,7 @@
f1 = float_add(f2, f)
jump(f, f1)
"""
- self.optimize_loop(ops, 'Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_virtual_oois(self):
ops = """
@@ -867,7 +867,7 @@
guard_true(i11) []
jump(p0, p1, p2)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected, expected2)
+ self.optimize_loop(ops, expected, expected2)
def test_virtual_default_field(self):
ops = """
@@ -888,7 +888,7 @@
[]
jump()
"""
- self.optimize_loop(ops, 'Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_virtual_3(self):
ops = """
@@ -904,7 +904,7 @@
i1 = int_add(i, 1)
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_virtual_4(self):
ops = """
@@ -931,7 +931,7 @@
i3 = int_add(i0, i1)
jump(i3, i2)
"""
- self.optimize_loop(ops, 'Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_virtual_5(self):
ops = """
@@ -961,7 +961,7 @@
i3 = int_add(i0, i1)
jump(i3, i2, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_virtual_constant_isnull(self):
ops = """
@@ -980,7 +980,7 @@
[]
jump()
"""
- self.optimize_loop(ops, 'Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_virtual_constant_isnonnull(self):
@@ -1000,7 +1000,7 @@
[]
jump()
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_nonvirtual_1(self):
ops = """
@@ -1022,7 +1022,7 @@
escape(p1)
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_nonvirtual_2(self):
ops = """
@@ -1049,7 +1049,7 @@
i2 = int_add(i1, i)
jump(i, i2)
"""
- self.optimize_loop(ops, 'Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_nonvirtual_later(self):
ops = """
@@ -1071,7 +1071,7 @@
i3 = int_add(i, i2)
jump(i3)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_nonvirtual_dont_write_null_fields_on_force(self):
ops = """
@@ -1091,7 +1091,7 @@
i2 = getfield_gc(p1, descr=valuedescr)
jump(i2)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_getfield_gc_pure_1(self):
ops = """
@@ -1105,7 +1105,7 @@
[i]
jump(i)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_getfield_gc_pure_2(self):
ops = """
@@ -1118,7 +1118,7 @@
jump()
"""
self.node.value = 5
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_getfield_gc_nonpure_2(self):
ops = """
@@ -1131,7 +1131,7 @@
[i]
jump(i)
"""
- self.optimize_loop(ops, 'Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_varray_1(self):
ops = """
@@ -1148,7 +1148,7 @@
[i1]
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_varray_alloc_and_set(self):
ops = """
@@ -1166,7 +1166,7 @@
[]
jump()
"""
- self.optimize_loop(ops, 'Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_varray_float(self):
ops = """
@@ -1183,7 +1183,7 @@
[f1]
jump(f1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_array_non_optimized(self):
ops = """
@@ -1199,7 +1199,7 @@
p1 = new_array(i1, descr=arraydescr)
jump(i1, p1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_nonvirtual_array_dont_write_null_fields_on_force(self):
ops = """
@@ -1217,7 +1217,7 @@
escape(p1)
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_varray_2(self):
ops = """
@@ -1245,7 +1245,7 @@
guard_value(i3, 15) []
jump(5)
"""
- self.optimize_loop(ops, 'Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_p123_array(self):
ops = """
@@ -1271,7 +1271,7 @@
jump(i1, p1)
"""
# We cannot track virtuals that survive for more than two iterations.
- self.optimize_loop(ops, 'Not, Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_varray_forced_1(self):
ops = """
@@ -1293,7 +1293,7 @@
escape(i2)
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_vstruct_1(self):
ops = """
@@ -1315,7 +1315,7 @@
escape(i1)
jump(i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_p123_vstruct(self):
ops = """
@@ -1341,7 +1341,7 @@
jump(i1, p1)
"""
# We cannot track virtuals that survive for more than two iterations.
- self.optimize_loop(ops, 'Not, Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_duplicate_getfield_1(self):
ops = """
@@ -1366,7 +1366,7 @@
escape(i2)
jump(p1, p2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_getfield_after_setfield(self):
ops = """
@@ -1382,7 +1382,7 @@
escape(i1)
jump(p1, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_setfield_of_different_type_does_not_clear(self):
ops = """
@@ -1400,7 +1400,7 @@
escape(i1)
jump(p1, p2, i1)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_setfield_of_same_type_clears(self):
ops = """
@@ -1411,7 +1411,7 @@
escape(i3)
jump(p1, p2, i1, i3)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_duplicate_getfield_mergepoint_has_no_side_effects(self):
ops = """
@@ -1431,7 +1431,7 @@
escape(i1)
jump(p1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getfield_ovf_op_does_not_clear(self):
ops = """
@@ -1453,7 +1453,7 @@
escape(i1)
jump(p1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getfield_setarrayitem_does_not_clear(self):
ops = """
@@ -1473,7 +1473,7 @@
escape(i1)
jump(p1, p2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getfield_constant(self):
ops = """
@@ -1491,7 +1491,7 @@
escape(i1)
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getfield_guard_value_const(self):
ops = """
@@ -1510,7 +1510,7 @@
escape(i1)
jump()
"""
- self.optimize_loop(ops, 'Constant(myptr)', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getfield_sideeffects_1(self):
ops = """
@@ -1522,7 +1522,7 @@
escape(i2)
jump(p1)
"""
- self.optimize_loop(ops, 'Not', ops)
+ self.optimize_loop(ops, ops)
def test_duplicate_getfield_sideeffects_2(self):
ops = """
@@ -1533,7 +1533,7 @@
escape(i2)
jump(p1, i1)
"""
- self.optimize_loop(ops, 'Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_duplicate_setfield_1(self):
ops = """
@@ -1547,7 +1547,7 @@
setfield_gc(p1, i2, descr=valuedescr)
jump(p1, i1, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_setfield_2(self):
ops = """
@@ -1564,7 +1564,7 @@
escape(i1)
jump(p1, i1, i3)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_setfield_3(self):
ops = """
@@ -1577,7 +1577,7 @@
"""
# potential aliasing of p1 and p2 means that we cannot kill the
# the setfield_gc
- self.optimize_loop(ops, 'Not, Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_duplicate_setfield_4(self):
ops = """
@@ -1617,7 +1617,7 @@
setfield_gc(p1, i4, descr=nextdescr)
jump(p1, i1, i2, p3, i3)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_duplicate_setfield_5(self):
ops = """
@@ -1639,7 +1639,7 @@
escape(i1)
jump(p0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_setfield_sideeffects_1(self):
ops = """
@@ -1649,7 +1649,7 @@
setfield_gc(p1, i2, descr=valuedescr)
jump(p1, i1, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_duplicate_setfield_residual_guard_1(self):
ops = """
@@ -1675,7 +1675,7 @@
setfield_gc(p1, i2, descr=valuedescr)
jump(p1, i1, i2, 1)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_duplicate_setfield_residual_guard_2(self):
# the difference with the previous test is that the field value is
@@ -1702,7 +1702,7 @@
setfield_gc(p1, NULL, descr=nextdescr)
jump(p1, i2, 1)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_duplicate_setfield_residual_guard_3(self):
ops = """
@@ -1728,7 +1728,7 @@
setfield_gc(p1, NULL, descr=nextdescr)
jump(p1, i2, 1)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_setfield_residual_guard_4(self):
# test that the setfield_gc does not end up between int_eq and
@@ -1751,7 +1751,7 @@
setfield_gc(p1, i2, descr=valuedescr)
jump(p1, i1, i2, 5)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_duplicate_setfield_aliasing(self):
# a case where aliasing issues (and not enough cleverness) mean
@@ -1763,7 +1763,7 @@
setfield_gc(p1, i3, descr=valuedescr)
jump(p1, p2, i1, i2, i3)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_duplicate_setfield_guard_value_const(self):
ops = """
@@ -1778,7 +1778,7 @@
setfield_gc(ConstPtr(myptr), i2, descr=valuedescr)
jump(i1, i2)
"""
- self.optimize_loop(ops, 'Constant(myptr), Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getarrayitem_1(self):
ops = """
@@ -1803,7 +1803,7 @@
escape(p3)
jump(p1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getarrayitem_after_setarrayitem_1(self):
ops = """
@@ -1819,7 +1819,7 @@
escape(p2)
jump(p1, p2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getarrayitem_after_setarrayitem_2(self):
ops = """
@@ -1841,7 +1841,7 @@
escape(p3)
jump(p1, p2, p3, i1)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getarrayitem_after_setarrayitem_3(self):
ops = """
@@ -1868,7 +1868,7 @@
escape(p4)
jump(p1, p2, p3, p4, i1)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_getarrayitem_pure_does_not_invalidate(self):
ops = """
@@ -1889,7 +1889,7 @@
escape(p3)
jump(p1, p2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getarrayitem_after_setarrayitem_two_arrays(self):
ops = """
@@ -1910,7 +1910,7 @@
escape(p4)
jump(p1, p2, p3, p4, i1)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_setfield_virtual(self):
ops = """
@@ -1939,7 +1939,7 @@
setfield_gc(p1, p2, descr=nextdescr)
jump(p1, i2, 1, p4)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
def test_bug_1(self):
ops = """
@@ -1961,7 +1961,8 @@
p3 = escape()
jump(i0, p3)
"""
- self.optimize_loop(ops, 'Not, Virtual(node_vtable, nextdescr=Not)',
+ xxx # was Not, Virtual(node_vtable, nextdescr=Not)
+ self.optimize_loop(ops,
expected)
def test_bug_2(self):
@@ -1984,7 +1985,8 @@
p3 = escape()
jump(i0, p3)
"""
- self.optimize_loop(ops, 'Not, VArray(arraydescr2, Not)',
+ xxx # was Not, VArray(arraydescr2, Not)
+ self.optimize_loop(ops,
expected)
def test_bug_3(self):
@@ -2030,7 +2032,7 @@
p2a = new_with_vtable(ConstClass(node_vtable))
jump(p2a, p3a)
"""
- self.optimize_loop(ops, 'Virtual(node_vtable2, nextdescr=Not, otherdescr=Not)', expected)
+ self.optimize_loop(ops, expected) # XXX Virtual(node_vtable2, nextdescr=Not, otherdescr=Not)
def test_bug_3bis(self):
ops = """
@@ -2062,7 +2064,7 @@
escape(p3a)
jump(p2a, p3a)
"""
- self.optimize_loop(ops, 'Virtual(node_vtable2, nextdescr=Not, otherdescr=Not)', expected)
+ self.optimize_loop(ops, expected) # XXX was Virtual(node_vtable2, nextdescr=Not, otherdescr=Not)
def test_invalid_loop_1(self):
ops = """
@@ -2117,7 +2119,7 @@
i3 = int_add(i1, i2)
jump(p2, i0, i1, i3, p2)
"""
- self.optimize_loop(ops, "Not, Not, Not, Not, Not", expected)
+ self.optimize_loop(ops, expected)
def test_merge_guard_nonnull_guard_class(self):
ops = """
@@ -2139,7 +2141,7 @@
i3 = int_add(i1, i2)
jump(p2, i0, i1, i3)
"""
- self.optimize_loop(ops, "Not, Not, Not, Not, Not", expected, preamble)
+ self.optimize_loop(ops, expected, preamble)
self.check_expanded_fail_descr("i0", rop.GUARD_NONNULL_CLASS)
def test_merge_guard_nonnull_guard_value(self):
@@ -2156,7 +2158,7 @@
i3 = int_add(i1, i2)
jump(p2, i0, i1, i3, p2)
"""
- self.optimize_loop(ops, "Not, Not, Not, Not, Not", expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr("i0", rop.GUARD_VALUE)
def test_merge_guard_nonnull_guard_class_guard_value(self):
@@ -2176,7 +2178,7 @@
i4 = int_sub(i3, 1)
jump(p2, i0, i1, i4, p2)
"""
- self.optimize_loop(ops, "Not, Not, Not, Not, Not", expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr("i0", rop.GUARD_VALUE)
def test_guard_class_oois(self):
@@ -2192,7 +2194,7 @@
guard_class(p1, ConstClass(node_vtable2)) []
jump(p1)
"""
- self.optimize_loop(ops, "Not", expected)
+ self.optimize_loop(ops, expected)
def test_oois_of_itself(self):
ops = """
@@ -2210,7 +2212,7 @@
p1 = getfield_gc(p0, descr=nextdescr)
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_remove_duplicate_pure_op(self):
ops = """
@@ -2240,7 +2242,7 @@
guard_true(i1) []
jump(p1, p2)
"""
- self.optimize_loop(ops, "Not, Not", expected)
+ self.optimize_loop(ops, expected)
def test_remove_duplicate_pure_op_with_descr(self):
ops = """
@@ -2260,7 +2262,7 @@
guard_true(i1) []
jump(p1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_remove_duplicate_pure_op_ovf(self):
ops = """
@@ -2287,7 +2289,7 @@
escape(i3)
jump(i1)
"""
- self.optimize_loop(ops, "Not", expected)
+ self.optimize_loop(ops, expected)
def test_int_and_or_with_zero(self):
ops = """
@@ -2302,7 +2304,7 @@
[i0, i1]
jump(i1, i0)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_fold_partially_constant_ops(self):
ops = """
@@ -2314,7 +2316,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
ops = """
[i0]
@@ -2325,7 +2327,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
ops = """
[i0]
@@ -2336,7 +2338,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_fold_partially_constant_ops_ovf(self):
ops = """
@@ -2349,7 +2351,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
ops = """
[i0]
@@ -2361,7 +2363,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
ops = """
[i0]
@@ -2373,7 +2375,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
# ----------
@@ -2501,7 +2503,7 @@
guard_true(i1, descr=fdescr) [i3]
jump(1, i3)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('15, i3', rop.GUARD_TRUE)
def test_expand_fail_2(self):
@@ -2518,7 +2520,7 @@
guard_true(i1, descr=fdescr) [i2]
jump(1, i2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('''ptr
where ptr is a node_vtable, valuedescr=i2
''', rop.GUARD_TRUE)
@@ -2540,7 +2542,7 @@
guard_true(i1, descr=fdescr) [i3, i2, p3]
jump(i2, 1, i3, p3)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('''i3, p1
where p1 is a node_vtable, valuedescr=1, nextdescr=p2
where p2 is a node_vtable, valuedescr=i2, nextdescr=p3
@@ -2567,7 +2569,7 @@
guard_true(i1, descr=fdescr) [i3, i2]
jump(1, i2, i3)
"""
- self.optimize_loop(ops % arg, 'Not, Not, Not', expected)
+ self.optimize_loop(ops % arg, expected)
self.check_expanded_fail_descr('''i3, i3, %s
where p1 is a node_vtable, valuedescr=i2, nextdescr=p2
where p2 is a node_vtable, valuedescr=i2''' % arg,
@@ -2590,7 +2592,7 @@
guard_true(i1, descr=fdescr) [i3, i4, i2]
jump(i2, 1, i3, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('''i3, i4, p1, p2
where p1 is a node_vtable, valuedescr=i4, nextdescr=p2
where p2 is a node_vtable, valuedescr=i2, nextdescr=p1
@@ -2609,6 +2611,7 @@
guard_true(i0, descr=fdescr) [i1b]
jump(i1, i1, i1)
"""
+ xxx
self.optimize_loop(ops, '''Virtual(node_vtable, valuedescr=Not),
Not, Not''', expected)
self.check_expanded_fail_descr('''p0
@@ -2630,7 +2633,7 @@
guard_true(i1, descr=fdescr) [i1]
jump(1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('''p1
where p1 is a varray arraydescr: 25, i1
''', rop.GUARD_TRUE)
@@ -2651,7 +2654,7 @@
guard_true(i1, descr=fdescr) [i1, p1]
jump(1, p1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('''p2
where p2 is a vstruct ssize, adescr=i1, bdescr=p1
''', rop.GUARD_TRUE)
@@ -2680,6 +2683,7 @@
guard_true(i1, descr=fdescr) [ia, iv, i2]
jump(1, 1, i2, NULL, i2)
"""
+ xxx
self.optimize_loop(ops, '''
Not,
VArray(arraydescr2,
@@ -2714,7 +2718,7 @@
setfield_gc(p1, NULL, descr=nextdescr)
jump(p1, i2, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.loop.inputargs[0].value = self.nodebox.value
self.check_expanded_fail_descr('''
p1.nextdescr = p2
@@ -2739,7 +2743,7 @@
setfield_gc(ConstPtr(myptr), NULL, descr=nextdescr)
jump(i2, i4)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('''
ConstPtr(myptr).nextdescr = p2
where p2 is a node_vtable, valuedescr=i2
@@ -2766,7 +2770,7 @@
escape(i1)
jump(p1, p2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_residual_call_invalidate_some_caches(self):
ops = """
@@ -2794,7 +2798,7 @@
escape(i2)
jump(p1, p2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_residual_call_invalidate_arrays(self):
ops = """
@@ -2821,7 +2825,7 @@
escape(p4)
jump(p1, p2, i1)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_residual_call_invalidate_some_arrays(self):
ops = """
@@ -2856,7 +2860,7 @@
escape(i4)
jump(p1, p2, i1)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_residual_call_invalidates_some_read_caches_1(self):
ops = """
@@ -2876,7 +2880,7 @@
setfield_gc(p2, i3, descr=adescr)
jump(p1, i1, p2, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_residual_call_invalidates_some_read_caches_2(self):
ops = """
@@ -2896,7 +2900,7 @@
setfield_gc(p2, i3, descr=adescr)
jump(p1, i1, p2, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_residual_call_invalidates_some_read_caches_3(self):
ops = """
@@ -2908,7 +2912,7 @@
setfield_gc(p2, i3, descr=adescr)
jump(p1, i1, p2, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_call_assembler_invalidates_caches(self):
ops = '''
@@ -2918,7 +2922,7 @@
setfield_gc(p1, i3, descr=valuedescr)
jump(p1, i3)
'''
- self.optimize_loop(ops, 'Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_call_pure_invalidates_caches(self):
# CALL_PURE should still force the setfield_gc() to occur before it
@@ -2936,7 +2940,7 @@
setfield_gc(p1, i3, descr=valuedescr)
jump(p1, i3)
'''
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_call_pure_constant_folding(self):
# CALL_PURE is not marked as is_always_pure(), because it is wrong
@@ -2959,7 +2963,7 @@
i4 = call(123456, 4, i0, 6, descr=plaincalldescr)
jump(i0, 42, i4)
'''
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_vref_nonvirtual_nonescape(self):
ops = """
@@ -2973,7 +2977,7 @@
i0 = force_token()
jump(p1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_vref_nonvirtual_escape(self):
ops = """
@@ -2996,7 +3000,7 @@
"""
# XXX we should optimize a bit more the case of a nonvirtual.
# in theory it is enough to just do 'p2 = p1'.
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_vref_virtual_1(self):
ops = """
@@ -3036,7 +3040,7 @@
setfield_gc(p2, -3, descr=virtualtokendescr)
jump(p0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_vref_virtual_2(self):
ops = """
@@ -3078,7 +3082,7 @@
"""
# the point of this test is that 'i1' should show up in the fail_args
# of 'guard_not_forced', because it was stored in the virtual 'p1b'.
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('''p2, p1
where p1 is a node_vtable, nextdescr=p1b
where p1b is a node_vtable, valuedescr=i1
@@ -3109,7 +3113,7 @@
setfield_gc(p0, NULL, descr=refdescr)
jump(p0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
# the fail_args contain [i3, i1, p0]:
# - i3 is from the virtual expansion of p2
# - i1 is from the virtual expansion of p1
@@ -3147,7 +3151,7 @@
guard_not_forced() []
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_vref_nonvirtual_and_lazy_setfield(self):
ops = """
@@ -3172,7 +3176,7 @@
guard_not_forced() [i1]
jump(i1, p1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_arraycopy_1(self):
ops = '''
@@ -3189,7 +3193,7 @@
[]
jump()
'''
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_arraycopy_2(self):
ops = '''
@@ -3206,7 +3210,7 @@
[]
jump()
'''
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_arraycopy_not_virtual(self):
ops = '''
@@ -3226,7 +3230,7 @@
escape(p2)
jump()
'''
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_arraycopy_no_elem(self):
""" this was actually observed in the wild
@@ -3241,7 +3245,7 @@
[p1]
jump(p1)
'''
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt(self):
ops = """
@@ -3258,7 +3262,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_noguard(self):
ops = """
@@ -3273,7 +3277,7 @@
i2 = int_lt(i0, 5)
jump(i2)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_noopt(self):
ops = """
@@ -3292,7 +3296,7 @@
guard_true(i2) []
jump(4)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_rev(self):
ops = """
@@ -3309,7 +3313,7 @@
guard_false(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_tripple(self):
ops = """
@@ -3328,7 +3332,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_add(self):
ops = """
@@ -3347,7 +3351,7 @@
i2 = int_add(i0, 10)
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_add_before(self):
ops = """
@@ -3366,7 +3370,7 @@
guard_true(i3) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_add_ovf(self):
ops = """
@@ -3386,7 +3390,7 @@
i2 = int_add(i0, 10)
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_add_ovf_before(self):
ops = """
@@ -3407,7 +3411,7 @@
guard_true(i3) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_sub(self):
ops = """
@@ -3426,7 +3430,7 @@
i2 = int_sub(i0, 10)
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_sub_before(self):
ops = """
@@ -3445,7 +3449,7 @@
guard_true(i3) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_ltle(self):
ops = """
@@ -3462,7 +3466,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lelt(self):
ops = """
@@ -3479,7 +3483,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_gt(self):
ops = """
@@ -3496,7 +3500,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_gtge(self):
ops = """
@@ -3513,7 +3517,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_gegt(self):
ops = """
@@ -3530,7 +3534,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_ovf(self):
ops = """
@@ -3552,7 +3556,7 @@
i3 = int_add(i0, 1)
jump(i3)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_arraylen(self):
ops = """
@@ -3572,7 +3576,7 @@
setarrayitem_gc(p0, 0, p1)
jump(i0, p0)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_strlen(self):
ops = """
@@ -3588,7 +3592,7 @@
i0 = strlen(p0)
jump(p0)
"""
- self.optimize_strunicode_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_addsub_const(self):
ops = """
@@ -3605,7 +3609,7 @@
i4 = int_mul(i0, i1)
jump(i4)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_addsub_int(self):
ops = """
@@ -3622,7 +3626,7 @@
i4 = int_add(i0, i1)
jump(i4, i10)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_addsub_int2(self):
ops = """
@@ -3639,7 +3643,7 @@
i4 = int_add(i0, i1)
jump(i4, i10)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_framestackdepth_overhead(self):
ops = """
@@ -3665,7 +3669,7 @@
setfield_gc(p0, i1, descr=valuedescr)
jump(p0, i22, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_setgetfield_raw(self):
ops = """
@@ -3684,7 +3688,7 @@
setfield_raw(p7, i33, descr=nextdescr)
jump(p4, p7, i30, i33)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_pure(self):
ops = """
@@ -3701,7 +3705,7 @@
"""
self.node.value = 5
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_addsub_ovf(self):
ops = """
@@ -3719,7 +3723,7 @@
i2 = int_sub(i1, 5)
jump(i2)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_subadd_ovf(self):
ops = """
@@ -3737,7 +3741,7 @@
i2 = int_add(i1, 5)
jump(i2)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_and(self):
ops = """
@@ -3782,7 +3786,7 @@
guard_true(i15) []
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_subsub_ovf(self):
ops = """
@@ -3806,7 +3810,7 @@
i3 = int_sub(1, i0)
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_eq(self):
ops = """
@@ -3831,7 +3835,7 @@
guard_true(i3) []
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_bound_eq_const(self):
ops = """
@@ -3851,7 +3855,7 @@
escape(10)
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_bound_eq_const_not(self):
ops = """
@@ -3869,7 +3873,7 @@
jump(i2)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_ne_const(self):
ops = """
@@ -3886,7 +3890,7 @@
jump(10)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_ne_const_not(self):
ops = """
@@ -3903,7 +3907,7 @@
i2 = int_add(i0, 3)
jump(i2)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_ltne(self):
ops = """
@@ -3920,7 +3924,7 @@
guard_true(i2) []
jump(i0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lege_const(self):
ops = """
@@ -3941,7 +3945,7 @@
jump(10)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_mul_ovf(self):
ops = """
@@ -3973,7 +3977,7 @@
guard_true(i8) []
jump(i0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_mul_ovf_before(self):
ops = """
@@ -4002,7 +4006,7 @@
guard_true(i5) []
jump(i0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_sub_ovf_before(self):
ops = """
@@ -4031,15 +4035,14 @@
guard_true(i5) []
jump(i0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
# ----------
- def optimize_strunicode_loop(self, ops, spectext, optops):
+ def optimize_strunicode_loop(self, ops, optops):
# check with the arguments passed in
- self.optimize_loop(ops, spectext, optops)
+ self.optimize_loop(ops, optops)
# check with replacing 'str' with 'unicode' everywhere
self.optimize_loop(ops.replace('str','unicode').replace('s"', 'u"'),
- spectext,
optops.replace('str','unicode').replace('s"', 'u"'))
def test_newstr_1(self):
@@ -4054,7 +4057,7 @@
[i0]
jump(i0)
"""
- self.optimize_strunicode_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_newstr_2(self):
ops = """
@@ -4070,7 +4073,7 @@
[i0, i1]
jump(i1, i0)
"""
- self.optimize_strunicode_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_concat_1(self):
ops = """
@@ -4091,7 +4094,7 @@
copystrcontent(p2, p3, 0, i4, i5)
jump(p2, p3)
"""
- self.optimize_strunicode_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_concat_vstr2_str(self):
ops = """
@@ -4114,7 +4117,7 @@
copystrcontent(p2, p3, 0, 2, i4)
jump(i1, i0, p3)
"""
- self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_concat_str_vstr2(self):
ops = """
@@ -4138,7 +4141,7 @@
i6 = int_add(i5, 1) # will be killed by the backend
jump(i1, i0, p3)
"""
- self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_concat_str_str_str(self):
ops = """
@@ -4165,7 +4168,7 @@
copystrcontent(p3, p5, 0, i12b, i3b)
jump(p2, p3, p5)
"""
- self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_concat_str_cstr1(self):
ops = """
@@ -4184,7 +4187,7 @@
i5 = int_add(i4, 1) # will be killed by the backend
jump(p3)
"""
- self.optimize_strunicode_loop(ops, 'Not', expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_concat_consts(self):
ops = """
@@ -4200,7 +4203,7 @@
escape(s"abcde")
jump()
"""
- self.optimize_strunicode_loop(ops, '', expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_slice_1(self):
ops = """
@@ -4215,7 +4218,7 @@
copystrcontent(p1, p2, i1, 0, i3)
jump(p2, i1, i2)
"""
- self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_slice_2(self):
ops = """
@@ -4229,7 +4232,7 @@
copystrcontent(p1, p2, 0, 0, i2)
jump(p2, i2)
"""
- self.optimize_strunicode_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_slice_3(self):
ops = """
@@ -4247,7 +4250,7 @@
copystrcontent(p1, p3, i6, 0, i5)
jump(p3, i1, i2, i3, i4)
"""
- self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_slice_getitem1(self):
ops = """
@@ -4265,7 +4268,7 @@
escape(i4)
jump(p1, i1, i2, i3)
"""
- self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_slice_plain(self):
ops = """
@@ -4283,7 +4286,7 @@
escape(i4)
jump(i3, i4)
"""
- self.optimize_strunicode_loop(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_slice_concat(self):
ops = """
@@ -4304,10 +4307,10 @@
copystrcontent(p2, p4, 0, i3, i4b)
jump(p4, i1, i2, p2)
"""
- self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_strunicode_loop(ops, expected)
# ----------
- def optimize_strunicode_loop_extradescrs(self, ops, spectext, optops):
+ def optimize_strunicode_loop_extradescrs(self, ops, optops):
from pypy.jit.metainterp.optimizeopt import string
def my_callinfo_for_oopspec(oopspecindex):
calldescrtype = type(LLtypeMixin.strequaldescr)
@@ -4322,7 +4325,7 @@
saved = string.callinfo_for_oopspec
try:
string.callinfo_for_oopspec = my_callinfo_for_oopspec
- self.optimize_strunicode_loop(ops, spectext, optops)
+ self.optimize_strunicode_loop(ops, optops)
finally:
string.callinfo_for_oopspec = saved
@@ -4333,7 +4336,7 @@
escape(i0)
jump(p1, p2)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', ops)
+ self.optimize_strunicode_loop_extradescrs(ops, ops)
def test_str_equal_noop2(self):
ops = """
@@ -4358,7 +4361,7 @@
escape(i0)
jump(p1, p2, p3)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not',
+ self.optimize_strunicode_loop_extradescrs(ops,
expected)
def test_str_equal_slice1(self):
@@ -4376,7 +4379,7 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ self.optimize_strunicode_loop_extradescrs(ops,
expected)
def test_str_equal_slice2(self):
@@ -4394,7 +4397,7 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ self.optimize_strunicode_loop_extradescrs(ops,
expected)
def test_str_equal_slice3(self):
@@ -4414,7 +4417,7 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ self.optimize_strunicode_loop_extradescrs(ops,
expected)
def test_str_equal_slice4(self):
@@ -4432,7 +4435,7 @@
escape(i0)
jump(p1, i1, i2)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not',
+ self.optimize_strunicode_loop_extradescrs(ops,
expected)
def test_str_equal_slice5(self):
@@ -4452,7 +4455,7 @@
escape(i0)
jump(p1, i1, i2, i3)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not',
+ self.optimize_strunicode_loop_extradescrs(ops,
expected)
def test_str_equal_none1(self):
@@ -4468,7 +4471,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_none2(self):
ops = """
@@ -4483,7 +4486,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_nonnull1(self):
ops = """
@@ -4500,7 +4503,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_nonnull2(self):
ops = """
@@ -4518,7 +4521,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_nonnull3(self):
ops = """
@@ -4535,7 +4538,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_nonnull4(self):
ops = """
@@ -4560,7 +4563,7 @@
escape(i0)
jump(p1, p2)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_chars0(self):
ops = """
@@ -4575,7 +4578,7 @@
escape(1)
jump(i1)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_chars1(self):
ops = """
@@ -4592,7 +4595,7 @@
escape(i0)
jump(i1)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_chars2(self):
ops = """
@@ -4613,7 +4616,7 @@
escape(i0)
jump(i1, i2)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_chars3(self):
ops = """
@@ -4628,7 +4631,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_lengthmismatch1(self):
ops = """
@@ -4644,7 +4647,7 @@
escape(0)
jump(i1)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str2unicode_constant(self):
ops = """
@@ -4658,7 +4661,7 @@
escape(u"xy")
jump()
"""
- self.optimize_strunicode_loop_extradescrs(ops, '', expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str2unicode_nonconstant(self):
ops = """
@@ -4667,7 +4670,7 @@
escape(p1)
jump(p1)
"""
- self.optimize_strunicode_loop_extradescrs(ops, 'Not', ops)
+ self.optimize_strunicode_loop_extradescrs(ops, ops)
# more generally, supporting non-constant but virtual cases is
# not obvious, because of the exception UnicodeDecodeError that
# can be raised by ll_str2unicode()
@@ -4686,7 +4689,7 @@
## [i0]
## jump(1)
## """
-## self.optimize_loop(ops, 'Not', expected)
+## self.optimize_loop(ops, expected)
## def test_instanceof_guard_class(self):
## ops = """
@@ -4700,4 +4703,4 @@
## guard_class(p0, ConstClass(node_vtable)) []
## jump(1, p0)
## """
-## self.optimize_loop(ops, 'Not, Not', expected)
+## self.optimize_loop(ops, expected)
From cfbolz at codespeak.net Sat Oct 30 18:23:42 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 18:23:42 +0200 (CEST)
Subject: [pypy-svn] r78544 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030162342.D2732282C00@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 18:23:40 2010
New Revision: 78544
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
those just work
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 18:23:40 2010
@@ -1961,9 +1961,7 @@
p3 = escape()
jump(i0, p3)
"""
- xxx # was Not, Virtual(node_vtable, nextdescr=Not)
- self.optimize_loop(ops,
- expected)
+ self.optimize_loop(ops, expected)
def test_bug_2(self):
ops = """
@@ -1985,9 +1983,7 @@
p3 = escape()
jump(i0, p3)
"""
- xxx # was Not, VArray(arraydescr2, Not)
- self.optimize_loop(ops,
- expected)
+ self.optimize_loop(ops, expected)
def test_bug_3(self):
ops = """
From cfbolz at codespeak.net Sat Oct 30 18:26:58 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 18:26:58 +0200 (CEST)
Subject: [pypy-svn] r78545 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030162658.0A1BC36E0CB@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 18:26:57 2010
New Revision: 78545
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py
Log:
fix import
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py Sat Oct 30 18:26:57 2010
@@ -1,6 +1,7 @@
import py
from pypy.rpython.lltypesystem import lltype, llmemory, rffi
-from pypy.jit.metainterp.optimizeopt.virtualize import VirtualValue, OptValue, VArrayValue
+from pypy.jit.metainterp.optimizeopt.optimizer import OptValue
+from pypy.jit.metainterp.optimizeopt.virtualize import VirtualValue, VArrayValue
from pypy.jit.metainterp.optimizeopt.virtualize import VStructValue
from pypy.jit.metainterp.resume import *
from pypy.jit.metainterp.history import BoxInt, BoxPtr, ConstInt
From arigo at codespeak.net Sat Oct 30 18:32:08 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 30 Oct 2010 18:32:08 +0200 (CEST)
Subject: [pypy-svn] r78546 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test
Message-ID: <20101030163208.7FF04282C27@codespeak.net>
Author: arigo
Date: Sat Oct 30 18:32:05 2010
New Revision: 78546
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
Log:
Remove the 'spectext' argument from test_optimizebasic too.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Sat Oct 30 18:32:05 2010
@@ -1,4 +1,4 @@
-from pypy.jit.metainterp.history import Const, ConstInt
+from pypy.jit.metainterp.history import Const, ConstInt, BoxInt
from pypy.jit.metainterp.resoperation import rop, ResOperation
from pypy.jit.metainterp.optimizeutil import _findall, sort_descrs
from pypy.jit.metainterp.optimizeutil import descrlist_dict
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py Sat Oct 30 18:32:05 2010
@@ -243,7 +243,7 @@
assert equaloplists(optimized.operations,
expected.operations, False, remap)
- def optimize_loop(self, ops, spectext, optops):
+ def optimize_loop(self, ops, optops):
loop = self.parse(ops)
#
self.loop = loop
@@ -285,7 +285,7 @@
guard_value(i0, 0) [i0]
jump(1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_constant_propagate(self):
ops = """
@@ -302,7 +302,7 @@
[]
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_constant_propagate_ovf(self):
ops = """
@@ -320,7 +320,7 @@
[]
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_constfold_all(self):
from pypy.jit.backend.llgraph.llimpl import TYPES # xxx fish
@@ -354,7 +354,7 @@
escape(%d)
jump()
""" % expected_value
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
# ----------
@@ -370,7 +370,7 @@
guard_class(p0, ConstClass(node_vtable)) []
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_remove_guard_class_2(self):
ops = """
@@ -386,7 +386,7 @@
escape(p0)
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_remove_guard_class_constant(self):
ops = """
@@ -399,7 +399,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_constant_boolrewrite_lt(self):
ops = """
@@ -416,7 +416,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_constant_boolrewrite_gt(self):
ops = """
@@ -433,7 +433,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_constant_boolrewrite_reflex(self):
ops = """
@@ -450,7 +450,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_constant_boolrewrite_reflex_invers(self):
ops = """
@@ -467,7 +467,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_remove_consecutive_guard_value_constfold(self):
ops = """
@@ -487,7 +487,7 @@
escape(3)
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_remove_guard_value_if_constant(self):
ops = """
@@ -514,7 +514,7 @@
guard_class(p0, ConstClass(node_vtable)) []
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_int_is_true_1(self):
ops = """
@@ -531,7 +531,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_int_is_true_is_zero(self):
py.test.skip("XXX implement me")
@@ -549,7 +549,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_ooisnull_oononnull_2(self):
ops = """
@@ -563,7 +563,7 @@
guard_nonnull(p0) []
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_ooisnull_on_null_ptr_1(self):
ops = """
@@ -579,7 +579,7 @@
guard_isnull(p0) []
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_ooisnull_oononnull_via_virtual(self):
ops = """
@@ -596,7 +596,7 @@
guard_nonnull(p0) []
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_oois_1(self):
ops = """
@@ -617,7 +617,7 @@
guard_class(p0, ConstClass(node_vtable)) []
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_nonnull_1(self):
ops = """
@@ -639,7 +639,7 @@
setfield_gc(p0, 5, descr=valuedescr)
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_const_guard_value(self):
ops = """
@@ -652,7 +652,7 @@
[]
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_constptr_guard_value(self):
ops = """
@@ -661,7 +661,7 @@
guard_value(p1, ConstPtr(myptr)) []
jump()
"""
- self.optimize_loop(ops, '', ops)
+ self.optimize_loop(ops, ops)
def test_guard_value_to_guard_true(self):
ops = """
@@ -676,7 +676,7 @@
guard_true(i1) [i]
jump(i)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_guard_value_to_guard_false(self):
ops = """
@@ -691,7 +691,7 @@
guard_false(i1) [i]
jump(i)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_guard_value_on_nonbool(self):
ops = """
@@ -706,7 +706,7 @@
guard_value(i1, 0) [i]
jump(-3)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_int_is_true_of_bool(self):
ops = """
@@ -723,7 +723,7 @@
guard_false(i2) [i0, i1]
jump(i0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
@@ -738,7 +738,7 @@
jump(i1, p1, p2)
"""
# We cannot track virtuals that survive for more than two iterations.
- self.optimize_loop(ops, 'Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_p123_nested(self):
ops = """
@@ -754,7 +754,7 @@
"""
# The same as test_p123_simple, but with a virtual containing another
# virtual.
- self.optimize_loop(ops, 'Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_p123_anti_nested(self):
ops = """
@@ -770,7 +770,7 @@
"""
# The same as test_p123_simple, but in the end the "old" p2 contains
# a "young" virtual p2sub. Make sure it is all forced.
- self.optimize_loop(ops, 'Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
# ----------
@@ -794,7 +794,7 @@
i3 = call(i2, descr=nonwritedescr)
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
# ----------
@@ -819,7 +819,7 @@
guard_value(i2, 1) []
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
# ----------
@@ -927,7 +927,7 @@
guard_true(i11) []
jump(p0, p1, p2)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected2)
+ self.optimize_loop(ops, expected2)
def test_virtual_default_field(self):
ops = """
@@ -963,7 +963,7 @@
i1 = int_add(i, 1)
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_virtual_4(self):
ops = """
@@ -1027,7 +1027,7 @@
[i0]
jump(1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_virtual_constant_isnonnull(self):
@@ -1043,7 +1043,7 @@
[i0]
jump(0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_nonvirtual_1(self):
ops = """
@@ -1065,7 +1065,7 @@
escape(p1)
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_nonvirtual_2(self):
ops = """
@@ -1078,7 +1078,7 @@
jump(i, p1)
"""
expected = ops
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_nonvirtual_later(self):
ops = """
@@ -1100,7 +1100,7 @@
i3 = int_add(i, i2)
jump(i3)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_nonvirtual_dont_write_null_fields_on_force(self):
ops = """
@@ -1120,7 +1120,7 @@
i2 = getfield_gc(p1, descr=valuedescr)
jump(i2)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_getfield_gc_pure_1(self):
ops = """
@@ -1134,7 +1134,7 @@
[i]
jump(i)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_getfield_gc_pure_2(self):
ops = """
@@ -1147,7 +1147,7 @@
jump(5)
"""
self.node.value = 5
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_getfield_gc_nonpure_2(self):
ops = """
@@ -1156,7 +1156,7 @@
jump(i1)
"""
expected = ops
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_varray_1(self):
ops = """
@@ -1173,7 +1173,7 @@
[i1]
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_varray_alloc_and_set(self):
ops = """
@@ -1187,7 +1187,7 @@
[i1]
jump(0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_varray_float(self):
ops = """
@@ -1204,7 +1204,7 @@
[f1]
jump(f1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_array_non_optimized(self):
ops = """
@@ -1220,7 +1220,7 @@
p1 = new_array(i1, descr=arraydescr)
jump(i1, p1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_nonvirtual_array_dont_write_null_fields_on_force(self):
ops = """
@@ -1238,7 +1238,7 @@
escape(p1)
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_varray_2(self):
ops = """
@@ -1271,7 +1271,7 @@
jump(i1, p1, p2)
"""
# We cannot track virtuals that survive for more than two iterations.
- self.optimize_loop(ops, 'Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_varray_forced_1(self):
ops = """
@@ -1293,7 +1293,7 @@
escape(i2)
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_vstruct_1(self):
ops = """
@@ -1322,7 +1322,7 @@
jump(i1, p1, p2)
"""
# We cannot track virtuals that survive for more than two iterations.
- self.optimize_loop(ops, 'Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_duplicate_getfield_1(self):
ops = """
@@ -1347,7 +1347,7 @@
escape(i2)
jump(p1, p2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_getfield_after_setfield(self):
ops = """
@@ -1363,7 +1363,7 @@
escape(i1)
jump(p1, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_setfield_of_different_type_does_not_clear(self):
ops = """
@@ -1381,7 +1381,7 @@
escape(i1)
jump(p1, p2, i1)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_setfield_of_same_type_clears(self):
ops = """
@@ -1392,7 +1392,7 @@
escape(i3)
jump(p1, p2, i1, i3)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_duplicate_getfield_mergepoint_has_no_side_effects(self):
ops = """
@@ -1412,7 +1412,7 @@
escape(i1)
jump(p1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getfield_ovf_op_does_not_clear(self):
ops = """
@@ -1434,7 +1434,7 @@
escape(i1)
jump(p1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getfield_setarrayitem_does_not_clear(self):
ops = """
@@ -1454,7 +1454,7 @@
escape(i1)
jump(p1, p2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getfield_constant(self):
ops = """
@@ -1472,7 +1472,7 @@
escape(i1)
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getfield_guard_value_const(self):
ops = """
@@ -1504,7 +1504,7 @@
escape(i2)
jump(p1)
"""
- self.optimize_loop(ops, 'Not', ops)
+ self.optimize_loop(ops, ops)
def test_duplicate_getfield_sideeffects_2(self):
ops = """
@@ -1515,7 +1515,7 @@
escape(i2)
jump(p1, i1)
"""
- self.optimize_loop(ops, 'Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_duplicate_setfield_1(self):
ops = """
@@ -1529,7 +1529,7 @@
setfield_gc(p1, i2, descr=valuedescr)
jump(p1, i1, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_setfield_2(self):
ops = """
@@ -1546,7 +1546,7 @@
escape(i1)
jump(p1, i1, i3)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_setfield_3(self):
ops = """
@@ -1559,7 +1559,7 @@
"""
# potential aliasing of p1 and p2 means that we cannot kill the
# the setfield_gc
- self.optimize_loop(ops, 'Not, Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_duplicate_setfield_4(self):
ops = """
@@ -1588,7 +1588,7 @@
setfield_gc(p1, i4, descr=nextdescr)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_setfield_5(self):
ops = """
@@ -1610,7 +1610,7 @@
escape(i1)
jump(p0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_setfield_sideeffects_1(self):
ops = """
@@ -1620,7 +1620,7 @@
setfield_gc(p1, i2, descr=valuedescr)
jump(p1, i1, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_duplicate_setfield_residual_guard_1(self):
ops = """
@@ -1631,7 +1631,7 @@
setfield_gc(p1, i2, descr=valuedescr)
jump(p1, i1, i2, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_duplicate_setfield_residual_guard_2(self):
# the difference with the previous test is that the field value is
@@ -1652,7 +1652,7 @@
setfield_gc(p1, NULL, descr=nextdescr)
jump(p1, i2, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_setfield_residual_guard_3(self):
ops = """
@@ -1672,7 +1672,7 @@
setfield_gc(p1, NULL, descr=nextdescr)
jump(p1, i2, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_setfield_residual_guard_4(self):
# test that the setfield_gc does not end up between int_eq and
@@ -1686,7 +1686,7 @@
setfield_gc(p1, i2, descr=valuedescr)
jump(p1, i1, i2, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_duplicate_setfield_aliasing(self):
# a case where aliasing issues (and not enough cleverness) mean
@@ -1698,7 +1698,7 @@
setfield_gc(p1, i3, descr=valuedescr)
jump(p1, p2, i1, i2, i3)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_duplicate_setfield_guard_value_const(self):
ops = """
@@ -1739,7 +1739,7 @@
escape(p3)
jump(p1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getarrayitem_after_setarrayitem_1(self):
ops = """
@@ -1755,7 +1755,7 @@
escape(p2)
jump(p1, p2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getarrayitem_after_setarrayitem_2(self):
ops = """
@@ -1777,7 +1777,7 @@
escape(p3)
jump(p1, p2, p3, i1)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getarrayitem_after_setarrayitem_3(self):
ops = """
@@ -1804,7 +1804,7 @@
escape(p4)
jump(p1, p2, p3, p4, i1)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_getarrayitem_pure_does_not_invalidate(self):
ops = """
@@ -1825,7 +1825,7 @@
escape(p3)
jump(p1, p2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_duplicate_getarrayitem_after_setarrayitem_two_arrays(self):
ops = """
@@ -1846,7 +1846,7 @@
escape(p4)
jump(p1, p2, p3, p4, i1)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_bug_1(self):
ops = """
@@ -2017,7 +2017,7 @@
i3 = int_add(i1, i2)
jump(p2, i0, i1, i3, p2)
"""
- self.optimize_loop(ops, "Not, Not, Not, Not, Not", expected)
+ self.optimize_loop(ops, expected)
def test_merge_guard_nonnull_guard_class(self):
self.make_fail_descr()
@@ -2034,7 +2034,7 @@
i3 = int_add(i1, i2)
jump(p2, i0, i1, i3, p2)
"""
- self.optimize_loop(ops, "Not, Not, Not, Not, Not", expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr("i0", rop.GUARD_NONNULL_CLASS)
def test_merge_guard_nonnull_guard_value(self):
@@ -2052,7 +2052,7 @@
i3 = int_add(i1, i2)
jump(p2, i0, i1, i3, p2)
"""
- self.optimize_loop(ops, "Not, Not, Not, Not, Not", expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr("i0", rop.GUARD_VALUE)
def test_merge_guard_nonnull_guard_class_guard_value(self):
@@ -2073,7 +2073,7 @@
i4 = int_sub(i3, 1)
jump(p2, i0, i1, i4, p2)
"""
- self.optimize_loop(ops, "Not, Not, Not, Not, Not", expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr("i0", rop.GUARD_VALUE)
def test_guard_class_oois(self):
@@ -2089,7 +2089,7 @@
guard_class(p1, ConstClass(node_vtable2)) []
jump(p1)
"""
- self.optimize_loop(ops, "Not", expected)
+ self.optimize_loop(ops, expected)
def test_oois_of_itself(self):
ops = """
@@ -2107,7 +2107,7 @@
p1 = getfield_gc(p0, descr=nextdescr)
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_remove_duplicate_pure_op(self):
ops = """
@@ -2137,7 +2137,7 @@
guard_true(i1) []
jump(p1, p2)
"""
- self.optimize_loop(ops, "Not, Not", expected)
+ self.optimize_loop(ops, expected)
def test_remove_duplicate_pure_op_with_descr(self):
ops = """
@@ -2157,7 +2157,7 @@
guard_true(i1) []
jump(p1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_remove_duplicate_pure_op_ovf(self):
ops = """
@@ -2184,7 +2184,7 @@
escape(i3)
jump(i1)
"""
- self.optimize_loop(ops, "Not", expected)
+ self.optimize_loop(ops, expected)
def test_int_and_or_with_zero(self):
ops = """
@@ -2199,7 +2199,7 @@
[i0, i1]
jump(i1, i0)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_fold_partially_constant_ops(self):
ops = """
@@ -2211,7 +2211,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
ops = """
[i0]
@@ -2222,7 +2222,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
ops = """
[i0]
@@ -2233,7 +2233,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_fold_partially_constant_ops_ovf(self):
ops = """
@@ -2246,7 +2246,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
ops = """
[i0]
@@ -2258,7 +2258,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
ops = """
[i0]
@@ -2270,7 +2270,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
# ----------
@@ -2416,7 +2416,7 @@
guard_true(i1, descr=fdescr) [i3]
jump(1, i3)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('15, i3', rop.GUARD_TRUE)
def test_expand_fail_2(self):
@@ -2434,7 +2434,7 @@
guard_true(i1, descr=fdescr) [i2]
jump(1, i2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('''ptr
where ptr is a node_vtable, valuedescr=i2
''', rop.GUARD_TRUE)
@@ -2457,7 +2457,7 @@
guard_true(i1, descr=fdescr) [i3, i2, p3]
jump(i2, 1, i3, p3)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('''i3, p1
where p1 is a node_vtable, valuedescr=1, nextdescr=p2
where p2 is a node_vtable, valuedescr=i2, nextdescr=p3
@@ -2484,7 +2484,7 @@
guard_true(i1, descr=fdescr) [i3, i2]
jump(1, i2, i3)
"""
- self.optimize_loop(ops % arg, 'Not, Not, Not', expected)
+ self.optimize_loop(ops % arg, expected)
self.check_expanded_fail_descr('''i3, i3, %s
where p1 is a node_vtable, valuedescr=i2, nextdescr=p2
where p2 is a node_vtable, valuedescr=i2''' % arg,
@@ -2508,7 +2508,7 @@
guard_true(i1, descr=fdescr) [i3, i4, i2]
jump(i2, 1, i3, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('''i3, i4, p1, p2
where p1 is a node_vtable, valuedescr=i4, nextdescr=p2
where p2 is a node_vtable, valuedescr=i2, nextdescr=p1
@@ -2551,7 +2551,7 @@
guard_true(i1, descr=fdescr) [i1]
jump(1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('''p1
where p1 is a varray arraydescr: 25, i1
''', rop.GUARD_TRUE)
@@ -2573,7 +2573,7 @@
guard_true(i1, descr=fdescr) [i1, p1]
jump(1, p1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('''p2
where p2 is a vstruct ssize, adescr=i1, bdescr=p1
''', rop.GUARD_TRUE)
@@ -2639,7 +2639,7 @@
setfield_gc(p1, NULL, descr=nextdescr)
jump(p1, i2, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.loop.inputargs[0].value = self.nodebox.value
self.check_expanded_fail_descr('''
p1.nextdescr = p2
@@ -2665,7 +2665,7 @@
setfield_gc(ConstPtr(myptr), NULL, descr=nextdescr)
jump(i2, i4)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('''
ConstPtr(myptr).nextdescr = p2
where p2 is a node_vtable, valuedescr=i2
@@ -2692,7 +2692,7 @@
escape(i1)
jump(p1, p2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_residual_call_invalidate_some_caches(self):
ops = """
@@ -2720,7 +2720,7 @@
escape(i2)
jump(p1, p2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_residual_call_invalidate_arrays(self):
ops = """
@@ -2747,7 +2747,7 @@
escape(p4)
jump(p1, p2, i1)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_residual_call_invalidate_some_arrays(self):
ops = """
@@ -2782,7 +2782,7 @@
escape(i4)
jump(p1, p2, i1)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_residual_call_invalidates_some_read_caches_1(self):
ops = """
@@ -2802,7 +2802,7 @@
setfield_gc(p2, i3, descr=adescr)
jump(p1, i1, p2, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_residual_call_invalidates_some_read_caches_2(self):
ops = """
@@ -2822,7 +2822,7 @@
setfield_gc(p2, i3, descr=adescr)
jump(p1, i1, p2, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_residual_call_invalidates_some_read_caches_3(self):
ops = """
@@ -2834,7 +2834,7 @@
setfield_gc(p2, i3, descr=adescr)
jump(p1, i1, p2, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_call_assembler_invalidates_caches(self):
ops = '''
@@ -2844,7 +2844,7 @@
setfield_gc(p1, i3, descr=valuedescr)
jump(p1, i3)
'''
- self.optimize_loop(ops, 'Not, Not', ops)
+ self.optimize_loop(ops, ops)
def test_call_pure_invalidates_caches(self):
# CALL_PURE should still force the setfield_gc() to occur before it
@@ -2862,7 +2862,7 @@
setfield_gc(p1, i3, descr=valuedescr)
jump(p1, i3)
'''
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_call_pure_constant_folding(self):
# CALL_PURE is not marked as is_always_pure(), because it is wrong
@@ -2885,7 +2885,7 @@
i4 = call(123456, 4, i0, 6, descr=plaincalldescr)
jump(i0, 42, i4)
'''
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_vref_nonvirtual_nonescape(self):
ops = """
@@ -2899,7 +2899,7 @@
i0 = force_token()
jump(p1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_vref_nonvirtual_escape(self):
ops = """
@@ -2922,7 +2922,7 @@
"""
# XXX we should optimize a bit more the case of a nonvirtual.
# in theory it is enough to just do 'p2 = p1'.
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_vref_virtual_1(self):
ops = """
@@ -2962,7 +2962,7 @@
setfield_gc(p2, -3, descr=virtualtokendescr)
jump(p0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_vref_virtual_2(self):
self.make_fail_descr()
@@ -3005,7 +3005,7 @@
"""
# the point of this test is that 'i1' should show up in the fail_args
# of 'guard_not_forced', because it was stored in the virtual 'p1b'.
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
self.check_expanded_fail_descr('''p2, p1
where p1 is a node_vtable, nextdescr=p1b
where p1b is a node_vtable, valuedescr=i1
@@ -3037,7 +3037,7 @@
setfield_gc(p0, NULL, descr=refdescr)
jump(p0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
# the fail_args contain [i3, i1, p0]:
# - i3 is from the virtual expansion of p2
# - i1 is from the virtual expansion of p1
@@ -3076,7 +3076,7 @@
guard_not_forced() []
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_vref_nonvirtual_and_lazy_setfield(self):
self.make_fail_descr()
@@ -3102,7 +3102,7 @@
guard_not_forced() [i1]
jump(i1, p1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_arraycopy_1(self):
ops = '''
@@ -3119,7 +3119,7 @@
[i0]
jump(1)
'''
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_arraycopy_2(self):
ops = '''
@@ -3136,7 +3136,7 @@
[i0]
jump(3)
'''
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_arraycopy_not_virtual(self):
ops = '''
@@ -3154,7 +3154,7 @@
setarrayitem_gc(p2, 2, 10, descr=arraydescr)
jump(p2)
'''
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_arraycopy_no_elem(self):
""" this was actually observed in the wild
@@ -3169,7 +3169,7 @@
[p1]
jump(p1)
'''
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt(self):
ops = """
@@ -3186,7 +3186,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_noguard(self):
ops = """
@@ -3201,7 +3201,7 @@
i2 = int_lt(i0, 5)
jump(i2)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_noopt(self):
ops = """
@@ -3220,7 +3220,7 @@
guard_true(i2) []
jump(4)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_rev(self):
ops = """
@@ -3237,7 +3237,7 @@
guard_false(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_tripple(self):
ops = """
@@ -3256,7 +3256,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_add(self):
ops = """
@@ -3275,7 +3275,7 @@
i2 = int_add(i0, 10)
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_add_before(self):
ops = """
@@ -3294,7 +3294,7 @@
guard_true(i3) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_add_ovf(self):
ops = """
@@ -3314,7 +3314,7 @@
i2 = int_add(i0, 10)
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_add_ovf_before(self):
ops = """
@@ -3335,7 +3335,7 @@
guard_true(i3) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_sub(self):
ops = """
@@ -3354,7 +3354,7 @@
i2 = int_sub(i0, 10)
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lt_sub_before(self):
ops = """
@@ -3373,7 +3373,7 @@
guard_true(i3) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_ltle(self):
ops = """
@@ -3390,7 +3390,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lelt(self):
ops = """
@@ -3407,7 +3407,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_gt(self):
ops = """
@@ -3424,7 +3424,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_gtge(self):
ops = """
@@ -3441,7 +3441,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_gegt(self):
ops = """
@@ -3458,7 +3458,7 @@
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_ovf(self):
ops = """
@@ -3480,7 +3480,7 @@
i3 = int_add(i0, 1)
jump(i3)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_arraylen(self):
ops = """
@@ -3500,7 +3500,7 @@
setarrayitem_gc(p0, 0, p1)
jump(i0, p0)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_strlen(self):
ops = """
@@ -3516,7 +3516,7 @@
i0 = strlen(p0)
jump(p0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_addsub_const(self):
ops = """
@@ -3533,7 +3533,7 @@
i4 = int_mul(i0, i1)
jump(i4)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_addsub_int(self):
ops = """
@@ -3550,7 +3550,7 @@
i4 = int_add(i0, i1)
jump(i4, i10)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_addsub_int2(self):
ops = """
@@ -3567,7 +3567,7 @@
i4 = int_add(i0, i1)
jump(i4, i10)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_framestackdepth_overhead(self):
ops = """
@@ -3597,7 +3597,7 @@
setfield_gc(p0, i1, descr=valuedescr)
jump(p0, i22)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_addsub_ovf(self):
ops = """
@@ -3615,7 +3615,7 @@
i2 = int_sub(i1, 5)
jump(i2)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_subadd_ovf(self):
ops = """
@@ -3633,7 +3633,7 @@
i2 = int_add(i1, 5)
jump(i2)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_and(self):
ops = """
@@ -3678,7 +3678,7 @@
guard_true(i15) []
jump(i1)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_subsub_ovf(self):
ops = """
@@ -3702,7 +3702,7 @@
i3 = int_sub(1, i0)
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_eq(self):
ops = """
@@ -3723,7 +3723,7 @@
guard_true(i3) []
jump(i0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_eq_const(self):
ops = """
@@ -3740,7 +3740,7 @@
jump(10)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_eq_const_not(self):
ops = """
@@ -3758,7 +3758,7 @@
jump(i2)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_ne_const(self):
ops = """
@@ -3775,7 +3775,7 @@
jump(10)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_ne_const_not(self):
ops = """
@@ -3792,7 +3792,7 @@
i2 = int_add(i0, 3)
jump(i2)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_ltne(self):
ops = """
@@ -3809,7 +3809,7 @@
guard_true(i2) []
jump(i0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_bound_lege_const(self):
ops = """
@@ -3830,7 +3830,7 @@
jump(10)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_mul_ovf(self):
ops = """
@@ -3862,7 +3862,7 @@
guard_true(i8) []
jump(i0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_mul_ovf_before(self):
ops = """
@@ -3891,7 +3891,7 @@
guard_true(i5) []
jump(i0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_sub_ovf_before(self):
ops = """
@@ -3920,7 +3920,7 @@
guard_true(i5) []
jump(i0, i1)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_newstr_1(self):
ops = """
@@ -3934,7 +3934,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_newstr_2(self):
ops = """
@@ -3950,7 +3950,7 @@
[i0, i1]
jump(i1, i0)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_str_concat_1(self):
ops = """
@@ -3971,7 +3971,7 @@
copystrcontent(p2, p3, 0, i4, i5)
jump(p2, p3)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_str_concat_vstr2_str(self):
ops = """
@@ -3994,7 +3994,7 @@
copystrcontent(p2, p3, 0, 2, i4)
jump(i1, i0, p3)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_str_concat_str_vstr2(self):
ops = """
@@ -4018,7 +4018,7 @@
i6 = int_add(i5, 1) # will be killed by the backend
jump(i1, i0, p3)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_str_concat_str_str_str(self):
ops = """
@@ -4045,7 +4045,7 @@
copystrcontent(p3, p5, 0, i12b, i3b)
jump(p2, p3, p5)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_str_concat_str_cstr1(self):
ops = """
@@ -4064,7 +4064,7 @@
i5 = int_add(i4, 1) # will be killed by the backend
jump(p3)
"""
- self.optimize_loop(ops, 'Not', expected)
+ self.optimize_loop(ops, expected)
def test_str_concat_consts(self):
ops = """
@@ -4080,7 +4080,7 @@
escape("abcde")
jump()
"""
- self.optimize_loop(ops, '', expected)
+ self.optimize_loop(ops, expected)
def test_str_slice_1(self):
ops = """
@@ -4095,7 +4095,7 @@
copystrcontent(p1, p2, i1, 0, i3)
jump(p2, i1, i2)
"""
- self.optimize_loop(ops, 'Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_str_slice_2(self):
ops = """
@@ -4109,7 +4109,7 @@
copystrcontent(p1, p2, 0, 0, i2)
jump(p2, i2)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_str_slice_3(self):
ops = """
@@ -4127,7 +4127,7 @@
copystrcontent(p1, p3, i6, 0, i5)
jump(p3, i1, i2, i3, i4)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_str_slice_getitem1(self):
ops = """
@@ -4145,7 +4145,7 @@
escape(i4)
jump(p1, i1, i2, i3)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_str_slice_plain(self):
ops = """
@@ -4163,7 +4163,7 @@
escape(i4)
jump(i3, i4)
"""
- self.optimize_loop(ops, 'Not, Not', expected)
+ self.optimize_loop(ops, expected)
def test_str_slice_concat(self):
ops = """
@@ -4184,10 +4184,10 @@
copystrcontent(p2, p4, 0, i3, i4b)
jump(p4, i1, i2, p2)
"""
- self.optimize_loop(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop(ops, expected)
# ----------
- def optimize_loop_extradescrs(self, ops, spectext, optops):
+ def optimize_loop_extradescrs(self, ops, optops):
from pypy.jit.metainterp.optimizeopt import string
def my_callinfo_for_oopspec(oopspecindex):
calldescrtype = type(LLtypeMixin.strequaldescr)
@@ -4202,7 +4202,7 @@
saved = string.callinfo_for_oopspec
try:
string.callinfo_for_oopspec = my_callinfo_for_oopspec
- self.optimize_loop(ops, spectext, optops)
+ self.optimize_loop(ops, optops)
finally:
string.callinfo_for_oopspec = saved
@@ -4213,7 +4213,7 @@
escape(i0)
jump(p1, p2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', ops)
+ self.optimize_loop_extradescrs(ops, ops)
def test_str_equal_noop2(self):
ops = """
@@ -4238,7 +4238,7 @@
escape(i0)
jump(p1, p2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_slice1(self):
ops = """
@@ -4255,7 +4255,7 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_slice2(self):
ops = """
@@ -4272,7 +4272,7 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_slice3(self):
ops = """
@@ -4291,7 +4291,7 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_slice4(self):
ops = """
@@ -4308,7 +4308,7 @@
escape(i0)
jump(p1, i1, i2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_slice5(self):
ops = """
@@ -4327,7 +4327,7 @@
escape(i0)
jump(p1, i1, i2, i3)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_none1(self):
ops = """
@@ -4342,7 +4342,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_none2(self):
ops = """
@@ -4357,7 +4357,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_nonnull1(self):
ops = """
@@ -4374,7 +4374,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_nonnull2(self):
ops = """
@@ -4392,7 +4392,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_nonnull3(self):
ops = """
@@ -4409,7 +4409,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_nonnull4(self):
ops = """
@@ -4434,7 +4434,7 @@
escape(i0)
jump(p1, p2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_chars0(self):
ops = """
@@ -4449,7 +4449,7 @@
escape(1)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_chars1(self):
ops = """
@@ -4466,7 +4466,7 @@
escape(i0)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_chars2(self):
ops = """
@@ -4487,7 +4487,7 @@
escape(i0)
jump(i1, i2)
"""
- self.optimize_loop_extradescrs(ops, 'Not, Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_chars3(self):
ops = """
@@ -4502,7 +4502,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
def test_str_equal_lengthmismatch1(self):
ops = """
@@ -4518,7 +4518,7 @@
escape(0)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, 'Not', expected)
+ self.optimize_loop_extradescrs(ops, expected)
# XXX unicode operations
# XXX str2unicode
@@ -4537,7 +4537,7 @@
## [i0]
## jump(1)
## """
-## self.optimize_loop(ops, 'Not', expected)
+## self.optimize_loop(ops, expected)
## def test_instanceof_guard_class(self):
## ops = """
@@ -4551,4 +4551,4 @@
## guard_class(p0, ConstClass(node_vtable)) []
## jump(1, p0)
## """
-## self.optimize_loop(ops, 'Not, Not', expected)
+## self.optimize_loop(ops, expected)
From cfbolz at codespeak.net Sat Oct 30 18:32:13 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 18:32:13 +0200 (CEST)
Subject: [pypy-svn] r78547 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030163213.894AD282C28@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 18:32:11 2010
New Revision: 78547
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
fix a test
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 18:32:11 2010
@@ -2109,13 +2109,19 @@
guard_value(p1, ConstPtr(myptr)) [i1]
jump(p2, i0, i1, i3, p2)
"""
- expected = """
+ preamble = """
[p1, i0, i1, i2, p2]
guard_value(p1, ConstPtr(myptr)) [i0]
i3 = int_add(i1, i2)
- jump(p2, i0, i1, i3, p2)
+ jump(p2, i0, i1, i3)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [p2, i0, i1, i2]
+ guard_value(p2, ConstPtr(myptr)) [i0]
+ i3 = int_add(i1, i2)
+ jump(ConstPtr(myptr), i0, i1, i3)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_merge_guard_nonnull_guard_class(self):
ops = """
From cfbolz at codespeak.net Sat Oct 30 18:33:42 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 18:33:42 +0200 (CEST)
Subject: [pypy-svn] r78548 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030163342.737D4282BF2@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 18:33:39 2010
New Revision: 78548
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
next
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 18:33:39 2010
@@ -2154,13 +2154,19 @@
guard_value(p1, ConstPtr(myptr)) [i1]
jump(p2, i0, i1, i3, p2)
"""
- expected = """
+ preamble = """
[p1, i0, i1, i2, p2]
guard_value(p1, ConstPtr(myptr), descr=fdescr) [i0]
i3 = int_add(i1, i2)
- jump(p2, i0, i1, i3, p2)
+ jump(p2, i0, i1, i3)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [p2, i0, i1, i2]
+ guard_value(p2, ConstPtr(myptr), descr=fdescr2) [i0]
+ i3 = int_add(i1, i2)
+ jump(ConstPtr(myptr), i0, i1, i3)
+ """
+ self.optimize_loop(ops, expected, preamble)
self.check_expanded_fail_descr("i0", rop.GUARD_VALUE)
def test_merge_guard_nonnull_guard_class_guard_value(self):
From cfbolz at codespeak.net Sat Oct 30 18:35:06 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 18:35:06 +0200 (CEST)
Subject: [pypy-svn] r78549 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030163506.D66CE282C28@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 18:35:04 2010
New Revision: 78549
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
everything repeats
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 18:35:04 2010
@@ -2179,14 +2179,21 @@
guard_value(p1, ConstPtr(myptr)) [i1]
jump(p2, i0, i1, i4, p2)
"""
- expected = """
+ preamble = """
[p1, i0, i1, i2, p2]
guard_value(p1, ConstPtr(myptr), descr=fdescr) [i0]
i3 = int_add(i1, i2)
i4 = int_sub(i3, 1)
- jump(p2, i0, i1, i4, p2)
+ jump(p2, i0, i1, i4)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [p2, i0, i1, i2]
+ guard_value(p2, ConstPtr(myptr), descr=fdescr2) [i0]
+ i3 = int_add(i1, i2)
+ i4 = int_sub(i3, 1)
+ jump(ConstPtr(myptr), i0, i1, i4)
+ """
+ self.optimize_loop(ops, expected, preamble)
self.check_expanded_fail_descr("i0", rop.GUARD_VALUE)
def test_guard_class_oois(self):
From cfbolz at codespeak.net Sat Oct 30 18:36:45 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 18:36:45 +0200 (CEST)
Subject: [pypy-svn] r78550 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030163645.D7692282C2B@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 18:36:42 2010
New Revision: 78550
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
everything repeats
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 18:36:42 2010
@@ -2204,12 +2204,16 @@
guard_true(i) []
jump(p1)
"""
- expected = """
+ preamble = """
[p1]
guard_class(p1, ConstClass(node_vtable2)) []
jump(p1)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [p1]
+ jump(p1)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_oois_of_itself(self):
ops = """
From cfbolz at codespeak.net Sat Oct 30 18:38:12 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 18:38:12 +0200 (CEST)
Subject: [pypy-svn] r78551 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030163812.3455A36E3FB@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 18:38:09 2010
New Revision: 78551
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
nothing is left
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 18:38:09 2010
@@ -2226,12 +2226,16 @@
guard_false(i2) []
jump(p0)
"""
- expected = """
+ preamble = """
[p0]
p1 = getfield_gc(p0, descr=nextdescr)
jump(p0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [p0]
+ jump(p0)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_remove_duplicate_pure_op(self):
ops = """
From arigo at codespeak.net Sat Oct 30 18:45:14 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 30 Oct 2010 18:45:14 +0200 (CEST)
Subject: [pypy-svn] r78552 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030164514.D2D08282C2C@codespeak.net>
Author: arigo
Date: Sat Oct 30 18:45:05 2010
New Revision: 78552
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeutil.py
Log:
Manually merge the changes done to test_optimizefindnode
and test_optimizeopt in trunk.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py Sat Oct 30 18:45:05 2010
@@ -806,10 +806,10 @@
guard_value(i2, 1) []
i3 = call_loopinvariant(1, i1, descr=nonwritedescr)
guard_no_exception() []
- guard_value(i2, 1) []
+ guard_value(i3, 1) []
i4 = call_loopinvariant(1, i1, descr=nonwritedescr)
guard_no_exception() []
- guard_value(i2, 1) []
+ guard_value(i4, 1) []
jump(i1)
"""
expected = """
@@ -3516,7 +3516,7 @@
i0 = strlen(p0)
jump(p0)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_addsub_const(self):
ops = """
@@ -3922,6 +3922,14 @@
"""
self.optimize_loop(ops, expected)
+ # ----------
+ def optimize_strunicode_loop(self, ops, optops):
+ # check with the arguments passed in
+ self.optimize_loop(ops, optops)
+ # check with replacing 'str' with 'unicode' everywhere
+ self.optimize_loop(ops.replace('str','unicode').replace('s"', 'u"'),
+ optops.replace('str','unicode').replace('s"', 'u"'))
+
def test_newstr_1(self):
ops = """
[i0]
@@ -3934,7 +3942,7 @@
[i0]
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_newstr_2(self):
ops = """
@@ -3950,7 +3958,7 @@
[i0, i1]
jump(i1, i0)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_concat_1(self):
ops = """
@@ -3971,7 +3979,7 @@
copystrcontent(p2, p3, 0, i4, i5)
jump(p2, p3)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_concat_vstr2_str(self):
ops = """
@@ -3994,7 +4002,7 @@
copystrcontent(p2, p3, 0, 2, i4)
jump(i1, i0, p3)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_concat_str_vstr2(self):
ops = """
@@ -4018,7 +4026,7 @@
i6 = int_add(i5, 1) # will be killed by the backend
jump(i1, i0, p3)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_concat_str_str_str(self):
ops = """
@@ -4045,12 +4053,12 @@
copystrcontent(p3, p5, 0, i12b, i3b)
jump(p2, p3, p5)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_concat_str_cstr1(self):
ops = """
[p2]
- p3 = call(0, p2, "x", descr=strconcatdescr)
+ p3 = call(0, p2, s"x", descr=strconcatdescr)
jump(p3)
"""
expected = """
@@ -4064,28 +4072,28 @@
i5 = int_add(i4, 1) # will be killed by the backend
jump(p3)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_concat_consts(self):
ops = """
[]
- p1 = same_as("ab")
- p2 = same_as("cde")
+ p1 = same_as(s"ab")
+ p2 = same_as(s"cde")
p3 = call(0, p1, p2, descr=strconcatdescr)
escape(p3)
jump()
"""
expected = """
[]
- escape("abcde")
+ escape(s"abcde")
jump()
"""
- self.optimize_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_slice_1(self):
ops = """
[p1, i1, i2]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
jump(p2, i1, i2)
"""
expected = """
@@ -4095,12 +4103,12 @@
copystrcontent(p1, p2, i1, 0, i3)
jump(p2, i1, i2)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_slice_2(self):
ops = """
[p1, i2]
- p2 = call(0, p1, 0, i2, descr=slicedescr)
+ p2 = call(0, p1, 0, i2, descr=strslicedescr)
jump(p2, i2)
"""
expected = """
@@ -4109,13 +4117,13 @@
copystrcontent(p1, p2, 0, 0, i2)
jump(p2, i2)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_slice_3(self):
ops = """
[p1, i1, i2, i3, i4]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
- p3 = call(0, p2, i3, i4, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
+ p3 = call(0, p2, i3, i4, descr=strslicedescr)
jump(p3, i1, i2, i3, i4)
"""
expected = """
@@ -4127,12 +4135,12 @@
copystrcontent(p1, p3, i6, 0, i5)
jump(p3, i1, i2, i3, i4)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_slice_getitem1(self):
ops = """
[p1, i1, i2, i3]
- p2 = call(0, p1, i1, i2, descr=slicedescr)
+ p2 = call(0, p1, i1, i2, descr=strslicedescr)
i4 = strgetitem(p2, i3)
escape(i4)
jump(p1, i1, i2, i3)
@@ -4145,7 +4153,7 @@
escape(i4)
jump(p1, i1, i2, i3)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_slice_plain(self):
ops = """
@@ -4153,7 +4161,7 @@
p1 = newstr(2)
strsetitem(p1, 0, i3)
strsetitem(p1, 1, i4)
- p2 = call(0, p1, 1, 2, descr=slicedescr)
+ p2 = call(0, p1, 1, 2, descr=strslicedescr)
i5 = strgetitem(p2, 0)
escape(i5)
jump(i3, i4)
@@ -4163,12 +4171,12 @@
escape(i4)
jump(i3, i4)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected)
def test_str_slice_concat(self):
ops = """
[p1, i1, i2, p2]
- p3 = call(0, p1, i1, i2, descr=slicedescr)
+ p3 = call(0, p1, i1, i2, descr=strslicedescr)
p4 = call(0, p3, p2, descr=strconcatdescr)
jump(p4, i1, i2, p2)
"""
@@ -4184,10 +4192,10 @@
copystrcontent(p2, p4, 0, i3, i4b)
jump(p4, i1, i2, p2)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected)
# ----------
- def optimize_loop_extradescrs(self, ops, optops):
+ def optimize_strunicode_loop_extradescrs(self, ops, optops):
from pypy.jit.metainterp.optimizeopt import string
def my_callinfo_for_oopspec(oopspecindex):
calldescrtype = type(LLtypeMixin.strequaldescr)
@@ -4202,7 +4210,7 @@
saved = string.callinfo_for_oopspec
try:
string.callinfo_for_oopspec = my_callinfo_for_oopspec
- self.optimize_loop(ops, optops)
+ self.optimize_strunicode_loop(ops, optops)
finally:
string.callinfo_for_oopspec = saved
@@ -4213,7 +4221,7 @@
escape(i0)
jump(p1, p2)
"""
- self.optimize_loop_extradescrs(ops, ops)
+ self.optimize_strunicode_loop_extradescrs(ops, ops)
def test_str_equal_noop2(self):
ops = """
@@ -4238,12 +4246,12 @@
escape(i0)
jump(p1, p2, p3)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_slice1(self):
ops = """
[p1, i1, i2, p3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p4, p3, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4255,12 +4263,12 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_slice2(self):
ops = """
[p1, i1, i2, p3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p3, p4, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4272,13 +4280,13 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_slice3(self):
ops = """
[p1, i1, i2, p3]
guard_nonnull(p3) []
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
i0 = call(0, p3, p4, descr=strequaldescr)
escape(i0)
jump(p1, i1, i2, p3)
@@ -4291,13 +4299,13 @@
escape(i0)
jump(p1, i1, i2, p3)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_slice4(self):
ops = """
[p1, i1, i2]
- p3 = call(0, p1, i1, i2, descr=slicedescr)
- i0 = call(0, p3, "x", descr=strequaldescr)
+ p3 = call(0, p1, i1, i2, descr=strslicedescr)
+ i0 = call(0, p3, s"x", descr=strequaldescr)
escape(i0)
jump(p1, i1, i2)
"""
@@ -4308,12 +4316,12 @@
escape(i0)
jump(p1, i1, i2)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_slice5(self):
ops = """
[p1, i1, i2, i3]
- p4 = call(0, p1, i1, i2, descr=slicedescr)
+ p4 = call(0, p1, i1, i2, descr=strslicedescr)
p5 = newstr(1)
strsetitem(p5, 0, i3)
i0 = call(0, p5, p4, descr=strequaldescr)
@@ -4327,7 +4335,7 @@
escape(i0)
jump(p1, i1, i2, i3)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_none1(self):
ops = """
@@ -4342,7 +4350,7 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_none2(self):
ops = """
@@ -4357,30 +4365,30 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_nonnull1(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "hello world", descr=strequaldescr)
+ i0 = call(0, p1, s"hello world", descr=strequaldescr)
escape(i0)
jump(p1)
"""
expected = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "hello world", descr=streq_nonnull_descr)
+ i0 = call(0, p1, s"hello world", descr=streq_nonnull_descr)
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_nonnull2(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "", descr=strequaldescr)
+ i0 = call(0, p1, s"", descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4392,13 +4400,13 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_nonnull3(self):
ops = """
[p1]
guard_nonnull(p1) []
- i0 = call(0, p1, "x", descr=strequaldescr)
+ i0 = call(0, p1, s"x", descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4409,13 +4417,13 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_nonnull4(self):
ops = """
[p1, p2]
p4 = call(0, p1, p2, descr=strconcatdescr)
- i0 = call(0, "hello world", p4, descr=strequaldescr)
+ i0 = call(0, s"hello world", p4, descr=strequaldescr)
escape(i0)
jump(p1, p2)
"""
@@ -4430,17 +4438,17 @@
i5 = strlen(p2)
i6 = int_add(i4, i5) # will be killed by the backend
copystrcontent(p2, p4, 0, i4, i5)
- i0 = call(0, "hello world", p4, descr=streq_nonnull_descr)
+ i0 = call(0, s"hello world", p4, descr=streq_nonnull_descr)
escape(i0)
jump(p1, p2)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_chars0(self):
ops = """
[i1]
p1 = newstr(0)
- i0 = call(0, p1, "", descr=strequaldescr)
+ i0 = call(0, p1, s"", descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4449,14 +4457,14 @@
escape(1)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_chars1(self):
ops = """
[i1]
p1 = newstr(1)
strsetitem(p1, 0, i1)
- i0 = call(0, p1, "x", descr=strequaldescr)
+ i0 = call(0, p1, s"x", descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4466,7 +4474,7 @@
escape(i0)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_chars2(self):
ops = """
@@ -4474,7 +4482,7 @@
p1 = newstr(2)
strsetitem(p1, 0, i1)
strsetitem(p1, 1, i2)
- i0 = call(0, p1, "xy", descr=strequaldescr)
+ i0 = call(0, p1, s"xy", descr=strequaldescr)
escape(i0)
jump(i1, i2)
"""
@@ -4483,16 +4491,16 @@
p1 = newstr(2)
strsetitem(p1, 0, i1)
strsetitem(p1, 1, i2)
- i0 = call(0, p1, "xy", descr=streq_lengthok_descr)
+ i0 = call(0, p1, s"xy", descr=streq_lengthok_descr)
escape(i0)
jump(i1, i2)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_chars3(self):
ops = """
[p1]
- i0 = call(0, "x", p1, descr=strequaldescr)
+ i0 = call(0, s"x", p1, descr=strequaldescr)
escape(i0)
jump(p1)
"""
@@ -4502,14 +4510,14 @@
escape(i0)
jump(p1)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
def test_str_equal_lengthmismatch1(self):
ops = """
[i1]
p1 = newstr(1)
strsetitem(p1, 0, i1)
- i0 = call(0, "xy", p1, descr=strequaldescr)
+ i0 = call(0, s"xy", p1, descr=strequaldescr)
escape(i0)
jump(i1)
"""
@@ -4518,10 +4526,33 @@
escape(0)
jump(i1)
"""
- self.optimize_loop_extradescrs(ops, expected)
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
- # XXX unicode operations
- # XXX str2unicode
+ def test_str2unicode_constant(self):
+ ops = """
+ []
+ p0 = call(0, "xy", descr=s2u_descr) # string -> unicode
+ escape(p0)
+ jump()
+ """
+ expected = """
+ []
+ escape(u"xy")
+ jump()
+ """
+ self.optimize_strunicode_loop_extradescrs(ops, expected)
+
+ def test_str2unicode_nonconstant(self):
+ ops = """
+ [p0]
+ p1 = call(0, p0, descr=s2u_descr) # string -> unicode
+ escape(p1)
+ jump(p1)
+ """
+ self.optimize_strunicode_loop_extradescrs(ops, ops)
+ # more generally, supporting non-constant but virtual cases is
+ # not obvious, because of the exception UnicodeDecodeError that
+ # can be raised by ll_str2unicode()
##class TestOOtype(BaseTestOptimizeBasic, OOtypeMixin):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeutil.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeutil.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeutil.py Sat Oct 30 18:45:05 2010
@@ -110,33 +110,32 @@
EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE))
arraycopydescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
EffectInfo([], [], [], oopspecindex=EffectInfo.OS_ARRAYCOPY))
- strconcatdescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_CONCAT))
- slicedescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_SLICE))
- strequaldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_EQUAL))
- streq_slice_checknull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_CHECKNULL))
- streq_slice_nonnull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_NONNULL))
- streq_slice_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_SLICE_CHAR))
- streq_nonnull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_NONNULL))
- streq_nonnull_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_NONNULL_CHAR))
- streq_checknull_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_CHECKNULL_CHAR))
- streq_lengthok_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
- EffectInfo([], [], [],
- oopspecindex=EffectInfo.OS_STREQ_LENGTHOK))
+
+ for _name, _os in [
+ ('strconcatdescr', 'OS_STR_CONCAT'),
+ ('strslicedescr', 'OS_STR_SLICE'),
+ ('strequaldescr', 'OS_STR_EQUAL'),
+ ('streq_slice_checknull_descr', 'OS_STREQ_SLICE_CHECKNULL'),
+ ('streq_slice_nonnull_descr', 'OS_STREQ_SLICE_NONNULL'),
+ ('streq_slice_char_descr', 'OS_STREQ_SLICE_CHAR'),
+ ('streq_nonnull_descr', 'OS_STREQ_NONNULL'),
+ ('streq_nonnull_char_descr', 'OS_STREQ_NONNULL_CHAR'),
+ ('streq_checknull_char_descr', 'OS_STREQ_CHECKNULL_CHAR'),
+ ('streq_lengthok_descr', 'OS_STREQ_LENGTHOK'),
+ ]:
+ _oopspecindex = getattr(EffectInfo, _os)
+ locals()[_name] = \
+ cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=_oopspecindex))
+ #
+ _oopspecindex = getattr(EffectInfo, _os.replace('STR', 'UNI'))
+ locals()[_name.replace('str', 'unicode')] = \
+ cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=_oopspecindex))
+
+ s2u_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT,
+ EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR2UNICODE))
+ #
class LoopToken(AbstractDescr):
pass
From hakanardo at codespeak.net Sat Oct 30 18:56:00 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 18:56:00 +0200 (CEST)
Subject: [pypy-svn] r78553 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030165600.488F136E3FF@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 18:55:58 2010
New Revision: 78553
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(armin, cfbolz, hakanardo) fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 18:55:58 2010
@@ -2007,28 +2007,32 @@
"""
preamble = """
[p1]
- guard_nonnull(p1) []
- guard_class(p1, ConstClass(node_vtable2)) []
+ guard_nonnull_class(p1, ConstClass(node_vtable2)) []
p2 = getfield_gc(p1, descr=nextdescr)
guard_class(p2, ConstClass(node_vtable)) []
p3 = getfield_gc(p1, descr=otherdescr)
guard_class(p3, ConstClass(node_vtable)) []
+ setfield_gc(p3, p2, descr=otherdescr)
p3a = new_with_vtable(ConstClass(node_vtable))
escape(p3a)
- setfield_gc(p3, p2, descr=otherdescr)
- jump(p2a, p3a)
+ jump(p3a)
"""
expected = """
- [p2, p3]
- guard_class(p2, ConstClass(node_vtable)) []
- guard_class(p3, ConstClass(node_vtable)) []
- setfield_gc(p3, p2, descr=otherdescr)
- p3a = new_with_vtable(ConstClass(node_vtable))
- escape(p3a)
- p2a = new_with_vtable(ConstClass(node_vtable))
- jump(p2a, p3a)
+ [p3a]
+ # p1=p1a(next=p2a, other=p3a), p2()
+ # p2 = getfield_gc(p1, descr=nextdescr) # p2a
+ # p3 = getfield_gc(p1, descr=otherdescr)# p3a
+ # setfield_gc(p3, p2, descr=otherdescr) # p3a.other = p2a
+ # p1a = new_with_vtable(ConstClass(node_vtable2))
+ # p2a = new_with_vtable(ConstClass(node_vtable))
+ p2 = new_with_vtable(ConstClass(node_vtable))
+ setfield_gc(p3a, p2, descr=otherdescr) # p3a.other = p2a
+ p3anew = new_with_vtable(ConstClass(node_vtable))
+ escape(p3anew)
+ jump(p3anew)
"""
- self.optimize_loop(ops, expected) # XXX Virtual(node_vtable2, nextdescr=Not, otherdescr=Not)
+ #self.optimize_loop(ops, expected) # XXX Virtual(node_vtable2, nextdescr=Not, otherdescr=Not)
+ self.optimize_loop(ops, expected, preamble)
def test_bug_3bis(self):
ops = """
From hakanardo at codespeak.net Sat Oct 30 19:03:28 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 19:03:28 +0200 (CEST)
Subject: [pypy-svn] r78554 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030170328.2747C282C2E@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 19:03:26 2010
New Revision: 78554
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(armin, hakanardo) fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 19:03:26 2010
@@ -2054,17 +2054,31 @@
setfield_gc(p1a, p3a, descr=otherdescr)
jump(p1a)
"""
- expected = """
- [p2, p3]
+ preamble = """
+ [p1]
+ guard_nonnull_class(p1, ConstClass(node_vtable2)) []
+ p2 = getfield_gc(p1, descr=nextdescr)
guard_class(p2, ConstClass(node_vtable)) []
+ p3 = getfield_gc(p1, descr=otherdescr)
guard_class(p3, ConstClass(node_vtable)) []
+ # p1a = new_with_vtable(ConstClass(node_vtable2))
+ p2a = new_with_vtable(ConstClass(node_vtable))
+ setfield_gc(p3, p2a, descr=otherdescr)
+ p3a = new_with_vtable(ConstClass(node_vtable))
+ escape(p3a)
+ # setfield_gc(p1a, p2a, descr=nextdescr)
+ # setfield_gc(p1a, p3a, descr=otherdescr)
+ jump(p2a, p3a)
+ """
+ expected = """
+ [p2, p3]
p2a = new_with_vtable(ConstClass(node_vtable))
setfield_gc(p3, p2a, descr=otherdescr)
p3a = new_with_vtable(ConstClass(node_vtable))
escape(p3a)
jump(p2a, p3a)
"""
- self.optimize_loop(ops, expected) # XXX was Virtual(node_vtable2, nextdescr=Not, otherdescr=Not)
+ self.optimize_loop(ops, expected, preamble)
def test_invalid_loop_1(self):
ops = """
From cfbolz at codespeak.net Sat Oct 30 19:07:33 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 19:07:33 +0200 (CEST)
Subject: [pypy-svn] r78555 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030170733.1B4E936E3BC@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 19:07:31 2010
New Revision: 78555
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
kill tests that are about expanding the fail args. they have nearly nothing to
do with unrolling and still exist in optimizebasic
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 19:07:31 2010
@@ -2527,179 +2527,6 @@
boxes = reader.consume_boxes()
self._verify_fail_args(boxes, fdescr.oparse, expectedtext)
- def test_expand_fail_1(self):
- ops = """
- [i1, i3]
- # first rename i3 into i4
- p1 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, i3, descr=valuedescr)
- i4 = getfield_gc(p1, descr=valuedescr)
- #
- i2 = int_add(10, 5)
- guard_true(i1, descr=fdescr) [i2, i4]
- jump(i1, i4)
- """
- expected = """
- [i1, i3]
- guard_true(i1, descr=fdescr) [i3]
- jump(1, i3)
- """
- self.optimize_loop(ops, expected)
- self.check_expanded_fail_descr('15, i3', rop.GUARD_TRUE)
-
- def test_expand_fail_2(self):
- ops = """
- [i1, i2]
- p1 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, i2, descr=valuedescr)
- setfield_gc(p1, p1, descr=nextdescr)
- guard_true(i1, descr=fdescr) [p1]
- jump(i1, i2)
- """
- expected = """
- [i1, i2]
- guard_true(i1, descr=fdescr) [i2]
- jump(1, i2)
- """
- self.optimize_loop(ops, expected)
- self.check_expanded_fail_descr('''ptr
- where ptr is a node_vtable, valuedescr=i2
- ''', rop.GUARD_TRUE)
-
- def test_expand_fail_3(self):
- ops = """
- [i1, i2, i3, p3]
- p1 = new_with_vtable(ConstClass(node_vtable))
- p2 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, 1, descr=valuedescr)
- setfield_gc(p1, p2, descr=nextdescr)
- setfield_gc(p2, i2, descr=valuedescr)
- setfield_gc(p2, p3, descr=nextdescr)
- guard_true(i1, descr=fdescr) [i3, p1]
- jump(i2, i1, i3, p3)
- """
- expected = """
- [i1, i2, i3, p3]
- guard_true(i1, descr=fdescr) [i3, i2, p3]
- jump(i2, 1, i3, p3)
- """
- self.optimize_loop(ops, expected)
- self.check_expanded_fail_descr('''i3, p1
- where p1 is a node_vtable, valuedescr=1, nextdescr=p2
- where p2 is a node_vtable, valuedescr=i2, nextdescr=p3
- ''', rop.GUARD_TRUE)
-
- def test_expand_fail_4(self):
- for arg in ['p1', 'i2,p1', 'p1,p2', 'p2,p1',
- 'i2,p1,p2', 'i2,p2,p1']:
- self.setup_method() # humpf
- ops = """
- [i1, i2, i3]
- p1 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, i3, descr=valuedescr)
- i4 = getfield_gc(p1, descr=valuedescr) # copy of i3
- p2 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, i2, descr=valuedescr)
- setfield_gc(p1, p2, descr=nextdescr)
- setfield_gc(p2, i2, descr=valuedescr)
- guard_true(i1, descr=fdescr) [i4, i3, %s]
- jump(i1, i2, i3)
- """
- expected = """
- [i1, i2, i3]
- guard_true(i1, descr=fdescr) [i3, i2]
- jump(1, i2, i3)
- """
- self.optimize_loop(ops % arg, expected)
- self.check_expanded_fail_descr('''i3, i3, %s
- where p1 is a node_vtable, valuedescr=i2, nextdescr=p2
- where p2 is a node_vtable, valuedescr=i2''' % arg,
- rop.GUARD_TRUE)
-
- def test_expand_fail_5(self):
- ops = """
- [i1, i2, i3, i4]
- p1 = new_with_vtable(ConstClass(node_vtable))
- p2 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, i4, descr=valuedescr)
- setfield_gc(p1, p2, descr=nextdescr)
- setfield_gc(p2, i2, descr=valuedescr)
- setfield_gc(p2, p1, descr=nextdescr) # a cycle
- guard_true(i1, descr=fdescr) [i3, i4, p1, p2]
- jump(i2, i1, i3, i4)
- """
- expected = """
- [i1, i2, i3, i4]
- guard_true(i1, descr=fdescr) [i3, i4, i2]
- jump(i2, 1, i3, i4)
- """
- self.optimize_loop(ops, expected)
- self.check_expanded_fail_descr('''i3, i4, p1, p2
- where p1 is a node_vtable, valuedescr=i4, nextdescr=p2
- where p2 is a node_vtable, valuedescr=i2, nextdescr=p1
- ''', rop.GUARD_TRUE)
-
- def test_expand_fail_6(self):
- ops = """
- [p0, i0, i1]
- guard_true(i0, descr=fdescr) [p0]
- p1 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p1, i1, descr=valuedescr)
- jump(p1, i1, i1)
- """
- expected = """
- [i1b, i0, i1]
- guard_true(i0, descr=fdescr) [i1b]
- jump(i1, i1, i1)
- """
- xxx
- self.optimize_loop(ops, '''Virtual(node_vtable, valuedescr=Not),
- Not, Not''', expected)
- self.check_expanded_fail_descr('''p0
- where p0 is a node_vtable, valuedescr=i1b
- ''', rop.GUARD_TRUE)
-
- def test_expand_fail_varray(self):
- ops = """
- [i1]
- p1 = new_array(3, descr=arraydescr)
- setarrayitem_gc(p1, 1, i1, descr=arraydescr)
- setarrayitem_gc(p1, 0, 25, descr=arraydescr)
- guard_true(i1, descr=fdescr) [p1]
- i2 = getarrayitem_gc(p1, 1, descr=arraydescr)
- jump(i2)
- """
- expected = """
- [i1]
- guard_true(i1, descr=fdescr) [i1]
- jump(1)
- """
- self.optimize_loop(ops, expected)
- self.check_expanded_fail_descr('''p1
- where p1 is a varray arraydescr: 25, i1
- ''', rop.GUARD_TRUE)
-
- def test_expand_fail_vstruct(self):
- ops = """
- [i1, p1]
- p2 = new(descr=ssize)
- setfield_gc(p2, i1, descr=adescr)
- setfield_gc(p2, p1, descr=bdescr)
- guard_true(i1, descr=fdescr) [p2]
- i3 = getfield_gc(p2, descr=adescr)
- p3 = getfield_gc(p2, descr=bdescr)
- jump(i3, p3)
- """
- expected = """
- [i1, p1]
- guard_true(i1, descr=fdescr) [i1, p1]
- jump(1, p1)
- """
- self.optimize_loop(ops, expected)
- self.check_expanded_fail_descr('''p2
- where p2 is a vstruct ssize, adescr=i1, bdescr=p1
- ''', rop.GUARD_TRUE)
-
def test_expand_fail_v_all_1(self):
ops = """
[i1, p1a, i2]
From arigo at codespeak.net Sat Oct 30 19:07:57 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 30 Oct 2010 19:07:57 +0200 (CEST)
Subject: [pypy-svn] r78556 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030170757.B60F1282C2E@codespeak.net>
Author: arigo
Date: Sat Oct 30 19:07:55 2010
New Revision: 78556
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
Improve the printing done by equaloplists().
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py Sat Oct 30 19:07:55 2010
@@ -129,7 +129,8 @@
# ____________________________________________________________
-def equaloplists(oplist1, oplist2, strict_fail_args=True, remap={}):
+def equaloplists(oplist1, oplist2, strict_fail_args=True, remap={},
+ text_right=None):
# try to use the full width of the terminal to display the list
# unfortunately, does not work with the default capture method of py.test
# (which is fd), you you need to use either -s or --capture=sys, else you
@@ -137,7 +138,8 @@
totwidth = py.io.get_terminal_width()
width = totwidth / 2 - 1
print ' Comparing lists '.center(totwidth, '-')
- print '%s| %s' % ('optimized'.center(width), 'expected'.center(width))
+ text_right = text_right or 'expected'
+ print '%s| %s' % ('optimized'.center(width), text_right.center(width))
for op1, op2 in zip(oplist1, oplist2):
txt1 = str(op1)
txt2 = str(op2)
@@ -167,7 +169,7 @@
fail_args2 = set([remap.get(y, y) for y in op2.getfailargs()])
assert fail_args1 == fail_args2
assert len(oplist1) == len(oplist2)
- print '-'*57
+ print '-'*totwidth
return True
def test_equaloplists():
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 19:07:55 2010
@@ -55,14 +55,14 @@
descr.rd_snapshot = resume.Snapshot(None, _sortboxes(fail_args))
return descr
- def assert_equal(self, optimized, expected):
+ def assert_equal(self, optimized, expected, text_right=None):
assert len(optimized.inputargs) == len(expected.inputargs)
remap = {}
for box1, box2 in zip(optimized.inputargs, expected.inputargs):
assert box1.__class__ == box2.__class__
remap[box2] = box1
assert equaloplists(optimized.operations,
- expected.operations, False, remap)
+ expected.operations, False, remap, text_right)
def optimize_loop(self, ops, optops, expected_preamble=None):
loop = self.parse(ops)
@@ -89,7 +89,8 @@
self.assert_equal(loop, expected)
if expected_preamble:
- self.assert_equal(loop.preamble, expected_preamble)
+ self.assert_equal(loop.preamble, expected_preamble,
+ text_right='expected preamble')
return loop
From hakanardo at codespeak.net Sat Oct 30 19:10:08 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 19:10:08 +0200 (CEST)
Subject: [pypy-svn] r78557 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030171008.D351636E46C@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 19:10:07 2010
New Revision: 78557
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(armin, cfbolz, hakanardo) fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 19:10:07 2010
@@ -2090,10 +2090,9 @@
jump(p2)
"""
py.test.raises(InvalidLoop, self.optimize_loop,
- ops, 'Virtual(node_vtable)', None)
+ ops, ops)
def test_invalid_loop_2(self):
- py.test.skip("this would fail if we had Fixed again in the specnodes")
ops = """
[p1]
guard_class(p1, ConstClass(node_vtable2)) []
@@ -2103,7 +2102,7 @@
jump(p2)
"""
py.test.raises(InvalidLoop, self.optimize_loop,
- ops, '...', None)
+ ops, ops)
def test_invalid_loop_3(self):
ops = """
@@ -2116,9 +2115,8 @@
setfield_gc(p3, p4, descr=nextdescr)
jump(p3)
"""
- py.test.raises(InvalidLoop, self.optimize_loop, ops,
- 'Virtual(node_vtable, nextdescr=Virtual(node_vtable))',
- None)
+ py.test.raises(InvalidLoop, self.optimize_loop, ops, ops)
+
def test_merge_guard_class_guard_value(self):
ops = """
From hakanardo at codespeak.net Sat Oct 30 19:14:28 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 19:14:28 +0200 (CEST)
Subject: [pypy-svn] r78558 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030171428.C2BC1282C30@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 19:14:27 2010
New Revision: 78558
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(armin, cfbolz, hakanardo) fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 19:14:27 2010
@@ -2271,7 +2271,7 @@
guard_true(i2) []
jump(p1, p2)
"""
- expected = """
+ preamble = """
[p1, p2]
i1 = ptr_eq(p1, p2)
i3 = int_add(i1, 1)
@@ -2282,7 +2282,13 @@
guard_true(i1) []
jump(p1, p2)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [p1, p2]
+ escape(2)
+ escape(2)
+ jump(p1, p2)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_remove_duplicate_pure_op_with_descr(self):
ops = """
From cfbolz at codespeak.net Sat Oct 30 19:15:29 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 19:15:29 +0200 (CEST)
Subject: [pypy-svn] r78559 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030171529.1424136E476@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 19:15:27 2010
New Revision: 78559
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
further improvement possible
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 19:15:27 2010
@@ -2821,6 +2821,7 @@
# time. Check that it is either constant-folded (and replaced by
# the result of the call, recorded as the first arg), or turned into
# a regular CALL.
+ # XXX can this test be improved with unrolling?
ops = '''
[i0, i1, i2]
escape(i1)
@@ -2829,14 +2830,21 @@
i4 = call_pure(43, 123456, 4, i0, 6, descr=plaincalldescr)
jump(i0, i3, i4)
'''
- expected = '''
+ preamble = '''
[i0, i1, i2]
escape(i1)
escape(i2)
i4 = call(123456, 4, i0, 6, descr=plaincalldescr)
- jump(i0, 42, i4)
+ jump(i0, i4)
'''
- self.optimize_loop(ops, expected)
+ expected = '''
+ [i0, i2]
+ escape(42)
+ escape(i2)
+ i4 = call(123456, 4, i0, 6, descr=plaincalldescr)
+ jump(i0, i4)
+ '''
+ self.optimize_loop(ops, expected, preamble)
def test_vref_nonvirtual_nonescape(self):
ops = """
From hakanardo at codespeak.net Sat Oct 30 19:19:00 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 19:19:00 +0200 (CEST)
Subject: [pypy-svn] r78560 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030171900.ED228282C33@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 19:18:59 2010
New Revision: 78560
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 19:18:59 2010
@@ -2301,14 +2301,18 @@
guard_true(i3) []
jump(p1)
"""
- expected = """
+ preamble = """
[p1]
i0 = arraylen_gc(p1, descr=arraydescr)
i1 = int_gt(i0, 0)
guard_true(i1) []
jump(p1)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [p1]
+ jump(p1)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_remove_duplicate_pure_op_ovf(self):
ops = """
@@ -2325,7 +2329,7 @@
escape(i4)
jump(i1)
"""
- expected = """
+ preamble = """
[i1]
i3 = int_add_ovf(i1, 1)
guard_no_overflow() []
@@ -2333,9 +2337,15 @@
guard_true(i3b) []
escape(i3)
escape(i3)
- jump(i1)
+ jump(i1, i3)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i1, i3]
+ escape(i3)
+ escape(i3)
+ jump(i1, i3)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_int_and_or_with_zero(self):
ops = """
From cfbolz at codespeak.net Sat Oct 30 19:20:06 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 19:20:06 +0200 (CEST)
Subject: [pypy-svn] r78561 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030172006.1FE8136E477@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 19:20:04 2010
New Revision: 78561
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
fix
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 19:20:04 2010
@@ -3147,13 +3147,18 @@
guard_true(i2) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_lt(i0, 4)
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+
+ self.optimize_loop(ops, expected, preamble)
def test_bound_lt_noguard(self):
ops = """
From cfbolz at codespeak.net Sat Oct 30 19:20:45 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sat, 30 Oct 2010 19:20:45 +0200 (CEST)
Subject: [pypy-svn] r78562 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030172045.0B502282C35@codespeak.net>
Author: cfbolz
Date: Sat Oct 30 19:20:43 2010
New Revision: 78562
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
kill some tests. their equivalent in test_optimizebasic should be fixed
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 19:20:43 2010
@@ -2542,97 +2542,6 @@
boxes = reader.consume_boxes()
self._verify_fail_args(boxes, fdescr.oparse, expectedtext)
- def test_expand_fail_v_all_1(self):
- ops = """
- [i1, p1a, i2]
- p6s = getarrayitem_gc(p1a, 0, descr=arraydescr2)
- p7v = getfield_gc(p6s, descr=bdescr)
- p5s = new(descr=ssize)
- setfield_gc(p5s, i2, descr=adescr)
- setfield_gc(p5s, p7v, descr=bdescr)
- setarrayitem_gc(p1a, 1, p5s, descr=arraydescr2)
- guard_true(i1, descr=fdescr) [p1a]
- p2s = new(descr=ssize)
- p3v = new_with_vtable(ConstClass(node_vtable))
- p4a = new_array(2, descr=arraydescr2)
- setfield_gc(p2s, i1, descr=adescr)
- setfield_gc(p2s, p3v, descr=bdescr)
- setfield_gc(p3v, i2, descr=valuedescr)
- setarrayitem_gc(p4a, 0, p2s, descr=arraydescr2)
- jump(i1, p4a, i2)
- """
- expected = """
- [i1, ia, iv, pnull, i2]
- guard_true(i1, descr=fdescr) [ia, iv, i2]
- jump(1, 1, i2, NULL, i2)
- """
- xxx
- self.optimize_loop(ops, '''
- Not,
- VArray(arraydescr2,
- VStruct(ssize,
- adescr=Not,
- bdescr=Virtual(node_vtable,
- valuedescr=Not)),
- Not),
- Not''', expected)
- self.check_expanded_fail_descr('''p1a
- where p1a is a varray arraydescr2: p6s, p5s
- where p6s is a vstruct ssize, adescr=ia, bdescr=p7v
- where p5s is a vstruct ssize, adescr=i2, bdescr=p7v
- where p7v is a node_vtable, valuedescr=iv
- ''', rop.GUARD_TRUE)
-
- def test_expand_fail_lazy_setfield_1(self):
- ops = """
- [p1, i2, i3]
- p2 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p2, i2, descr=valuedescr)
- setfield_gc(p1, p2, descr=nextdescr)
- guard_true(i3, descr=fdescr) []
- i4 = int_neg(i2)
- setfield_gc(p1, NULL, descr=nextdescr)
- jump(p1, i2, i4)
- """
- expected = """
- [p1, i2, i3]
- guard_true(i3, descr=fdescr) [p1, i2]
- i4 = int_neg(i2)
- setfield_gc(p1, NULL, descr=nextdescr)
- jump(p1, i2, i4)
- """
- self.optimize_loop(ops, expected)
- self.loop.inputargs[0].value = self.nodebox.value
- self.check_expanded_fail_descr('''
- p1.nextdescr = p2
- where p2 is a node_vtable, valuedescr=i2
- ''', rop.GUARD_TRUE)
-
- def test_expand_fail_lazy_setfield_2(self):
- ops = """
- [i2, i3]
- p2 = new_with_vtable(ConstClass(node_vtable))
- setfield_gc(p2, i2, descr=valuedescr)
- setfield_gc(ConstPtr(myptr), p2, descr=nextdescr)
- guard_true(i3, descr=fdescr) []
- i4 = int_neg(i2)
- setfield_gc(ConstPtr(myptr), NULL, descr=nextdescr)
- jump(i2, i4)
- """
- expected = """
- [i2, i3]
- guard_true(i3, descr=fdescr) [i2]
- i4 = int_neg(i2)
- setfield_gc(ConstPtr(myptr), NULL, descr=nextdescr)
- jump(i2, i4)
- """
- self.optimize_loop(ops, expected)
- self.check_expanded_fail_descr('''
- ConstPtr(myptr).nextdescr = p2
- where p2 is a node_vtable, valuedescr=i2
- ''', rop.GUARD_TRUE)
-
-
class TestLLtype(OptimizeOptTest, LLtypeMixin):
def test_residual_call_does_not_invalidate_caches(self):
From arigo at codespeak.net Sat Oct 30 19:24:25 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 30 Oct 2010 19:24:25 +0200 (CEST)
Subject: [pypy-svn] r78563 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030172425.DE591282BE7@codespeak.net>
Author: arigo
Date: Sat Oct 30 19:24:24 2010
New Revision: 78563
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizefficall.py
Log:
Fix test_optimizefficall.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py Sat Oct 30 19:24:24 2010
@@ -226,7 +226,7 @@
_kind2count = {history.INT: 1, history.REF: 2, history.FLOAT: 3}
return sorted(boxes, key=lambda box: _kind2count[box.type])
-class BaseTestOptimizeBasic(BaseTest):
+class BaseTestBasic(BaseTest):
def invent_fail_descr(self, fail_args):
if fail_args is None:
@@ -259,12 +259,14 @@
OptVirtualize,
OptString,
OptHeap,
+ OptFfiCall,
Optimizer)
optimizations = [OptIntBounds(),
OptRewrite(),
OptVirtualize(),
OptString(),
OptHeap(),
+ OptFfiCall(),
]
optimizer = Optimizer(metainterp_sd, loop, optimizations,
not_a_bridge=True)
@@ -274,6 +276,9 @@
print '\n'.join([str(o) for o in loop.operations])
self.assert_equal(loop, expected)
+
+class BaseTestOptimizeBasic(BaseTestBasic):
+
def test_simple(self):
ops = """
[i]
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizefficall.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizefficall.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizefficall.py Sat Oct 30 19:24:24 2010
@@ -2,7 +2,8 @@
from pypy.rlib.libffi import Func, types
from pypy.jit.metainterp.history import AbstractDescr
from pypy.jit.codewriter.effectinfo import EffectInfo
-from pypy.jit.metainterp.test.test_optimizeopt import BaseTestOptimizeOpt, LLtypeMixin
+from pypy.jit.metainterp.test.test_optimizebasic import BaseTestBasic
+from pypy.jit.metainterp.test.test_optimizebasic import LLtypeMixin
class MyCallDescr(AbstractDescr):
"""
@@ -30,7 +31,7 @@
return id(self)
-class TestFfiCall(BaseTestOptimizeOpt, LLtypeMixin):
+class TestFfiCall(BaseTestBasic, LLtypeMixin):
class namespace:
cpu = LLtypeMixin.cpu
@@ -45,13 +46,15 @@
argtypes=[types.sint, types.double],
restype=types.sint)
#
- def calldescr(cpu, FUNC, oopspecindex):
- einfo = EffectInfo([], [], [], oopspecindex=oopspecindex)
+ def calldescr(cpu, FUNC, oopspecindex, extraeffect=None):
+ einfo = EffectInfo([], [], [], oopspecindex=oopspecindex,
+ extraeffect=extraeffect)
return cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, einfo)
#
libffi_prepare = calldescr(cpu, FUNC, EffectInfo.OS_LIBFFI_PREPARE)
libffi_push_arg = calldescr(cpu, FUNC, EffectInfo.OS_LIBFFI_PUSH_ARG)
- libffi_call = calldescr(cpu, FUNC, EffectInfo.OS_LIBFFI_CALL)
+ libffi_call = calldescr(cpu, FUNC, EffectInfo.OS_LIBFFI_CALL,
+ EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE)
namespace = namespace.__dict__
@@ -77,7 +80,7 @@
guard_no_exception() []
jump(i3, f1)
"""
- loop = self.optimize_loop(ops, 'Not, Not', expected)
+ loop = self.optimize_loop(ops, expected)
def test_ffi_call_nonconst(self):
ops = """
@@ -91,7 +94,7 @@
jump(i3, f1, p2)
"""
expected = ops
- loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
+ loop = self.optimize_loop(ops, expected)
def test_handle_virtualizables(self):
# this test needs an explanation to understand what goes on: see the
@@ -117,7 +120,7 @@
guard_no_exception() [p2]
jump(i3, f1, p2)
"""
- loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
+ loop = self.optimize_loop(ops, expected)
# ----------------------------------------------------------------------
# in pratice, the situations described in these tests should never happen,
@@ -136,7 +139,7 @@
jump(i3, f1)
"""
expected = ops
- loop = self.optimize_loop(ops, 'Not, Not', expected)
+ loop = self.optimize_loop(ops, expected)
def test_rollback_multiple_calls(self):
ops = """
@@ -159,7 +162,7 @@
jump(i3, i4, f1)
"""
expected = ops
- loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
+ loop = self.optimize_loop(ops, expected)
def test_rollback_multiple_prepare(self):
ops = """
@@ -182,7 +185,7 @@
jump(i3, i4, f1)
"""
expected = ops
- loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
+ loop = self.optimize_loop(ops, expected)
def test_optimize_nested_call(self):
ops = """
@@ -220,7 +223,7 @@
guard_no_exception() []
jump(i3, i4, f1)
"""
- loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
+ loop = self.optimize_loop(ops, expected)
def test_rollback_force_token(self):
ops = """
@@ -237,4 +240,4 @@
jump(i3, f1, p2)
"""
expected = ops
- loop = self.optimize_loop(ops, 'Not, Not, Not', expected)
+ loop = self.optimize_loop(ops, expected)
From arigo at codespeak.net Sat Oct 30 19:29:58 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 30 Oct 2010 19:29:58 +0200 (CEST)
Subject: [pypy-svn] r78564 - pypy/branch/jit-unroll-loops/pypy/rlib
Message-ID: <20101030172958.EA1C0282C37@codespeak.net>
Author: arigo
Date: Sat Oct 30 19:29:57 2010
New Revision: 78564
Modified:
pypy/branch/jit-unroll-loops/pypy/rlib/jit.py
Log:
Re-change the default optimizer to be OPTIMIZER_FULL.
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/jit.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/jit.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/jit.py Sat Oct 30 19:29:57 2010
@@ -260,8 +260,8 @@
'trace_eagerness': 200,
'trace_limit': 10000,
'inlining': False,
- #'optimizer': OPTIMIZER_FULL,
- 'optimizer': OPTIMIZER_NO_PERFECTSPEC,
+ 'optimizer': OPTIMIZER_FULL,
+ #'optimizer': OPTIMIZER_NO_PERFECTSPEC,
'debug' : DEBUG_STEPS,
}
unroll_parameters = unrolling_iterable(PARAMETERS.keys())
From arigo at codespeak.net Sat Oct 30 19:34:45 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sat, 30 Oct 2010 19:34:45 +0200 (CEST)
Subject: [pypy-svn] r78565 - pypy/branch/jit-unroll-loops/pypy/jit/metainterp
Message-ID: <20101030173445.D727B282C37@codespeak.net>
Author: arigo
Date: Sat Oct 30 19:34:44 2010
New Revision: 78565
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize.py
Log:
Fix? This file is really the same as optimize_nopspec right now.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize.py Sat Oct 30 19:34:44 2010
@@ -2,7 +2,7 @@
# ____________________________________________________________
-from pypy.jit.metainterp.optimizeopt import optimize_loop_1
+from pypy.jit.metainterp.optimizeopt import optimize_loop_1, optimize_bridge_1
def optimize_loop(metainterp_sd, old_loop_tokens, loop):
debug_start("jit-optimize")
@@ -33,6 +33,9 @@
cpu = metainterp_sd.cpu
metainterp_sd.logger_noopt.log_loop(bridge.inputargs, bridge.operations)
if old_loop_tokens:
+ old_loop_token = old_loop_tokens[0]
+ bridge.operations[-1].setdescr(old_loop_token) # patch jump target
+ optimize_bridge_1(metainterp_sd, bridge)
return old_loop_tokens[0]
return None
From hakanardo at codespeak.net Sat Oct 30 19:47:09 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sat, 30 Oct 2010 19:47:09 +0200 (CEST)
Subject: [pypy-svn] r78566 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101030174709.50EF0282BEA@codespeak.net>
Author: hakanardo
Date: Sat Oct 30 19:47:07 2010
New Revision: 78566
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sat Oct 30 19:47:07 2010
@@ -3082,7 +3082,7 @@
i2 = int_lt(i0, 5)
jump(i2)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_loop(ops, expected, expected)
def test_bound_lt_noopt(self):
ops = """
@@ -3093,15 +3093,19 @@
guard_true(i2) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_lt(i0, 4)
guard_false(i1) []
i2 = int_lt(i0, 5)
guard_true(i2) []
- jump(4)
+ jump()
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ []
+ jump()
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_lt_rev(self):
ops = """
@@ -3112,13 +3116,17 @@
guard_true(i2) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_lt(i0, 4)
guard_false(i1) []
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_lt_tripple(self):
ops = """
@@ -3131,13 +3139,17 @@
guard_true(i3) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_lt(i0, 0)
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_lt_add(self):
ops = """
@@ -3149,14 +3161,18 @@
guard_true(i3) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_lt(i0, 4)
guard_true(i1) []
i2 = int_add(i0, 10)
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_lt_add_before(self):
ops = """
@@ -3168,14 +3184,18 @@
guard_true(i1) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i2 = int_add(i0, 10)
i3 = int_lt(i2, 15)
guard_true(i3) []
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_lt_add_ovf(self):
ops = """
@@ -3188,14 +3208,18 @@
guard_true(i3) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_lt(i0, 4)
guard_true(i1) []
i2 = int_add(i0, 10)
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_lt_add_ovf_before(self):
ops = """
From afa at codespeak.net Sat Oct 30 20:32:02 2010
From: afa at codespeak.net (afa at codespeak.net)
Date: Sat, 30 Oct 2010 20:32:02 +0200 (CEST)
Subject: [pypy-svn] r78567 - in
pypy/branch/fast-forward/pypy/module/_hashlib: . test
Message-ID: <20101030183202.BC8BE282BAD@codespeak.net>
Author: afa
Date: Sat Oct 30 20:32:01 2010
New Revision: 78567
Modified:
pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py
pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py
Log:
Note to self: a function definition in a loop is probably an error!
all functions capture the same variale, and so get the
last value of the list!
Modified: pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py (original)
+++ pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py Sat Oct 30 20:32:01 2010
@@ -124,10 +124,13 @@
return space.wrap(w_hash)
# shortcut functions
-for name in algorithms:
- newname = 'new_%s' % (name,)
+def make_new_hash(name):
@func_renamer(newname)
@unwrap_spec(ObjSpace, str)
def new_hash(space, string=''):
return new(space, name, string)
- globals()[newname] = new_hash
+ return new_hash
+
+for name in algorithms:
+ newname = 'new_%s' % (name,)
+ globals()[newname] = make_new_hash(name)
Modified: pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py (original)
+++ pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py Sat Oct 30 20:32:01 2010
@@ -57,6 +57,10 @@
assert c_digest == digest
assert c_hexdigest == hexdigest
+ def test_shortcut(self):
+ import hashlib
+ assert repr(hashlib.md5()).startswith("
Author: afa
Date: Sat Oct 30 20:36:22 2010
New Revision: 78568
Modified:
pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py
Log:
Rename each function with a different name,
thanks Alex!
Modified: pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py
==============================================================================
--- pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py (original)
+++ pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py Sat Oct 30 20:36:22 2010
@@ -124,8 +124,8 @@
return space.wrap(w_hash)
# shortcut functions
-def make_new_hash(name):
- @func_renamer(newname)
+def make_new_hash(name, funcname):
+ @func_renamer(funcname)
@unwrap_spec(ObjSpace, str)
def new_hash(space, string=''):
return new(space, name, string)
@@ -133,4 +133,4 @@
for name in algorithms:
newname = 'new_%s' % (name,)
- globals()[newname] = make_new_hash(name)
+ globals()[newname] = make_new_hash(name, newname)
From david at codespeak.net Sat Oct 30 21:20:54 2010
From: david at codespeak.net (david at codespeak.net)
Date: Sat, 30 Oct 2010 21:20:54 +0200 (CEST)
Subject: [pypy-svn] r78569 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101030192054.B8811282BE7@codespeak.net>
Author: david
Date: Sat Oct 30 21:20:51 2010
New Revision: 78569
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
Log:
Improve and correct jumps from one codeblock to the next one allocated to fit isntructions
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Sat Oct 30 21:20:51 2010
@@ -15,6 +15,7 @@
def f(self, cond=cond.AL):
"""Generates a call to a helper function, takes its
arguments in r0 and r1, result is placed in r0"""
+ self.ensure_can_fit(self.size_of_gen_load_int*2+3*WORD)
self.PUSH(range(2, 12), cond=cond)
addr = rffi.cast(lltype.Signed, llhelper(signature, function))
self.gen_load_int(reg.r2.value, addr, cond=cond)
@@ -35,6 +36,9 @@
f.write(self._data[i])
f.close()
+ def ensure_can_fit(self, n):
+ raise NotImplentedError
+
def PUSH(self, regs, cond=cond.AL):
assert reg.sp not in regs
instr = self._encode_reg_list(cond << 28 | 0x92D << 16, regs)
@@ -109,6 +113,7 @@
data = rffi.cast(PTR, start)
self._init(data, map_size)
+
class ARMv7Builder(AbstractARMv7Builder):
def __init__(self):
@@ -119,7 +124,7 @@
self.checks = True
self.n_data=0
- _space_for_jump = 9 * WORD
+ _space_for_jump = 2 * WORD
def writechar(self, char):
if self.checks and not self._pos < self._size - self._space_for_jump:
self.checks = False
@@ -131,13 +136,17 @@
def _add_more_mem(self):
new_mem = alloc(self._size)
new_mem_addr = rffi.cast(lltype.Signed, new_mem)
- self.PUSH([reg.ip.value, reg.lr.value])
- self.gen_load_int(reg.lr.value, new_mem_addr)
- self.MOV_rr(reg.pc.value, reg.lr.value)
+ self.LDR_ri(reg.pc.value, reg.pc.value, -4)
+ self.write32(new_mem_addr)
self._dump_trace('data%d.asm' % self.n_data)
self.n_data += 1
self._data = new_mem
self._pos = 0
- self.LDM(reg.sp.value, [reg.ip.value, reg.lr.value], w=1) # XXX Replace with POP instr. someday
+
+ def ensure_can_fit(self, n):
+ """ensure after this call there is enough space for n instructions
+ in a contiguous memory chunk"""
+ if not self._pos + n + self._space_for_jump < self._size:
+ self._add_more_mem()
define_instructions(AbstractARMv7Builder)
From david at codespeak.net Sat Oct 30 23:02:31 2010
From: david at codespeak.net (david at codespeak.net)
Date: Sat, 30 Oct 2010 23:02:31 +0200 (CEST)
Subject: [pypy-svn] r78570 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101030210231.A99E4282B90@codespeak.net>
Author: david
Date: Sat Oct 30 23:02:28 2010
New Revision: 78570
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
pypy/branch/arm-backend/pypy/jit/backend/arm/conditions.py
pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py
Log:
Implement int and uint comparison operations
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Sat Oct 30 23:02:28 2010
@@ -16,12 +16,12 @@
"""Generates a call to a helper function, takes its
arguments in r0 and r1, result is placed in r0"""
self.ensure_can_fit(self.size_of_gen_load_int*2+3*WORD)
- self.PUSH(range(2, 12), cond=cond)
+ self.PUSH(range(2, 4), cond=cond)
addr = rffi.cast(lltype.Signed, llhelper(signature, function))
self.gen_load_int(reg.r2.value, addr, cond=cond)
self.gen_load_int(reg.lr.value, self.curraddr()+self.size_of_gen_load_int+WORD, cond=cond)
self.MOV_rr(reg.pc.value, reg.r2.value, cond=cond)
- self.LDM(reg.sp.value, range(2, 12), w=1, cond=cond) # XXX Replace with POP instr. someday
+ self.LDM(reg.sp.value, range(2, 4), w=1, cond=cond) # XXX Replace with POP instr. someday
return f
class AbstractARMv7Builder(object):
@@ -49,15 +49,6 @@
instr = self._encode_reg_list(instr, regs)
self.write32(instr)
- def CMP(self, rn, imm, cond=cond.AL):
- if 0 <= imm <= 255:
- self.write32(cond << 28
- | 0x35 << 20
- | (rn & 0xFF) << 16
- | (imm & 0xFFF))
- else:
- raise NotImplentedError
-
def BKPT(self, cond=cond.AL):
self.write32(cond << 28 | 0x1200070)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/conditions.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/conditions.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/conditions.py Sat Oct 30 23:02:28 2010
@@ -13,3 +13,8 @@
GT = 0xC
LE = 0xD
AL = 0xE
+
+opposites = [NE, EQ, CC, CS, PL, MI, VC, VS, LS, HI, LT, GE, LE, GT, AL]
+def get_opposite_of(operation):
+ return opposites[operation]
+
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Sat Oct 30 23:02:28 2010
@@ -52,6 +52,32 @@
return fcond
return f
+def gen_emit_cmp_op(condition, inverse=False):
+ def f(self, op, regalloc, fcond):
+ assert fcond == c.AL
+ if not inverse:
+ arg0 = op.getarg(0)
+ arg1 = op.getarg(1)
+ else:
+ arg0 = op.getarg(1)
+ arg1 = op.getarg(0)
+ res = regalloc.try_allocate_reg(op.result)
+ # XXX consider swapping argumentes if arg0 is const
+ if self._check_imm_arg(arg1) and not isinstance(arg0, ConstInt):
+ reg = regalloc.try_allocate_reg(arg0)
+ self.mc.CMP_ri(reg.value, imm=arg1.getint(), cond=fcond)
+ else:
+ reg = self._put_in_reg(arg0, regalloc)
+ reg2 = self._put_in_reg(arg1, regalloc)
+ self.mc.CMP_rr(reg.value, reg2.value)
+ regalloc.possibly_free_var(reg2)
+
+ inv = c.get_opposite_of(condition)
+ self.mc.MOV_ri(res.value, 1, cond=condition)
+ self.mc.MOV_ri(res.value, 0, cond=inv)
+ return condition
+ return f
+
class IntOpAsslember(object):
_mixin_ = True
@@ -137,8 +163,21 @@
emit_op_int_rshift = gen_emit_op_ri('ASR', imm_size=0x1F, commutative=False)
emit_op_uint_rshift = gen_emit_op_ri('LSR', imm_size=0x1F, commutative=False)
+ emit_op_int_lt = gen_emit_cmp_op(c.LT)
+ emit_op_int_le = gen_emit_cmp_op(c.LE)
+ emit_op_int_eq = gen_emit_cmp_op(c.EQ)
+ emit_op_int_ne = gen_emit_cmp_op(c.NE)
+ emit_op_int_gt = gen_emit_cmp_op(c.GT)
+ emit_op_int_ge = gen_emit_cmp_op(c.GE)
- def _check_imm_arg(self, arg, size):
+ emit_op_uint_le = gen_emit_cmp_op(c.LS)
+ emit_op_uint_gt = gen_emit_cmp_op(c.HI)
+
+ emit_op_uint_lt = gen_emit_cmp_op(c.HI, inverse=True)
+ emit_op_uint_ge = gen_emit_cmp_op(c.LS, inverse=True)
+
+
+ def _check_imm_arg(self, arg, size=0xFF):
#XXX check ranges for different operations
return isinstance(arg, ConstInt) and arg.getint() <= size and arg.getint() > 0
@@ -155,8 +194,10 @@
descr._arm_guard_cond = fcond
def emit_op_guard_true(self, op, regalloc, fcond):
- assert fcond == c.GT
- self._emit_guard(op, regalloc, fcond)
+ assert fcond == c.LE
+ cond = c.get_opposite_of(fcond)
+ assert cond == c.GT
+ self._emit_guard(op, regalloc, cond)
return c.AL
def emit_op_guard_false(self, op, regalloc, fcond):
@@ -183,17 +224,5 @@
return fcond
def emit_op_finish(self, op, regalloc, fcond):
- self._gen_path_to_exit_path(op, op.getarglist(), regalloc, fcond)
+ self._gen_path_to_exit_path(op, op.getarglist(), regalloc, c.AL)
return fcond
-
- def emit_op_int_le(self, op, regalloc, fcond):
- reg = regalloc.try_allocate_reg(op.getarg(0))
- assert isinstance(op.getarg(1), ConstInt)
- self.mc.CMP(reg.value, op.getarg(1).getint())
- return c.GT
-
- def emit_op_int_eq(self, op, regalloc, fcond):
- reg = regalloc.try_allocate_reg(op.getarg(0))
- assert isinstance(op.getarg(1), ConstInt)
- self.mc.CMP(reg.value, op.getarg(1).getint())
- return c.EQ
From david at codespeak.net Sat Oct 30 23:10:11 2010
From: david at codespeak.net (david at codespeak.net)
Date: Sat, 30 Oct 2010 23:10:11 +0200 (CEST)
Subject: [pypy-svn] r78571 -
pypy/branch/arm-backend/pypy/jit/backend/arm/test
Message-ID: <20101030211011.56AD436E0C1@codespeak.net>
Author: david
Date: Sat Oct 30 23:10:09 2010
New Revision: 78571
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
Log:
Update tests after removing CMP method
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py Sat Oct 30 23:10:09 2010
@@ -69,7 +69,7 @@
def test_cmp(self):
self.a.gen_func_prolog()
self.a.mc.gen_load_int(r.r1.value, 22)
- self.a.mc.CMP(r.r1.value, 123)
+ self.a.mc.CMP_ri(r.r1.value, 123)
self.a.mc.MOV_ri(r.r0.value, 1, c.LE)
self.a.mc.MOV_ri(r.r0.value, 0, c.GT)
self.a.gen_func_epilog()
@@ -78,7 +78,7 @@
def test_int_le_false(self):
self.a.gen_func_prolog()
self.a.mc.gen_load_int(r.r1.value, 2222)
- self.a.mc.CMP(r.r1.value, 123)
+ self.a.mc.CMP_ri(r.r1.value, 123)
self.a.mc.MOV_ri(r.r0.value, 1, c.LE)
self.a.mc.MOV_ri(r.r0.value, 0, c.GT)
self.a.gen_func_epilog()
@@ -88,7 +88,7 @@
self.a.gen_func_prolog()
self.a.mc.MOV_ri(r.r1.value, 1)
loop_head = self.a.mc.curraddr()
- self.a.mc.CMP(r.r1.value, 0) # z=0, z=1
+ self.a.mc.CMP_ri(r.r1.value, 0) # z=0, z=1
self.a.mc.MOV_ri(r.r1.value, 0, cond=c.NE)
self.a.mc.MOV_ri(r.r1.value, 7, cond=c.EQ)
self.a.mc.gen_load_int(r.r4.value, loop_head, cond=c.NE)
@@ -102,7 +102,7 @@
self.a.mc.MOV_ri(r.r1.value, 1)
loop_head = self.a.mc.curraddr()
self.a.mc.ADD_ri(r.r1.value, r.r1.value, 1)
- self.a.mc.CMP(r.r1.value, 9)
+ self.a.mc.CMP_ri(r.r1.value, 9)
self.a.mc.gen_load_int(r.r4.value, loop_head, cond=c.NE)
self.a.mc.MOV_rr(r.pc.value, r.r4.value, cond=c.NE)
self.a.mc.MOV_rr(r.r0.value, r.r1.value)
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Sat Oct 30 23:10:09 2010
@@ -113,7 +113,7 @@
self.assert_equal('SUB r3, r7, #4095')
def test_cmp_ri(self):
- self.cb.CMP(r.r3.value, 123)
+ self.cb.CMP_ri(r.r3.value, 123)
self.assert_equal('CMP r3, #123')
def test_mcr(self):
From arigo at codespeak.net Sun Oct 31 10:54:36 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 10:54:36 +0100 (CET)
Subject: [pypy-svn] r78580 - pypy/trunk/lib-python/modified-2.5.2/test
Message-ID: <20101031095436.77C96282B90@codespeak.net>
Author: arigo
Date: Sun Oct 31 10:54:33 2010
New Revision: 78580
Modified:
pypy/trunk/lib-python/modified-2.5.2/test/mapping_tests.py
Log:
Mark this as an implementation detail. This is a copy of test_dict. :-(
Modified: pypy/trunk/lib-python/modified-2.5.2/test/mapping_tests.py
==============================================================================
--- pypy/trunk/lib-python/modified-2.5.2/test/mapping_tests.py (original)
+++ pypy/trunk/lib-python/modified-2.5.2/test/mapping_tests.py Sun Oct 31 10:54:33 2010
@@ -1,6 +1,7 @@
# tests common to dict and UserDict
import unittest
import UserDict
+from test import test_support
class BasicTestMappingProtocol(unittest.TestCase):
@@ -525,7 +526,8 @@
self.assertEqual(va, int(ka))
kb, vb = tb = b.popitem()
self.assertEqual(vb, int(kb))
- self.assert_(not(copymode < 0 and ta != tb))
+ if test_support.check_impl_detail():
+ self.assert_(not(copymode < 0 and ta != tb))
self.assert_(not a)
self.assert_(not b)
From arigo at codespeak.net Sun Oct 31 10:58:13 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 10:58:13 +0100 (CET)
Subject: [pypy-svn] r78581 - pypy/branch/smalllong
Message-ID: <20101031095813.DCEC7282BDA@codespeak.net>
Author: arigo
Date: Sun Oct 31 10:58:12 2010
New Revision: 78581
Added:
pypy/branch/smalllong/
- copied from r78580, pypy/trunk/
Log:
Implement "small longs", i.e. longs implemented as an r_longlong or
as two r_longs (undecided so far).
From hakanardo at codespeak.net Sun Oct 31 11:11:57 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sun, 31 Oct 2010 11:11:57 +0100 (CET)
Subject: [pypy-svn] r78582 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test
Message-ID: <20101031101157.5EFBB282BDD@codespeak.net>
Author: hakanardo
Date: Sun Oct 31 11:11:55 2010
New Revision: 78582
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
fixed bound tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Sun Oct 31 11:11:55 2010
@@ -78,7 +78,7 @@
args.append(self.getvalue(arg).force_box())
newop.initarglist(args + inputargs[len(args):])
- #print 'P: ', str(newop)
+ print 'P: ', str(newop)
self.emit_operation(newop)
jmp = self.optimizer.newoperations[-1]
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sun Oct 31 11:11:55 2010
@@ -3232,7 +3232,7 @@
guard_true(i1) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i2 = int_add_ovf(i0, 10)
guard_no_overflow() []
@@ -3240,7 +3240,12 @@
guard_true(i3) []
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ i2 = int_add(i0, 10)
+ jump(i0)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_lt_sub(self):
ops = """
@@ -3252,14 +3257,18 @@
guard_true(i3) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_lt(i0, 4)
guard_true(i1) []
i2 = int_sub(i0, 10)
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_lt_sub_before(self):
ops = """
@@ -3271,14 +3280,18 @@
guard_true(i1) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i2 = int_sub(i0, 10)
i3 = int_lt(i2, -5)
guard_true(i3) []
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_ltle(self):
ops = """
@@ -3289,13 +3302,17 @@
guard_true(i2) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_lt(i0, 4)
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_lelt(self):
ops = """
@@ -3306,13 +3323,17 @@
guard_true(i2) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_le(i0, 4)
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_gt(self):
ops = """
@@ -3323,13 +3344,17 @@
guard_true(i2) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_gt(i0, 5)
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_gtge(self):
ops = """
@@ -3340,13 +3365,17 @@
guard_true(i2) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_gt(i0, 5)
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_gegt(self):
ops = """
@@ -3357,13 +3386,17 @@
guard_true(i2) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_ge(i0, 5)
guard_true(i1) []
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_ovf(self):
ops = """
@@ -3376,7 +3409,7 @@
guard_no_overflow() []
jump(i3)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_ge(i0, 0)
guard_true(i1) []
@@ -3385,7 +3418,14 @@
i3 = int_add(i0, 1)
jump(i3)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ i2 = int_lt(i0, 10)
+ guard_true(i2) []
+ i3 = int_add(i0, 1)
+ jump(i3)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_arraylen(self):
ops = """
@@ -3712,14 +3752,7 @@
i2 = int_add(i0, 3)
jump(i2)
"""
- expected = """
- [i0]
- i1 = int_ne(i0, 7)
- guard_false(i1) []
- jump(10)
-
- """
- self.optimize_loop(ops, expected)
+ py.test.raises(InvalidLoop, self.optimize_loop, ops, ops)
def test_bound_ne_const_not(self):
ops = """
@@ -3747,13 +3780,17 @@
guard_true(i2) []
jump(i0, i1)
"""
- expected = """
+ preamble = """
[i0, i1]
i2 = int_lt(i0, 7)
guard_true(i2) []
jump(i0, i1)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0, i1]
+ jump(i0, i1)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_lege_const(self):
ops = """
@@ -3765,16 +3802,7 @@
i3 = int_add(i0, 3)
jump(i3)
"""
- expected = """
- [i0]
- i1 = int_ge(i0, 7)
- guard_true(i1) []
- i2 = int_le(i0, 7)
- guard_true(i2) []
- jump(10)
-
- """
- self.optimize_loop(ops, expected)
+ py.test.raises(InvalidLoop, self.optimize_loop, ops, ops)
def test_mul_ovf(self):
ops = """
From hakanardo at codespeak.net Sun Oct 31 11:18:50 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sun, 31 Oct 2010 11:18:50 +0100 (CET)
Subject: [pypy-svn] r78583 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test
Message-ID: <20101031101850.61F625080B@codespeak.net>
Author: hakanardo
Date: Sun Oct 31 11:18:48 2010
New Revision: 78583
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(armin, cfbolz, hakanardo) Dont invalidate cached on set*_raw
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py Sun Oct 31 11:18:48 2010
@@ -107,7 +107,9 @@
return
opnum = op.getopnum()
if (opnum == rop.SETFIELD_GC or
+ opnum == rop.SETFIELD_RAW or
opnum == rop.SETARRAYITEM_GC or
+ opnum == rop.SETARRAYITEM_RAW or
opnum == rop.DEBUG_MERGE_POINT):
return
assert opnum != rop.CALL_PURE
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Sun Oct 31 11:18:48 2010
@@ -78,7 +78,7 @@
args.append(self.getvalue(arg).force_box())
newop.initarglist(args + inputargs[len(args):])
- print 'P: ', str(newop)
+ #print 'P: ', str(newop)
self.emit_operation(newop)
jmp = self.optimizer.newoperations[-1]
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sun Oct 31 11:18:48 2010
@@ -3552,12 +3552,14 @@
jump(p4, p7, i30)
"""
expected = """
- [p4, p7, i30, i2]
- i33 = int_add(i2, i30)
- setfield_raw(p7, i33, descr=nextdescr)
- jump(p4, p7, i30, i33)
+ [p4, p7, i30]
+ i1 = getfield_raw(p7, descr=nextdescr)
+ i2 = int_add(i1, i30)
+ setfield_raw(p7, 7, descr=nextdescr)
+ setfield_raw(p7, i2, descr=nextdescr)
+ jump(p4, p7, i30)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_loop(ops, expected, ops)
def test_pure(self):
ops = """
From hakanardo at codespeak.net Sun Oct 31 11:29:03 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sun, 31 Oct 2010 11:29:03 +0100 (CET)
Subject: [pypy-svn] r78584 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031102903.086F6282B90@codespeak.net>
Author: hakanardo
Date: Sun Oct 31 11:29:02 2010
New Revision: 78584
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(cfbolz, hakanardo) Test that caches are not invalidated on setarrayitem_raw
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sun Oct 31 11:29:02 2010
@@ -3544,6 +3544,7 @@
ops = """
[p4, p7, i30]
p16 = getfield_gc(p4, descr=valuedescr)
+ p17 = getarrayitem_gc(p4, 1, descr=arraydescr)
guard_value(p16, ConstPtr(myptr), descr=) []
i1 = getfield_raw(p7, descr=nextdescr)
i2 = int_add(i1, i30)
@@ -3561,6 +3562,28 @@
"""
self.optimize_loop(ops, expected, ops)
+ def test_setgetarrayitem_raw(self):
+ ops = """
+ [p4, p7, i30]
+ p16 = getfield_gc(p4, descr=valuedescr)
+ guard_value(p16, ConstPtr(myptr), descr=) []
+ p17 = getarrayitem_gc(p4, 1, descr=arraydescr)
+ i1 = getarrayitem_raw(p7, 1, descr=arraydescr)
+ i2 = int_add(i1, i30)
+ setarrayitem_raw(p7, 1, 7, descr=arraydescr)
+ setarrayitem_raw(p7, 1, i2, descr=arraydescr)
+ jump(p4, p7, i30)
+ """
+ expected = """
+ [p4, p7, i30]
+ i1 = getarrayitem_raw(p7, 1, descr=arraydescr)
+ i2 = int_add(i1, i30)
+ setarrayitem_raw(p7, 1, 7, descr=arraydescr)
+ setarrayitem_raw(p7, 1, i2, descr=arraydescr)
+ jump(p4, p7, i30)
+ """
+ self.optimize_loop(ops, expected, ops)
+
def test_pure(self):
ops = """
[p42]
@@ -3672,7 +3695,7 @@
guard_true(i4) []
jump(i0)
"""
- expected = """
+ preamble = """
[i0]
i1 = int_sub_ovf(1, i0)
guard_no_overflow() []
@@ -3681,7 +3704,11 @@
i3 = int_sub(1, i0)
jump(i0)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0]
+ jump(i0)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_bound_eq(self):
ops = """
From arigo at codespeak.net Sun Oct 31 11:30:02 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 11:30:02 +0100 (CET)
Subject: [pypy-svn] r78585 - in pypy/branch/smalllong/pypy: config
objspace/std objspace/std/test
Message-ID: <20101031103002.56848282B90@codespeak.net>
Author: arigo
Date: Sun Oct 31 11:30:00 2010
New Revision: 78585
Added:
pypy/branch/smalllong/pypy/objspace/std/smalllongobject.py
pypy/branch/smalllong/pypy/objspace/std/test/test_smalllongobject.py
Modified:
pypy/branch/smalllong/pypy/config/pypyoption.py
pypy/branch/smalllong/pypy/objspace/std/longobject.py
pypy/branch/smalllong/pypy/objspace/std/longtype.py
pypy/branch/smalllong/pypy/objspace/std/objspace.py
pypy/branch/smalllong/pypy/objspace/std/strutil.py
Log:
In-progress.
Modified: pypy/branch/smalllong/pypy/config/pypyoption.py
==============================================================================
--- pypy/branch/smalllong/pypy/config/pypyoption.py (original)
+++ pypy/branch/smalllong/pypy/config/pypyoption.py Sun Oct 31 11:30:00 2010
@@ -194,6 +194,9 @@
IntOption("prebuiltintto", "highest integer which is prebuilt",
default=100, cmdline="--prebuiltintto"),
+ BoolOption("withsmalllong", "use a version of 'long' in a C long long",
+ default=False),
+
BoolOption("withstrjoin", "use strings optimized for addition",
default=False),
Modified: pypy/branch/smalllong/pypy/objspace/std/longobject.py
==============================================================================
--- pypy/branch/smalllong/pypy/objspace/std/longobject.py (original)
+++ pypy/branch/smalllong/pypy/objspace/std/longobject.py Sun Oct 31 11:30:00 2010
@@ -60,6 +60,12 @@
registerimplementation(W_LongObject)
+def newbigint(space, w_longtype, bigint):
+ w_obj = space.allocate_instance(W_LongObject, w_longtype)
+ W_LongObject.__init__(w_obj, bigint)
+ return w_obj
+
+
# bool-to-long
def delegate_Bool2Long(space, w_bool):
return W_LongObject(rbigint.frombool(space.is_true(w_bool)))
Modified: pypy/branch/smalllong/pypy/objspace/std/longtype.py
==============================================================================
--- pypy/branch/smalllong/pypy/objspace/std/longtype.py (original)
+++ pypy/branch/smalllong/pypy/objspace/std/longtype.py Sun Oct 31 11:30:00 2010
@@ -4,45 +4,48 @@
from pypy.objspace.std.strutil import string_to_bigint, ParseStringError
def descr__new__(space, w_longtype, w_x=0, w_base=gateway.NoneNotWrapped):
- from pypy.objspace.std.longobject import W_LongObject
+ from pypy.objspace.std.longobject import W_LongObject, newbigint
+ from pypy.rlib.rbigint import rbigint
+ if space.config.objspace.std.withsmalllong:
+ from pypy.objspace.std.smalllongobject import W_SmallLongObject
+ else:
+ W_SmallLongObject = None
w_value = w_x # 'x' is the keyword argument name in CPython
if w_base is None:
# check for easy cases
- if type(w_value) is W_LongObject:
- bigint = w_value.num
+ if (W_SmallLongObject and type(w_value) is W_SmallLongObject
+ and space.is_w(w_longtype, space.w_long)):
+ return w_value
+ elif type(w_value) is W_LongObject:
+ return newbigint(space, w_longtype, w_value.num)
elif space.is_true(space.isinstance(w_value, space.w_str)):
- try:
- bigint = string_to_bigint(space.str_w(w_value))
- except ParseStringError, e:
- raise OperationError(space.w_ValueError,
- space.wrap(e.msg))
+ return string_to_w_long(space, w_longtype, space.str_w(w_value))
elif space.is_true(space.isinstance(w_value, space.w_unicode)):
- try:
- if space.config.objspace.std.withropeunicode:
- from pypy.objspace.std.ropeunicodeobject import unicode_to_decimal_w
- else:
- from pypy.objspace.std.unicodeobject import unicode_to_decimal_w
- bigint = string_to_bigint(unicode_to_decimal_w(space, w_value))
- except ParseStringError, e:
- raise OperationError(space.w_ValueError,
- space.wrap(e.msg))
+ if space.config.objspace.std.withropeunicode:
+ from pypy.objspace.std.ropeunicodeobject import unicode_to_decimal_w
+ else:
+ from pypy.objspace.std.unicodeobject import unicode_to_decimal_w
+ return string_to_w_long(space, w_longtype,
+ unicode_to_decimal_w(space, w_value))
else:
# otherwise, use the __long__() method
w_obj = space.long(w_value)
# 'long(x)' should return whatever x.__long__() returned
if space.is_w(w_longtype, space.w_long):
return w_obj
- if space.is_true(space.isinstance(w_obj, space.w_long)):
- assert isinstance(w_obj, W_LongObject) # XXX this could fail!
- # XXX find a way to do that even if w_obj is not a W_LongObject
+ # the following is all for the 'subclass_of_long(x)' case
+ if W_SmallLongObject and isinstance(w_obj, W_SmallLongObject):
+ bigint = w_obj.as_bigint()
+ elif isinstance(w_obj, W_LongObject):
bigint = w_obj.num
elif space.is_true(space.isinstance(w_obj, space.w_int)):
from pypy.rlib.rbigint import rbigint
- intval = space.int_w(w_obj)
- bigint = rbigint.fromint(intval)
+ bigint = rbigint.fromint(space.int_w(w_obj))
else:
raise OperationError(space.w_ValueError,
- space.wrap("value can't be converted to long"))
+ space.wrap("value can't be converted to long"))
+ return newbigint(space, w_longtype, bigint)
+ #
else:
base = space.int_w(w_base)
@@ -56,15 +59,25 @@
raise OperationError(space.w_TypeError,
space.wrap("long() can't convert non-string "
"with explicit base"))
+ return string_to_w_long(space, w_longtype, s, base)
+
+
+def string_to_w_long(space, w_longtype, s, base=10):
+ try:
+ bigint = string_to_bigint(s, base)
+ except ParseStringError, e:
+ raise OperationError(space.w_ValueError,
+ space.wrap(e.msg))
+ if (space.config.objspace.std.withsmalllong
+ and space.is_w(w_longtype, space.w_long)):
try:
- bigint = string_to_bigint(s, base)
- except ParseStringError, e:
- raise OperationError(space.w_ValueError,
- space.wrap(e.msg))
-
- w_obj = space.allocate_instance(W_LongObject, w_longtype)
- W_LongObject.__init__(w_obj, bigint)
- return w_obj
+ longlong = bigint.tolonglong()
+ except OverflowError:
+ pass
+ else:
+ from pypy.objspace.std.smalllongobject import W_SmallLongObject
+ return W_SmallLongObject(longlong)
+ return newbigint(space, w_longtype, bigint)
# ____________________________________________________________
Modified: pypy/branch/smalllong/pypy/objspace/std/objspace.py
==============================================================================
--- pypy/branch/smalllong/pypy/objspace/std/objspace.py (original)
+++ pypy/branch/smalllong/pypy/objspace/std/objspace.py Sun Oct 31 11:30:00 2010
@@ -176,6 +176,7 @@
#print 'wrapping', x, '->', w_result
return w_result
if isinstance(x, base_int):
+ xxxxxxxxx
return W_LongObject.fromrarith_int(x)
# _____ below here is where the annotator should not get _____
@@ -198,6 +199,16 @@
# The following cases are even stranger.
# Really really only for tests.
if type(x) is long:
+ if self.config.objspace.std.withsmalllong:
+ from pypy.rlib.rarithmetic import r_longlong
+ try:
+ rx = r_longlong(x)
+ except OverflowError:
+ pass
+ else:
+ from pypy.objspace.std.smalllongobject import \
+ W_SmallLongObject
+ return W_SmallLongObject(rx)
return W_LongObject.fromlong(x)
if isinstance(x, slice):
return W_SliceObject(self.wrap(x.start),
@@ -261,6 +272,9 @@
return W_ComplexObject(realval, imagval)
def newlong(self, val): # val is an int
+ if self.config.objspace.std.withsmalllong:
+ from pypy.objspace.std.smalllongobject import W_SmallLongObject
+ return W_SmallLongObject.fromint(self, val)
return W_LongObject.fromint(self, val)
def newtuple(self, list_w):
Added: pypy/branch/smalllong/pypy/objspace/std/smalllongobject.py
==============================================================================
--- (empty file)
+++ pypy/branch/smalllong/pypy/objspace/std/smalllongobject.py Sun Oct 31 11:30:00 2010
@@ -0,0 +1,19 @@
+"""
+Implementation of 'small' longs, stored as a C 'long long' value.
+Useful for 32-bit applications manipulating 64-bit values.
+"""
+from pypy.objspace.std.model import registerimplementation, W_Object
+from pypy.rlib.rarithmetic import r_longlong
+
+
+class W_SmallLongObject(W_Object):
+ from pypy.objspace.std.longtype import long_typedef as typedef
+
+ def __init__(w_self, value):
+ assert isinstance(value, r_longlong)
+ w_self._longlong = value
+
+ def as_bigint(w_self):
+ xxx
+
+registerimplementation(W_SmallLongObject)
Modified: pypy/branch/smalllong/pypy/objspace/std/strutil.py
==============================================================================
--- pypy/branch/smalllong/pypy/objspace/std/strutil.py (original)
+++ pypy/branch/smalllong/pypy/objspace/std/strutil.py Sun Oct 31 11:30:00 2010
@@ -3,7 +3,7 @@
"""
from pypy.rlib.rarithmetic import ovfcheck, break_up_float, parts_to_float,\
- INFINITY, NAN
+ INFINITY, NAN, r_longlong
from pypy.rlib.rbigint import rbigint, parse_digit_string
from pypy.interpreter.error import OperationError
import math
Added: pypy/branch/smalllong/pypy/objspace/std/test/test_smalllongobject.py
==============================================================================
--- (empty file)
+++ pypy/branch/smalllong/pypy/objspace/std/test/test_smalllongobject.py Sun Oct 31 11:30:00 2010
@@ -0,0 +1,12 @@
+from pypy.objspace.std.test import test_longobject
+from pypy.conftest import gettestobjspace
+
+
+class AppTestSmallLong(test_longobject.AppTestLong):
+ def setup_class(cls):
+ cls.space = gettestobjspace(**{"objspace.std.withsmalllong": True})
+
+ def test_sl_simple(self):
+ import __pypy__
+ s = __pypy__.internal_repr(5L)
+ assert 'SmallLong' in s
From hakanardo at codespeak.net Sun Oct 31 11:41:16 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sun, 31 Oct 2010 11:41:16 +0100 (CET)
Subject: [pypy-svn] r78586 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031104116.4ED4A282B9D@codespeak.net>
Author: hakanardo
Date: Sun Oct 31 11:41:14 2010
New Revision: 78586
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sun Oct 31 11:41:14 2010
@@ -3851,7 +3851,7 @@
guard_true(i8) []
jump(i0, i1)
"""
- expected = """
+ preamble = """
[i0, i1]
i2 = int_and(i0, 255)
i3 = int_lt(i1, 5)
@@ -3863,7 +3863,11 @@
guard_true(i8) []
jump(i0, i1)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [i0, i1]
+ jump(i0, i1)
+ """
+ self.optimize_loop(ops, expected, preamble)
def test_mul_ovf_before(self):
ops = """
@@ -3880,7 +3884,7 @@
guard_false(i6) []
jump(i0, i1)
"""
- expected = """
+ preamble = """
[i0, i1]
i2 = int_and(i0, 255)
i22 = int_add(i2, 1)
@@ -3890,9 +3894,18 @@
guard_true(i4) []
i5 = int_gt(i3, 2)
guard_true(i5) []
- jump(i0, i1)
+ jump(i0, i1, i22)
+ """
+ expected = """
+ [i0, i1, i22]
+ i3 = int_mul(i22, i1)
+ i4 = int_lt(i3, 10)
+ guard_true(i4) []
+ i5 = int_gt(i3, 2)
+ guard_true(i5) []
+ jump(i0, i1, i22)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_loop(ops, expected, preamble)
def test_sub_ovf_before(self):
ops = """
@@ -3910,7 +3923,7 @@
guard_false(i7) []
jump(i0, i1)
"""
- expected = """
+ preamble = """
[i0, i1]
i2 = int_and(i0, 255)
i3 = int_sub_ovf(i2, i1)
@@ -3919,9 +3932,18 @@
guard_true(i4) []
i5 = int_ge(i3, 2)
guard_true(i5) []
- jump(i0, i1)
+ jump(i0, i1, i2)
+ """
+ expected = """
+ [i0, i1, i2]
+ i3 = int_sub(i2, i1)
+ i4 = int_le(i3, 10)
+ guard_true(i4) []
+ i5 = int_ge(i3, 2)
+ guard_true(i5) []
+ jump(i0, i1, i2)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_loop(ops, expected, preamble)
# ----------
def optimize_strunicode_loop(self, ops, optops):
From cfbolz at codespeak.net Sun Oct 31 11:43:28 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sun, 31 Oct 2010 11:43:28 +0100 (CET)
Subject: [pypy-svn] r78587 -
pypy/branch/mapdict-without-jit/pypy/objspace/std
Message-ID: <20101031104328.E21D95080B@codespeak.net>
Author: cfbolz
Date: Sun Oct 31 11:43:27 2010
New Revision: 78587
Modified:
pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py
Log:
the jit shouldn't look into this function
Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py
==============================================================================
--- pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py (original)
+++ pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py Sun Oct 31 11:43:27 2010
@@ -48,6 +48,7 @@
else:
return self._index(selector)
+ @jit.dont_look_inside
def _index_cache(self, selector):
space = self.space
cache = space.fromcache(IndexCache)
From arigo at codespeak.net Sun Oct 31 12:03:06 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 12:03:06 +0100 (CET)
Subject: [pypy-svn] r78588 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test
Message-ID: <20101031110306.F2420282BDA@codespeak.net>
Author: arigo
Date: Sun Oct 31 12:03:03 2010
New Revision: 78588
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
Fix VStringConcatValue: it cannot easily pass the 'lengthbox'
around the unrolled loop, so we reset it to None and (re-)compute
it lazily.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py Sun Oct 31 12:03:03 2010
@@ -180,12 +180,22 @@
class VStringConcatValue(VAbstractStringValue):
"""The concatenation of two other strings."""
- def setup(self, left, right, lengthbox):
+ lengthbox = None # or the computed length
+
+ def setup(self, left, right):
self.left = left
self.right = right
- self.lengthbox = lengthbox
- def getstrlen(self, _, mode):
+ def getstrlen(self, newoperations, mode):
+ if self.lengthbox is None:
+ len1box = self.left.getstrlen(newoperations, mode)
+ if len1box is None:
+ return None
+ len2box = self.right.getstrlen(newoperations, mode)
+ if len2box is None:
+ return None
+ self.lengthbox = _int_add(newoperations, len1box, len2box)
+ # ^^^ may still be None, if newoperations is None
return self.lengthbox
@specialize.arg(1)
@@ -223,6 +233,7 @@
if self.box is None:
self.left.enum_forced_boxes(boxes, already_seen)
self.right.enum_forced_boxes(boxes, already_seen)
+ self.lengthbox = None
else:
boxes.append(self.box)
@@ -322,6 +333,8 @@
return ConstInt(box1.value + box2.value)
elif isinstance(box2, ConstInt) and box2.value == 0:
return box1
+ if newoperations is None:
+ return None
resbox = BoxInt()
newoperations.append(ResOperation(rop.INT_ADD, [box1, box2], resbox))
return resbox
@@ -481,11 +494,8 @@
vleft.ensure_nonnull()
vright.ensure_nonnull()
newoperations = self.optimizer.newoperations
- len1box = vleft.getstrlen(newoperations, mode)
- len2box = vright.getstrlen(newoperations, mode)
- lengthbox = _int_add(newoperations, len1box, len2box)
value = self.make_vstring_concat(op.result, op, mode)
- value.setup(vleft, vright, lengthbox)
+ value.setup(vleft, vright)
return True
def opt_call_stroruni_STR_SLICE(self, op, mode):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sun Oct 31 12:03:03 2010
@@ -3990,7 +3990,7 @@
jump(p2, p3)
"""
expected = """
- [p1, p2]
+ [p2, p1]
i1 = strlen(p1)
i2 = strlen(p2)
i3 = int_add(i1, i2)
@@ -4000,7 +4000,7 @@
i5 = strlen(p2)
i6 = int_add(i4, i5) # will be killed by the backend
copystrcontent(p2, p3, 0, i4, i5)
- jump(p2, p3)
+ jump(p3, p2)
"""
self.optimize_strunicode_loop(ops, expected)
From cfbolz at codespeak.net Sun Oct 31 12:09:29 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sun, 31 Oct 2010 12:09:29 +0100 (CET)
Subject: [pypy-svn] r78589 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031110929.3E9D9282BDA@codespeak.net>
Author: cfbolz
Date: Sun Oct 31 12:09:26 2010
New Revision: 78589
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
this is needed for the tests to make sense
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sun Oct 31 12:09:26 2010
@@ -3946,12 +3946,13 @@
self.optimize_loop(ops, expected, preamble)
# ----------
- def optimize_strunicode_loop(self, ops, optops):
+ def optimize_strunicode_loop(self, ops, optops, preamble):
# check with the arguments passed in
- self.optimize_loop(ops, optops)
+ self.optimize_loop(ops, optops, preamble)
# check with replacing 'str' with 'unicode' everywhere
- self.optimize_loop(ops.replace('str','unicode').replace('s"', 'u"'),
- optops.replace('str','unicode').replace('s"', 'u"'))
+ def r(s):
+ return r.replace('str','unicode').replace('s"', 'u"')
+ self.optimize_loop(r(ops), r(optops), r(preamble))
def test_newstr_1(self):
ops = """
From cfbolz at codespeak.net Sun Oct 31 12:11:03 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sun, 31 Oct 2010 12:11:03 +0100 (CET)
Subject: [pypy-svn] r78590 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt
Message-ID: <20101031111103.9BBB8282BDD@codespeak.net>
Author: cfbolz
Date: Sun Oct 31 12:11:01 2010
New Revision: 78590
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
Log:
remove the now useless argument to setup
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py Sun Oct 31 12:11:01 2010
@@ -7,7 +7,7 @@
from pypy.jit.metainterp.optimizeopt.string import OptString
from pypy.jit.metainterp.optimizeopt.unroll import OptUnroll
-def optimize_loop_1(metainterp_sd, loop, not_a_bridge=True):
+def optimize_loop_1(metainterp_sd, loop, unroll=True):
"""Optimize loop.operations to remove internal overheadish operations.
"""
optimizations = [OptIntBounds(),
@@ -17,9 +17,9 @@
OptHeap(),
OptFfiCall(),
]
- if not_a_bridge:
+ if unroll:
optimizations.insert(0, OptUnroll())
- optimizer = Optimizer(metainterp_sd, loop, optimizations, not_a_bridge)
+ optimizer = Optimizer(metainterp_sd, loop, optimizations)
optimizer.propagate_all_forward()
def optimize_bridge_1(metainterp_sd, bridge):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py Sun Oct 31 12:11:01 2010
@@ -9,7 +9,7 @@
"""Keeps track of the bounds placed on integers by the guards and
remove redundant guards"""
- def setup(self, virtuals):
+ def setup(self):
self.posponedop = None
self.nextop = None
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Sun Oct 31 12:11:01 2010
@@ -190,7 +190,7 @@
op = ResOperation(opnum, args, result)
self.optimizer.pure_operations[self.optimizer.make_args_key(op)] = op
- def setup(self, not_a_bridge):
+ def setup(self):
pass
def force_at_end_of_preamble(self):
@@ -199,7 +199,7 @@
class Optimizer(Optimization):
- def __init__(self, metainterp_sd, loop, optimizations=None, not_a_bridge=True):
+ def __init__(self, metainterp_sd, loop, optimizations=None):
self.metainterp_sd = metainterp_sd
self.cpu = metainterp_sd.cpu
self.loop = loop
@@ -220,7 +220,7 @@
optimizations[-1].next_optimization = self
for o in optimizations:
o.optimizer = self
- o.setup(not_a_bridge)
+ o.setup()
else:
self.first_optimization = self
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Sun Oct 31 12:11:01 2010
@@ -8,8 +8,7 @@
become the preamble or entry bridge (don't think there is a
distinction anymore)"""
- def setup(self, not_a_bridge):
- assert not_a_bridge
+ def setup(self):
self.cloned_operations = []
for op in self.optimizer.loop.operations:
self.cloned_operations.append(op.clone())
@@ -20,12 +19,13 @@
if op.getopnum() == rop.JUMP:
self.force_at_end_of_preamble()
loop = self.optimizer.loop
+ assert op.getdescr() is loop.token
loop.preamble.operations = self.optimizer.newoperations
self.optimizer.newoperations = []
jump_args = op.getarglist()
op.initarglist([])
- # Exceptions not caught in one itteration should not propagate to the next
- self.optimizer.exception_might_have_happened = False
+ # Exceptions not caught in one iteration should not propagate to the next
+ self.optimizer.exception_might_have_happened = False
inputargs = self.inline(self.cloned_operations,
loop.inputargs, jump_args)
loop.inputargs = inputargs
@@ -60,6 +60,8 @@
if not isinstance(a, Const):
inputargs.append(a)
+ # this loop is equivalent to the main optimization loop in
+ # Optimizer.propagate_all_forward
for newop in loop_operations:
newop.initarglist([self.inline_arg(a) for a in newop.getarglist()])
if newop.result:
From david at codespeak.net Sun Oct 31 13:01:36 2010
From: david at codespeak.net (david at codespeak.net)
Date: Sun, 31 Oct 2010 13:01:36 +0100 (CET)
Subject: [pypy-svn] r78592 - in
pypy/branch/arm-backend/pypy/jit/backend/arm: . helper
Message-ID: <20101031120136.4FB96282B90@codespeak.net>
Author: david
Date: Sun Oct 31 13:01:34 2010
New Revision: 78592
Added:
pypy/branch/arm-backend/pypy/jit/backend/arm/helper/
pypy/branch/arm-backend/pypy/jit/backend/arm/helper/__init__.py
pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py
Log:
Implement unary int operations and perform some refactoring on opassembler
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Sun Oct 31 13:01:34 2010
@@ -12,12 +12,14 @@
from pypy.rpython.lltypesystem import lltype, rffi, llmemory
from pypy.jit.backend.arm.opassembler import (GuardOpAssembler,
IntOpAsslember,
- OpAssembler)
+ OpAssembler,
+ UnaryIntOpAssembler)
# XXX Move to llsupport
from pypy.jit.backend.x86.support import values_array
-class AssemblerARM(GuardOpAssembler, IntOpAsslember, OpAssembler):
+class AssemblerARM(GuardOpAssembler, IntOpAsslember,
+ OpAssembler, UnaryIntOpAssembler):
def __init__(self, cpu, failargs_limit=1000):
self.mc = ARMv7Builder()
@@ -182,7 +184,6 @@
opnum = op.getopnum()
fcond = self.operations[opnum](self, op, regalloc, fcond)
self.gen_func_epilog()
- print inputargs, operations
if self._debug_asm:
self._dump_trace('loop.asm')
print 'Done assembling'
@@ -210,6 +211,10 @@
def _dump_trace(self, name):
self.mc._dump_trace(name)
+ def _check_imm_arg(self, arg, size=0xFF):
+ #XXX check ranges for different operations
+ return isinstance(arg, ConstInt) and arg.getint() <= size and arg.getint() > 0
+
def patch_trace(self, faildescr, bridge_addr):
# XXX make sure there is enough space at patch target
fcond = faildescr._arm_guard_cond
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/__init__.py
==============================================================================
Added: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py
==============================================================================
--- (empty file)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py Sun Oct 31 13:01:34 2010
@@ -0,0 +1,82 @@
+from pypy.jit.backend.arm import conditions as c
+from pypy.jit.backend.arm import registers as r
+from pypy.jit.metainterp.history import ConstInt, BoxInt, Box
+
+def gen_emit_op_unary_cmp(true_cond, false_cond):
+ def f(self, op, regalloc, fcond):
+ arg = op.getarg(0)
+ reg = self._put_in_reg(arg, regalloc)
+ res = regalloc.try_allocate_reg(op.result)
+ self.mc.CMP_ri(reg.value, 0)
+ self.mc.MOV_ri(res.value, 1, true_cond)
+ self.mc.MOV_ri(res.value, 0, false_cond)
+ regalloc.possibly_free_var(reg)
+ regalloc.possibly_free_var(res)
+ return fcond
+ return f
+
+def gen_emit_op_ri(opname, imm_size=0xFF, commutative=True):
+ def f(self, op, regalloc, fcond):
+ ri_op = getattr(self.mc, '%s_ri' % opname)
+ rr_op = getattr(self.mc, '%s_rr' % opname)
+
+ arg0 = op.getarg(0)
+ arg1 = op.getarg(1)
+ res = regalloc.try_allocate_reg(op.result)
+ if (commutative
+ and self._check_imm_arg(arg0, imm_size)
+ and not isinstance(arg1, ConstInt)):
+ reg = regalloc.try_allocate_reg(arg1)
+ ri_op(res.value, reg.value, imm=arg0.getint(), cond=fcond)
+ elif self._check_imm_arg(arg1, imm_size) and not isinstance(arg0, ConstInt):
+ reg = regalloc.try_allocate_reg(arg0)
+ ri_op(res.value, reg.value, imm=arg1.getint(), cond=fcond)
+ else:
+ reg = self._put_in_reg(arg0, regalloc)
+ reg2 = self._put_in_reg(arg1, regalloc)
+ rr_op(res.value, reg.value, reg2.value)
+ regalloc.possibly_free_var(reg2)
+
+ regalloc.possibly_free_var(res)
+ regalloc.possibly_free_var(reg)
+ return fcond
+ return f
+
+def gen_emit_op_by_helper_call(opname):
+ def f(self, op, regalloc, fcond):
+ arg1 = regalloc.make_sure_var_in_reg(op.getarg(0), selected_reg=r.r0)
+ arg2 = regalloc.make_sure_var_in_reg(op.getarg(1), selected_reg=r.r1)
+ assert arg1 == r.r0
+ assert arg2 == r.r1
+ res = regalloc.try_allocate_reg(op.result)
+ getattr(self.mc, opname)(fcond)
+ self.mc.MOV_rr(res.value, r.r0.value, cond=fcond)
+ regalloc.possibly_free_vars_for_op(op)
+ return fcond
+ return f
+
+def gen_emit_cmp_op(condition, inverse=False):
+ def f(self, op, regalloc, fcond):
+ assert fcond == c.AL
+ if not inverse:
+ arg0 = op.getarg(0)
+ arg1 = op.getarg(1)
+ else:
+ arg0 = op.getarg(1)
+ arg1 = op.getarg(0)
+ res = regalloc.try_allocate_reg(op.result)
+ # XXX consider swapping argumentes if arg0 is const
+ if self._check_imm_arg(arg1) and not isinstance(arg0, ConstInt):
+ reg = regalloc.try_allocate_reg(arg0)
+ self.mc.CMP_ri(reg.value, imm=arg1.getint(), cond=fcond)
+ else:
+ reg = self._put_in_reg(arg0, regalloc)
+ reg2 = self._put_in_reg(arg1, regalloc)
+ self.mc.CMP_rr(reg.value, reg2.value)
+ regalloc.possibly_free_var(reg2)
+
+ inv = c.get_opposite_of(condition)
+ self.mc.MOV_ri(res.value, 1, cond=condition)
+ self.mc.MOV_ri(res.value, 0, cond=inv)
+ return condition
+ return f
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py Sun Oct 31 13:01:34 2010
@@ -32,7 +32,8 @@
#'RRX_ri': {'op1':0x1A, 'op2':0, 'op3':0x3, 'op2cond':'0', 'result':False, 'base':True},
'ROR_ri': {'op1':0x1A, 'op2':0x0, 'op3':0x3, 'op2cond':'!0', 'result':True, 'base':False},
#BIC
- #MVN
+ 'MVN_rr': {'op1':0x1E, 'op2':0x0, 'op3':0x0, 'result':True, 'base':False},
+
}
data_proc_reg_shift_reg = {
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Sun Oct 31 13:01:34 2010
@@ -3,6 +3,10 @@
from pypy.jit.backend.arm import registers as r
from pypy.jit.backend.arm.arch import (WORD, FUNC_ALIGN, arm_int_div,
arm_int_div_sign, arm_int_mod_sign, arm_int_mod)
+
+from pypy.jit.backend.arm.helper.assembler import (gen_emit_op_by_helper_call,
+ gen_emit_op_unary_cmp,
+ gen_emit_op_ri, gen_emit_cmp_op)
from pypy.jit.backend.arm.codebuilder import ARMv7Builder, ARMv7InMemoryBuilder
from pypy.jit.backend.arm.regalloc import ARMRegisterManager
from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity
@@ -12,95 +16,31 @@
from pypy.rpython.annlowlevel import llhelper
from pypy.rpython.lltypesystem import lltype, rffi, llmemory
-def gen_emit_op_ri(opname, imm_size=0xFF, commutative=True):
- def f(self, op, regalloc, fcond):
- ri_op = getattr(self.mc, '%s_ri' % opname)
- rr_op = getattr(self.mc, '%s_rr' % opname)
-
- arg0 = op.getarg(0)
- arg1 = op.getarg(1)
- res = regalloc.try_allocate_reg(op.result)
- if (commutative
- and self._check_imm_arg(arg0, imm_size)
- and not isinstance(arg1, ConstInt)):
- reg = regalloc.try_allocate_reg(arg1)
- ri_op(res.value, reg.value, imm=arg0.getint(), cond=fcond)
- elif self._check_imm_arg(arg1, imm_size) and not isinstance(arg0, ConstInt):
- reg = regalloc.try_allocate_reg(arg0)
- ri_op(res.value, reg.value, imm=arg1.getint(), cond=fcond)
- else:
- reg = self._put_in_reg(arg0, regalloc)
- reg2 = self._put_in_reg(arg1, regalloc)
- rr_op(res.value, reg.value, reg2.value)
- regalloc.possibly_free_var(reg2)
-
- regalloc.possibly_free_var(res)
- regalloc.possibly_free_var(reg)
- return fcond
- return f
-
-def gen_emit_op_by_helper_call(opname):
- def f(self, op, regalloc, fcond):
- arg1 = regalloc.make_sure_var_in_reg(op.getarg(0), selected_reg=r.r0)
- arg2 = regalloc.make_sure_var_in_reg(op.getarg(1), selected_reg=r.r1)
- assert arg1 == r.r0
- assert arg2 == r.r1
- res = regalloc.try_allocate_reg(op.result)
- getattr(self.mc, opname)(fcond)
- self.mc.MOV_rr(res.value, r.r0.value, cond=fcond)
- regalloc.possibly_free_vars_for_op(op)
- return fcond
- return f
-
-def gen_emit_cmp_op(condition, inverse=False):
- def f(self, op, regalloc, fcond):
- assert fcond == c.AL
- if not inverse:
- arg0 = op.getarg(0)
- arg1 = op.getarg(1)
- else:
- arg0 = op.getarg(1)
- arg1 = op.getarg(0)
- res = regalloc.try_allocate_reg(op.result)
- # XXX consider swapping argumentes if arg0 is const
- if self._check_imm_arg(arg1) and not isinstance(arg0, ConstInt):
- reg = regalloc.try_allocate_reg(arg0)
- self.mc.CMP_ri(reg.value, imm=arg1.getint(), cond=fcond)
- else:
- reg = self._put_in_reg(arg0, regalloc)
- reg2 = self._put_in_reg(arg1, regalloc)
- self.mc.CMP_rr(reg.value, reg2.value)
- regalloc.possibly_free_var(reg2)
-
- inv = c.get_opposite_of(condition)
- self.mc.MOV_ri(res.value, 1, cond=condition)
- self.mc.MOV_ri(res.value, 0, cond=inv)
- return condition
- return f
-
class IntOpAsslember(object):
_mixin_ = True
- # XXX support constants larger than imm
-
def emit_op_int_add(self, op, regalloc, fcond):
# assuming only one argument is constant
res = regalloc.try_allocate_reg(op.result)
- if isinstance(op.getarg(0), ConstInt) or isinstance(op.getarg(1), ConstInt):
- if isinstance(op.getarg(1), ConstInt):
- reg = regalloc.try_allocate_reg(op.getarg(0))
- arg1 = op.getarg(1)
- elif isinstance(op.getarg(0), ConstInt):
- reg = regalloc.try_allocate_reg(op.getarg(1))
- arg1 = op.getarg(0)
+ a0 = op.getarg(0)
+ a1 = op.getarg(1)
+ imm_a0 = isinstance(a0, ConstInt) and (a0.getint() <= 0xFF or -1 * a0.getint() <= 0xFF)
+ imm_a1 = isinstance(a1, ConstInt) and (a1.getint() <= 0xFF or -1 * a1.getint() <= 0xFF)
+ if imm_a0 or imm_a1:
+ if imm_a1:
+ reg = regalloc.try_allocate_reg(a0)
+ arg1 = a1
+ elif imm_a0:
+ reg = regalloc.try_allocate_reg(a1)
+ arg1 = a0
value = arg1.getint()
if value < 0:
self.mc.SUB_ri(res.value, reg.value, -1 * value)
else:
self.mc.ADD_ri(res.value, reg.value, value)
else:
- r1 = regalloc.try_allocate_reg(op.getarg(0))
- r2 = regalloc.try_allocate_reg(op.getarg(1))
+ r1 = regalloc.try_allocate_reg(a0)
+ r2 = regalloc.try_allocate_reg(a1)
self.mc.ADD_rr(res.value, r1.value, r2.value)
regalloc.possibly_free_vars_for_op(op)
@@ -109,17 +49,21 @@
def emit_op_int_sub(self, op, regalloc, fcond):
# assuming only one argument is constant
res = regalloc.try_allocate_reg(op.result)
- if isinstance(op.getarg(0), ConstInt) or isinstance(op.getarg(1), ConstInt):
- if isinstance(op.getarg(1), ConstInt):
- reg = regalloc.try_allocate_reg(op.getarg(0))
- value = op.getarg(1).getint()
+ a0 = op.getarg(0)
+ a1 = op.getarg(1)
+ imm_a0 = isinstance(a0, ConstInt) and (a0.getint() <= 0xFF or -1 * a0.getint() <= 0xFF)
+ imm_a1 = isinstance(a1, ConstInt) and (a1.getint() <= 0xFF or -1 * a1.getint() <= 0xFF)
+ if imm_a0 or imm_a1:
+ if imm_a1:
+ reg = self._put_in_reg(a0, regalloc)
+ value = a1.getint()
if value < 0:
self.mc.ADD_ri(res.value, reg.value, -1 * value)
else:
self.mc.SUB_ri(res.value, reg.value, value)
- elif isinstance(op.getarg(0), ConstInt):
- reg = regalloc.try_allocate_reg(op.getarg(1))
- value = op.getarg(0).getint()
+ elif imm_a0:
+ reg = self._put_in_reg(a1, regalloc)
+ value = a0.getint()
if value < 0:
self.mc.ADD_ri(res.value, reg.value, -1 * value)
self.mc.MVN_rr(res.value, res.value)
@@ -127,8 +71,8 @@
# reverse substract ftw
self.mc.RSB_ri(res.value, reg.value, value)
else:
- r1 = regalloc.try_allocate_reg(op.getarg(0))
- r2 = regalloc.try_allocate_reg(op.getarg(1))
+ r1 = regalloc.try_allocate_reg(a0)
+ r2 = regalloc.try_allocate_reg(a1)
self.mc.SUB_rr(res.value, r1.value, r2.value)
regalloc.possibly_free_vars_for_op(op)
@@ -177,10 +121,32 @@
emit_op_uint_ge = gen_emit_cmp_op(c.LS, inverse=True)
- def _check_imm_arg(self, arg, size=0xFF):
- #XXX check ranges for different operations
- return isinstance(arg, ConstInt) and arg.getint() <= size and arg.getint() > 0
+class UnaryIntOpAssembler(object):
+ emit_op_int_is_true = gen_emit_op_unary_cmp(c.NE, c.EQ)
+ emit_op_int_is_zero = gen_emit_op_unary_cmp(c.EQ, c.NE)
+
+ def emit_op_int_invert(self, op, regalloc, fcond):
+ arg = op.getarg(0)
+ reg = self._put_in_reg(arg, regalloc)
+ res = regalloc.try_allocate_reg(op.result)
+
+ self.mc.MVN_rr(res.value, reg.value)
+ regalloc.possibly_free_var(reg)
+ regalloc.possibly_free_var(res)
+ return fcond
+
+ #XXX check for a better way of doing this
+ def emit_op_int_neg(self, op, regalloc, fcond):
+ arg = op.getarg(0)
+ reg = self._put_in_reg(arg, regalloc)
+ reg2 = self._put_in_reg(ConstInt(-1), regalloc)
+ res = regalloc.try_allocate_reg(op.result)
+ self.mc.MUL(res.value, reg.value, reg2.value)
+ regalloc.possibly_free_var(reg)
+ regalloc.possibly_free_var(reg2)
+ regalloc.possibly_free_var(res)
+ return fcond
class GuardOpAssembler(object):
_mixin_ = True
From cfbolz at codespeak.net Sun Oct 31 13:09:28 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sun, 31 Oct 2010 13:09:28 +0100 (CET)
Subject: [pypy-svn] r78593 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: . test
Message-ID: <20101031120928.2B779282B90@codespeak.net>
Author: cfbolz
Date: Sun Oct 31 13:09:26 2010
New Revision: 78593
Added:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/nounroll_optimize.py
- copied, changed from r78588, pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize_nopspec.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_nounroll.py
- copied unchanged from r78588, pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_nopspec.py
Removed:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize_nopspec.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_nopspec.py
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmstate.py
Log:
make optimize_nopspec not use unrolling
Copied: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/nounroll_optimize.py (from r78588, pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize_nopspec.py)
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize_nopspec.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/nounroll_optimize.py Sun Oct 31 13:09:26 2010
@@ -2,8 +2,6 @@
from pypy.rlib.debug import debug_start, debug_stop
from pypy.jit.metainterp.optimizeopt import optimize_loop_1, optimize_bridge_1
-# XXX this file should probably disable unrolling instead
-
def optimize_loop(metainterp_sd, old_loop_tokens, loop):
debug_start("jit-optimize")
try:
@@ -16,7 +14,7 @@
metainterp_sd.logger_noopt.log_loop(loop.inputargs, loop.operations)
if old_loop_tokens:
return old_loop_tokens[0]
- optimize_loop_1(metainterp_sd, loop)
+ optimize_loop_1(metainterp_sd, loop, False)
return None
def optimize_bridge(metainterp_sd, old_loop_tokens, bridge):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py Sun Oct 31 13:09:26 2010
@@ -4,7 +4,7 @@
from pypy.jit.metainterp.compile import ResumeGuardDescr
from pypy.jit.metainterp.compile import ResumeGuardCountersInt
from pypy.jit.metainterp.compile import compile_tmp_callback
-from pypy.jit.metainterp import optimize_nopspec, jitprof, typesystem, compile
+from pypy.jit.metainterp import nounroll_optimize, jitprof, typesystem, compile
from pypy.jit.metainterp.test.test_optimizeutil import LLtypeMixin
from pypy.jit.tool.oparser import parse
@@ -38,7 +38,7 @@
pass
class FakeState:
- optimize_loop = staticmethod(optimize_nopspec.optimize_loop)
+ optimize_loop = staticmethod(nounroll_optimize.optimize_loop)
debug_level = 0
def attach_unoptimized_bridge_from_interp(*args):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmstate.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmstate.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmstate.py Sun Oct 31 13:09:26 2010
@@ -199,9 +199,9 @@
self.optimize_loop = simple_optimize.optimize_loop
self.optimize_bridge = simple_optimize.optimize_bridge
elif optimizer == OPTIMIZER_NO_PERFECTSPEC:
- from pypy.jit.metainterp import optimize_nopspec
- self.optimize_loop = optimize_nopspec.optimize_loop
- self.optimize_bridge = optimize_nopspec.optimize_bridge
+ from pypy.jit.metainterp import nounroll_optimize
+ self.optimize_loop = nounroll_optimize.optimize_loop
+ self.optimize_bridge = nounroll_optimize.optimize_bridge
elif optimizer == OPTIMIZER_FULL:
from pypy.jit.metainterp import optimize
self.optimize_loop = optimize.optimize_loop
From david at codespeak.net Sun Oct 31 13:22:53 2010
From: david at codespeak.net (david at codespeak.net)
Date: Sun, 31 Oct 2010 13:22:53 +0100 (CET)
Subject: [pypy-svn] r78594 - pypy/branch/arm-backend/pypy/jit/backend/arm
Message-ID: <20101031122253.45E7836E0C1@codespeak.net>
Author: david
Date: Sun Oct 31 13:22:47 2010
New Revision: 78594
Modified:
pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
Log:
Improve trace exit path
Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py
==============================================================================
--- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original)
+++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Sun Oct 31 13:22:47 2010
@@ -100,9 +100,10 @@
self.mc.MOV_rr(r.r0.value, r.lr.value) # move mem block address, to r0 to pass as
# parameter to next procedure call
self.mc.MOV_rr(r.r1.value, r.sp.value) # pass the current stack pointer as second param
- self.mc.gen_load_int(r.r2.value, rffi.cast(lltype.Signed, decode_registers_addr))
- self.mc.gen_load_int(r.lr.value, self.mc.curraddr()+self.mc.size_of_gen_load_int+WORD)
- self.mc.MOV_rr(r.pc.value, r.r2.value)
+
+ self.mc.ADD_ri(r.lr.value, r.pc.value, 4)
+ self.mc.LDR_ri(r.pc.value, r.pc.value, -4)
+ self.mc.write32(rffi.cast(lltype.Signed, decode_registers_addr))
self.mc.MOV_rr(r.ip.value, r.r0.value)
self.mc.LDM(r.sp.value, range(12), w=1) # XXX Replace with POP instr. someday
From hakanardo at codespeak.net Sun Oct 31 13:25:56 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sun, 31 Oct 2010 13:25:56 +0100 (CET)
Subject: [pypy-svn] r78595 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test
Message-ID: <20101031122556.E1A21282BE3@codespeak.net>
Author: hakanardo
Date: Sun Oct 31 13:25:51 2010
New Revision: 78595
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
(hakanardo, arigo) workaround that dissables string optimisation of the preamble
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py Sun Oct 31 13:25:51 2010
@@ -10,15 +10,18 @@
def optimize_loop_1(metainterp_sd, loop, unroll=True):
"""Optimize loop.operations to remove internal overheadish operations.
"""
+ opt_str = OptString()
optimizations = [OptIntBounds(),
OptRewrite(),
OptVirtualize(),
- OptString(),
+ opt_str,
OptHeap(),
OptFfiCall(),
]
if unroll:
optimizations.insert(0, OptUnroll())
+ opt_str.enabled = False # FIXME: Workaround to disable string optimisation
+ # during preamble but to keep it during the loop
optimizer = Optimizer(metainterp_sd, loop, optimizations)
optimizer.propagate_all_forward()
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py Sun Oct 31 13:25:51 2010
@@ -162,7 +162,7 @@
for value in self._chars:
value.get_args_for_fail(modifier)
- def enum_forced_boxes(self, boxes, already_seen):
+ def FIXME_enum_forced_boxes(self, boxes, already_seen):
key = self.get_key_box()
if key in already_seen:
return
@@ -225,7 +225,7 @@
self.left.get_args_for_fail(modifier)
self.right.get_args_for_fail(modifier)
- def enum_forced_boxes(self, boxes, already_seen):
+ def FIXME_enum_forced_boxes(self, boxes, already_seen):
key = self.get_key_box()
if key in already_seen:
return
@@ -283,7 +283,7 @@
self.vstart.get_args_for_fail(modifier)
self.vlength.get_args_for_fail(modifier)
- def enum_forced_boxes(self, boxes, already_seen):
+ def FIXME_enum_forced_boxes(self, boxes, already_seen):
key = self.get_key_box()
if key in already_seen:
return
@@ -365,7 +365,8 @@
class OptString(optimizer.Optimization):
"Handling of strings and unicodes."
-
+ enabled = True
+
def make_vstring_plain(self, box, source_op, mode):
vvalue = VStringPlainValue(self.optimizer, box, source_op, mode)
self.make_equal_to(box, vvalue)
@@ -643,6 +644,10 @@
self.optimizer.newoperations.append(op)
def propagate_forward(self, op):
+ if not self.enabled:
+ self.emit_operation(op)
+ return
+
opnum = op.getopnum()
for value, func in optimize_ops:
if opnum == value:
@@ -651,6 +656,9 @@
else:
self.emit_operation(op)
+ def force_at_end_of_preamble(self):
+ self.enabled = True
+
optimize_ops = _findall(OptString, 'optimize_')
def _findall_call_oopspec():
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sun Oct 31 13:25:51 2010
@@ -3946,12 +3946,14 @@
self.optimize_loop(ops, expected, preamble)
# ----------
- def optimize_strunicode_loop(self, ops, optops, preamble):
+ def optimize_strunicode_loop(self, ops, optops, preamble=None):
+ if not preamble:
+ preamble = ops # FIXME: Force proper testing of preamble
# check with the arguments passed in
self.optimize_loop(ops, optops, preamble)
# check with replacing 'str' with 'unicode' everywhere
def r(s):
- return r.replace('str','unicode').replace('s"', 'u"')
+ return s.replace('str','unicode').replace('s"', 'u"')
self.optimize_loop(r(ops), r(optops), r(preamble))
def test_newstr_1(self):
@@ -3991,7 +3993,7 @@
jump(p2, p3)
"""
expected = """
- [p2, p1]
+ [p1, p2]
i1 = strlen(p1)
i2 = strlen(p2)
i3 = int_add(i1, i2)
@@ -4001,7 +4003,7 @@
i5 = strlen(p2)
i6 = int_add(i4, i5) # will be killed by the backend
copystrcontent(p2, p3, 0, i4, i5)
- jump(p3, p2)
+ jump(p2, p3)
"""
self.optimize_strunicode_loop(ops, expected)
@@ -4107,12 +4109,18 @@
escape(p3)
jump()
"""
+ preamble = """
+ []
+ p3 = call(0, s"ab", s"cde", descr=strconcatdescr)
+ escape(p3)
+ jump()
+ """
expected = """
[]
escape(s"abcde")
jump()
"""
- self.optimize_strunicode_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected, preamble)
def test_str_slice_1(self):
ops = """
@@ -4219,7 +4227,7 @@
self.optimize_strunicode_loop(ops, expected)
# ----------
- def optimize_strunicode_loop_extradescrs(self, ops, optops):
+ def optimize_strunicode_loop_extradescrs(self, ops, optops, preamble=None):
from pypy.jit.metainterp.optimizeopt import string
def my_callinfo_for_oopspec(oopspecindex):
calldescrtype = type(LLtypeMixin.strequaldescr)
@@ -4234,7 +4242,7 @@
saved = string.callinfo_for_oopspec
try:
string.callinfo_for_oopspec = my_callinfo_for_oopspec
- self.optimize_strunicode_loop(ops, optops)
+ self.optimize_strunicode_loop(ops, optops, preamble)
finally:
string.callinfo_for_oopspec = saved
@@ -4320,14 +4328,13 @@
"""
expected = """
[p1, i1, i2, p3]
- guard_nonnull(p3) []
i4 = int_sub(i2, i1)
i0 = call(0, p1, i1, i4, p3, descr=streq_slice_nonnull_descr)
escape(i0)
jump(p1, i1, i2, p3)
"""
self.optimize_strunicode_loop_extradescrs(ops,
- expected)
+ expected, ops)
def test_str_equal_slice4(self):
ops = """
@@ -4407,7 +4414,6 @@
"""
expected = """
[p1]
- guard_nonnull(p1) []
i0 = call(0, p1, s"hello world", descr=streq_nonnull_descr)
escape(i0)
jump(p1)
@@ -4424,7 +4430,6 @@
"""
expected = """
[p1]
- guard_nonnull(p1) []
i1 = strlen(p1)
i0 = int_eq(i1, 0)
escape(i0)
@@ -4442,7 +4447,6 @@
"""
expected = """
[p1]
- guard_nonnull(p1) []
i0 = call(0, p1, 120, descr=streq_nonnull_char_descr)
escape(i0)
jump(p1)
From hakanardo at codespeak.net Sun Oct 31 13:35:42 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sun, 31 Oct 2010 13:35:42 +0100 (CET)
Subject: [pypy-svn] r78596 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt
Message-ID: <20101031123542.33CC0282BDA@codespeak.net>
Author: hakanardo
Date: Sun Oct 31 13:35:36 2010
New Revision: 78596
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
Log:
(hakanardo, arigo) saner propagation of force_at_end_of_preamble
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py Sun Oct 31 13:35:36 2010
@@ -151,7 +151,6 @@
def force_at_end_of_preamble(self):
self.force_all_lazy_setfields()
- self.next_optimization.force_at_end_of_preamble()
def force_lazy_setfield(self, descr, before_guard=False):
try:
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Sun Oct 31 13:35:36 2010
@@ -194,8 +194,7 @@
pass
def force_at_end_of_preamble(self):
- if self.next_optimization:
- self.next_optimization.force_at_end_of_preamble()
+ pass
class Optimizer(Optimization):
@@ -222,7 +221,14 @@
o.optimizer = self
o.setup()
else:
+ optimizations = []
self.first_optimization = self
+
+ self.optimizations = optimizations
+
+ def force_at_end_of_preamble(self):
+ for o in self.optimizations:
+ o.force_at_end_of_preamble()
def forget_numberings(self, virtualbox):
self.metainterp_sd.profiler.count(jitprof.OPT_FORCINGS)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Sun Oct 31 13:35:36 2010
@@ -17,7 +17,7 @@
def propagate_forward(self, op):
if op.getopnum() == rop.JUMP:
- self.force_at_end_of_preamble()
+ self.optimizer.force_at_end_of_preamble()
loop = self.optimizer.loop
assert op.getdescr() is loop.token
loop.preamble.operations = self.optimizer.newoperations
From hakanardo at codespeak.net Sun Oct 31 13:37:55 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sun, 31 Oct 2010 13:37:55 +0100 (CET)
Subject: [pypy-svn] r78597 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031123755.2725F282BDA@codespeak.net>
Author: hakanardo
Date: Sun Oct 31 13:37:52 2010
New Revision: 78597
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
fixed test
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sun Oct 31 13:37:52 2010
@@ -3461,7 +3461,7 @@
i0 = strlen(p0)
jump(p0)
"""
- self.optimize_strunicode_loop(ops, expected)
+ self.optimize_strunicode_loop(ops, expected, expected)
def test_addsub_const(self):
ops = """
From arigo at codespeak.net Sun Oct 31 13:46:06 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 13:46:06 +0100 (CET)
Subject: [pypy-svn] r78598 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031124606.A8696282BE8@codespeak.net>
Author: arigo
Date: Sun Oct 31 13:46:01 2010
New Revision: 78598
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
Log:
Fix.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py Sun Oct 31 13:46:01 2010
@@ -268,8 +268,7 @@
OptHeap(),
OptFfiCall(),
]
- optimizer = Optimizer(metainterp_sd, loop, optimizations,
- not_a_bridge=True)
+ optimizer = Optimizer(metainterp_sd, loop, optimizations)
optimizer.propagate_all_forward()
#
expected = self.parse(optops)
From cfbolz at codespeak.net Sun Oct 31 14:12:59 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sun, 31 Oct 2010 14:12:59 +0100 (CET)
Subject: [pypy-svn] r78599 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: . test
Message-ID: <20101031131259.85EC6282BDA@codespeak.net>
Author: cfbolz
Date: Sun Oct 31 14:12:58 2010
New Revision: 78599
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_nounroll.py
Log:
(hakanardo, cfbolz, arigo): when not using unrolling, always register every loop
as an entry bridge. this is safe, because there is no specialization any more.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py Sun Oct 31 14:12:58 2010
@@ -79,7 +79,7 @@
metainterp.staticdata.log("reusing old loop")
return old_loop_token
- if loop.preamble.operations:
+ if loop.preamble.operations is not None:
send_loop_to_backend(metainterp_sd, loop, "loop")
send_loop_to_backend(metainterp_sd, loop.preamble, "entry bridge")
insert_loop_token(old_loop_tokens, loop.preamble.token)
@@ -89,6 +89,8 @@
else:
send_loop_to_backend(metainterp_sd, loop, "loop")
insert_loop_token(old_loop_tokens, loop_token)
+ jitdriver_sd.warmstate.attach_unoptimized_bridge_from_interp(
+ greenkey, loop.token)
return loop_token
def insert_loop_token(old_loop_tokens, loop_token):
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_nounroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_nounroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_nounroll.py Sun Oct 31 14:12:58 2010
@@ -20,8 +20,11 @@
def check_jumps(self, maxcount):
pass
+
class TestLLtype(LoopNoPSpecTest, LLJitMixin):
- pass
+ def check_tree_loop_count(self, count):
+ # we get one less entry bridge
+ return LLJitMixin.check_tree_loop_count(self, count - 1)
class TestOOtype(LoopNoPSpecTest, OOJitMixin):
pass
From cfbolz at codespeak.net Sun Oct 31 14:17:54 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sun, 31 Oct 2010 14:17:54 +0100 (CET)
Subject: [pypy-svn] r78600 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031131754.B61A7282BE3@codespeak.net>
Author: cfbolz
Date: Sun Oct 31 14:17:53 2010
New Revision: 78600
Added:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send_simple.py
- copied unchanged from r78598, pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_dummy.py
Removed:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_dummy.py
Log:
rename to reflect more what it does
From cfbolz at codespeak.net Sun Oct 31 14:23:19 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sun, 31 Oct 2010 14:23:19 +0100 (CET)
Subject: [pypy-svn] r78601 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031132319.87744282BE8@codespeak.net>
Author: cfbolz
Date: Sun Oct 31 14:23:18 2010
New Revision: 78601
Added:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_unroll.py
- copied unchanged from r78598, pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_spec.py
Removed:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_spec.py
Log:
another name fix
From cfbolz at codespeak.net Sun Oct 31 14:23:28 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sun, 31 Oct 2010 14:23:28 +0100 (CET)
Subject: [pypy-svn] r78602 - pypy/extradoc/talk/pycon2011
Message-ID: <20101031132328.DC6DC5080B@codespeak.net>
Author: cfbolz
Date: Sun Oct 31 14:23:27 2010
New Revision: 78602
Modified:
pypy/extradoc/talk/pycon2011/pypy-optimizations.txt
Log:
small fixes, an XXX
Modified: pypy/extradoc/talk/pycon2011/pypy-optimizations.txt
==============================================================================
--- pypy/extradoc/talk/pycon2011/pypy-optimizations.txt (original)
+++ pypy/extradoc/talk/pycon2011/pypy-optimizations.txt Sun Oct 31 14:23:27 2010
@@ -10,14 +10,14 @@
Classification: Discuss in depth
Abstract:
-The talk would contain a list of features that are hard to optimize in Python,
+The talk will contain a list of features that are hard to optimize in Python,
why and how we're fixing it in PyPy. This also might be used as a rough guide
what features in Python are optimizable right now and what are not.
The list of mentioned features is not exhaustive, however we will try to
focus at least on the following:
-* Dynamic language - In Python code we have no known types, like a static
+* Dynamic language - In Python code we have no known types, like a statically typed
language. Even operations like "a + b" can do anything, unless we know more
about the code.
@@ -27,10 +27,10 @@
* Object model - All user defined Python objects have a dictionary which stores
their attributes, as does every type. When Python does an attribute lookup
- this requires 2 dictionary lookups.
+ this requires at least two dictionary lookups.
- In PyPy we use an approach similar to one used by v8 with hidden classes
- (except more pypy specific) called shared dictionaries.
+ In PyPy we use an approach similar to the one used by V8 with hidden classes
+ (except more PyPy specific) called map dictionaries. XXX type versions?
* FFI calls - calling C from Python is costly and hard to optimize. In PyPy we
decided to go via ctypes, this part will explain how are we going to handle
From cfbolz at codespeak.net Sun Oct 31 14:43:52 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sun, 31 Oct 2010 14:43:52 +0100 (CET)
Subject: [pypy-svn] r78603 - in pypy/branch/jit-unroll-loops/pypy:
jit/metainterp jit/metainterp/test jit/tl jit/tl/tla rlib
Message-ID: <20101031134352.8B2DB282BE3@codespeak.net>
Author: cfbolz
Date: Sun Oct 31 14:43:50 2010
New Revision: 78603
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_nounroll.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmstate.py
pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py
pypy/branch/jit-unroll-loops/pypy/jit/tl/tla/test_tla.py
pypy/branch/jit-unroll-loops/pypy/rlib/jit.py
Log:
rename OPTIMIZER_NO_PERFECTSPEC to OPTIMIZER_NO_UNROLL
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_nounroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_nounroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_nounroll.py Sun Oct 31 14:43:50 2010
@@ -1,12 +1,12 @@
from pypy.jit.metainterp.test import test_loop, test_send
from pypy.jit.metainterp.warmspot import ll_meta_interp
-from pypy.rlib.jit import OPTIMIZER_NO_PERFECTSPEC
+from pypy.rlib.jit import OPTIMIZER_NO_UNROLL
from pypy.jit.metainterp.test.test_basic import LLJitMixin, OOJitMixin
class LoopNoPSpecTest(test_send.SendTests):
def meta_interp(self, func, args, **kwds):
- return ll_meta_interp(func, args, optimizer=OPTIMIZER_NO_PERFECTSPEC,
+ return ll_meta_interp(func, args, optimizer=OPTIMIZER_NO_UNROLL,
CPUClass=self.CPUClass,
type_system=self.type_system,
**kwds)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmstate.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmstate.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmstate.py Sun Oct 31 14:43:50 2010
@@ -8,7 +8,7 @@
from pypy.rlib.nonconst import NonConstant
from pypy.rlib.unroll import unrolling_iterable
from pypy.rlib.jit import (PARAMETERS, OPTIMIZER_SIMPLE, OPTIMIZER_FULL,
- OPTIMIZER_NO_PERFECTSPEC)
+ OPTIMIZER_NO_UNROLL)
from pypy.rlib.jit import DEBUG_PROFILE
from pypy.rlib.jit import BaseJitCell
from pypy.rlib.debug import debug_start, debug_stop, debug_print
@@ -198,7 +198,7 @@
from pypy.jit.metainterp import simple_optimize
self.optimize_loop = simple_optimize.optimize_loop
self.optimize_bridge = simple_optimize.optimize_bridge
- elif optimizer == OPTIMIZER_NO_PERFECTSPEC:
+ elif optimizer == OPTIMIZER_NO_UNROLL:
from pypy.jit.metainterp import nounroll_optimize
self.optimize_loop = nounroll_optimize.optimize_loop
self.optimize_bridge = nounroll_optimize.optimize_bridge
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py Sun Oct 31 14:43:50 2010
@@ -2,7 +2,7 @@
from pypy.rpython.lltypesystem import lltype
from pypy.jit.metainterp import warmspot
from pypy.module.pypyjit.policy import PyPyJitPolicy
-from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_NO_PERFECTSPEC
+from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_NO_UNROLL
def run_child(glob, loc):
@@ -35,5 +35,5 @@
warmspot.jittify_and_run(interp, graph, [], policy=policy,
listops=True, CPUClass=CPUClass,
backendopt=True, inline=False,
- optimizer=OPTIMIZER_NO_PERFECTSPEC)
+ optimizer=OPTIMIZER_NO_UNROLL)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/tla/test_tla.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tl/tla/test_tla.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tl/tla/test_tla.py Sun Oct 31 14:43:50 2010
@@ -156,7 +156,7 @@
# ____________________________________________________________
from pypy.jit.metainterp.test.test_basic import LLJitMixin
-from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_NO_PERFECTSPEC
+from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_NO_UNROLL
class TestLLtype(LLJitMixin):
def test_loop(self):
@@ -178,5 +178,5 @@
assert isinstance(w_result, tla.W_IntObject)
return w_result.intvalue
res = self.meta_interp(interp_w, [42], listops=True,
- optimizer=OPTIMIZER_NO_PERFECTSPEC)
+ optimizer=OPTIMIZER_NO_UNROLL)
assert res == 0
Modified: pypy/branch/jit-unroll-loops/pypy/rlib/jit.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/rlib/jit.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/rlib/jit.py Sun Oct 31 14:43:50 2010
@@ -248,7 +248,7 @@
"""Inconsistency in the JIT hints."""
OPTIMIZER_SIMPLE = 0
-OPTIMIZER_NO_PERFECTSPEC = 1
+OPTIMIZER_NO_UNROLL = 1
OPTIMIZER_FULL = 2
DEBUG_OFF = 0
@@ -261,7 +261,7 @@
'trace_limit': 10000,
'inlining': False,
'optimizer': OPTIMIZER_FULL,
- #'optimizer': OPTIMIZER_NO_PERFECTSPEC,
+ #'optimizer': OPTIMIZER_NO_UNROLL,
'debug' : DEBUG_STEPS,
}
unroll_parameters = unrolling_iterable(PARAMETERS.keys())
From arigo at codespeak.net Sun Oct 31 14:47:13 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 14:47:13 +0100 (CET)
Subject: [pypy-svn] r78604 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt
Message-ID: <20101031134713.0652D5080C@codespeak.net>
Author: arigo
Date: Sun Oct 31 14:47:12 2010
New Revision: 78604
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
Log:
simplification.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Sun Oct 31 14:47:12 2010
@@ -116,17 +116,7 @@
def inline_snapshot(self, snapshot):
if snapshot in self.snapshot_map:
return self.snapshot_map[snapshot]
- boxes = []
- for a in snapshot.boxes:
- if isinstance(a, Const):
- boxes.append(a)
- else:
- boxes.append(self.inline_arg(a))
- new_snapshot = Snapshot(self.inline_snapshot(snapshot.prev), boxes[:])
+ boxes = [self.inline_arg(a) for a in snapshot.boxes]
+ new_snapshot = Snapshot(self.inline_snapshot(snapshot.prev), boxes)
self.snapshot_map[snapshot] = new_snapshot
return new_snapshot
-
-
-
-
-
From arigo at codespeak.net Sun Oct 31 14:47:26 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 14:47:26 +0100 (CET)
Subject: [pypy-svn] r78605 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt
Message-ID: <20101031134726.B9574282BE3@codespeak.net>
Author: arigo
Date: Sun Oct 31 14:47:25 2010
New Revision: 78605
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
Log:
Add a missing case in __repr__.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Sun Oct 31 14:47:25 2010
@@ -147,6 +147,8 @@
def __repr__(self):
cls_name = self.known_class.value.adr.ptr._obj._TYPE._name
+ if self._fields is None:
+ return '' % (cls_name,)
field_names = [field.name for field in self._fields]
return "" % (cls_name, field_names)
From cfbolz at codespeak.net Sun Oct 31 14:48:05 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sun, 31 Oct 2010 14:48:05 +0100 (CET)
Subject: [pypy-svn] r78606 - pypy/extradoc/sprintinfo/ddorf2010
Message-ID: <20101031134805.A0E345080E@codespeak.net>
Author: cfbolz
Date: Sun Oct 31 14:48:04 2010
New Revision: 78606
Added:
pypy/extradoc/sprintinfo/ddorf2010/report.rst
Log:
draft of a sprint report
Added: pypy/extradoc/sprintinfo/ddorf2010/report.rst
==============================================================================
--- (empty file)
+++ pypy/extradoc/sprintinfo/ddorf2010/report.rst Sun Oct 31 14:48:04 2010
@@ -0,0 +1,62 @@
+D?sseldorf Sprint Report 2010
+==============================
+
+This years installment of the yearly PyPy D?sseldorf Sprint is drawing to a
+close. As usual, we worked in the seminar room of the `programming language
+group`_ at the University of D?sseldorf. The sprint was different from previous
+ones in that we had fewer people than usual and many actually live in
+D?sseldorf all the time.
+
+David spent the sprint working on the `arm-backend`_ branch, which is adding an
+ARM backend to the JIT. With the help of Armin he added support for bridges in
+the JIT and generally implemented missing operations, mostly for handling integers.
+
+.. _`arm-backend`: http://codespeak.net/svn/pypy/branch/arm-backend/
+
+Ronny and Anto worked the whole week trying to come up with a scheme for
+importing PyPy's SVN history into a mercurial repository without loosing too
+much information. This is a non-trivial task, because PyPy's history is gnarly.
+We are nearly at revision 79000 and when we started using it, Subversion was at
+version 0.1. All possible and impossible ways to mangle and mistreat a
+Subversion repository have been applied to PyPy's repo, so most of the
+importing tools just give up. Ronny and Anto came up with a new plan and new
+helper scripts every day, only to then discover another corner case that they
+hadn't thought of. Now they might actually have a final plan (but they said
+that every day, so who knows?).
+
+Carl Friedrich and Lukas started working in earnest on memory benchmarks to
+understand the memory behaviour of Python code better. They have now
+implemented a generic memory benchmark runner and a simple analysis that walks
+all objectsf and collects size information about them. They also added some
+benchmarks that were proposed in the comments of the recent `call for
+benchmarks`_. As soon as some results from that work are there, we will post
+about them.
+
+.. _`call for benchmarks`: http://morepypy.blogspot.com/2010/08/call-for-benchmarks.html
+
+There were also some minor tasks performed during the sprint. Armin implemented
+the ``_bisect`` module and the ``dict.popitem`` method in RPython. Armin and
+Carl Friedrich made the new memory-saving mapdict implementation more suitable
+to use without the JIT (blog post should come about that too, at some point).
+They also made classes with custom metaclasses a lot faster when the JIT is
+used.
+
+The last three days of the sprint were spent working on H?kan's
+`jit-unroll-loops`_ branch. The branch is meant to move loop invariants out of
+the loop, using techniques very similar to what is described in the recent post
+on `escape analysis across loop boundaries`_ (see? it will soon stop being
+science-fiction). Some of the ideas of this approach also come from LuaJIT_
+which also uses very aggressive loop invariant code motion in its optimizers.
+Moving loop invariants outside of the loop is very useful, because many of the
+lookups that Python programs do in loops are loop invariants. An example is if
+you call a function in a loop: The global lookup can often be done only once.
+
+This branch fundamentally changes some of the core assumptions of the JIT, so
+it is a huge amount of work to make it fit with all the other parts and to
+adapt all tests. That work is now nearly done, some failing tests remain. The
+next steps are to fix them and then do additional tests with the translated
+executable and look at the benchmarks.
+
+.. _`jit-unroll-loops`: http://codespeak.net/svn/pypy/branch/jit-unroll-loops/
+.. _`escape analysis across loop boundaries`: http://morepypy.blogspot.com/2010/09/using-escape-analysis-across-loop.html
+.. _LuaJIT: http://luajit.org/
From arigo at codespeak.net Sun Oct 31 14:48:17 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 14:48:17 +0100 (CET)
Subject: [pypy-svn] r78607 - pypy/branch/jit-unroll-loops/pypy/jit/metainterp
Message-ID: <20101031134817.28141282BE9@codespeak.net>
Author: arigo
Date: Sun Oct 31 14:48:15 2010
New Revision: 78607
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py
Log:
Make "snapshot" a local var, allowing us to inspect it with pdb
even after the function cleared it.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py Sun Oct 31 14:48:15 2010
@@ -299,9 +299,9 @@
storage = self.storage
# make sure that nobody attached resume data to this guard yet
assert storage.rd_numb is None
- assert storage.rd_snapshot is not None # is that true?
- numb, liveboxes_from_env, v = self.memo.number(values,
- storage.rd_snapshot)
+ snapshot = storage.rd_snapshot
+ assert snapshot is not None # is that true?
+ numb, liveboxes_from_env, v = self.memo.number(values, snapshot)
self.liveboxes_from_env = liveboxes_from_env
self.liveboxes = {}
storage.rd_numb = numb
From arigo at codespeak.net Sun Oct 31 14:51:31 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 14:51:31 +0100 (CET)
Subject: [pypy-svn] r78608 - in
pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test
Message-ID: <20101031135131.8E0FB5080F@codespeak.net>
Author: arigo
Date: Sun Oct 31 14:51:29 2010
New Revision: 78608
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
Log:
Mostly fix the tests by disabling them, which makes sense right now
because _verify_fail_args() would need to map from old to new box names
to inspect the loop produced by unroll.py.
Fix another piece of data that must not be propagated from the preamble
to the main loop.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Sun Oct 31 14:51:29 2010
@@ -227,6 +227,7 @@
self.optimizations = optimizations
def force_at_end_of_preamble(self):
+ self.resumedata_memo = resume.ResumeDataLoopMemo(self.metainterp_sd)
for o in self.optimizations:
o.force_at_end_of_preamble()
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sun Oct 31 14:51:29 2010
@@ -100,15 +100,27 @@
class FailDescr(compile.ResumeGuardDescr):
oparse = None
def _oparser_uses_descr_of_guard(self, oparse, fail_args):
- # typically called twice, before and after optimization
- if self.oparse is None:
- self.rd_frame_info_list = resume.FrameInfo(None,
- "code", 11)
- self.rd_snapshot = resume.Snapshot(None, fail_args)
+ # typically called 3 times: once when parsing 'ops',
+ # once when parsing 'preamble', once when parsing 'expected'.
self.oparse = oparse
+ self.rd_frame_info_list, self.rd_snapshot = snapshot(fail_args)
def _clone_if_mutable(self):
assert self is fdescr
return fdescr2
+ def __repr__(self):
+ if self is fdescr:
+ return 'fdescr'
+ if self is fdescr2:
+ return 'fdescr2'
+ return compile.ResumeGuardDescr.__repr__(self)
+ #
+ def snapshot(fail_args, got=[]):
+ if not got: # only the first time, i.e. when parsing 'ops'
+ rd_frame_info_list = resume.FrameInfo(None, "code", 11)
+ rd_snapshot = resume.Snapshot(None, fail_args)
+ got.append(rd_frame_info_list)
+ got.append(rd_snapshot)
+ return got
#
fdescr = instantiate(FailDescr)
self.namespace['fdescr'] = fdescr
@@ -2161,7 +2173,7 @@
jump(p2, i0, i1, i3)
"""
self.optimize_loop(ops, expected, preamble)
- self.check_expanded_fail_descr("i0", rop.GUARD_NONNULL_CLASS)
+ #self.check_expanded_fail_descr("i0", rop.GUARD_NONNULL_CLASS)
def test_merge_guard_nonnull_guard_value(self):
ops = """
@@ -2184,7 +2196,7 @@
jump(ConstPtr(myptr), i0, i1, i3)
"""
self.optimize_loop(ops, expected, preamble)
- self.check_expanded_fail_descr("i0", rop.GUARD_VALUE)
+ #self.check_expanded_fail_descr("i0", rop.GUARD_VALUE)
def test_merge_guard_nonnull_guard_class_guard_value(self):
ops = """
@@ -2211,7 +2223,7 @@
jump(ConstPtr(myptr), i0, i1, i4)
"""
self.optimize_loop(ops, expected, preamble)
- self.check_expanded_fail_descr("i0", rop.GUARD_VALUE)
+ #self.check_expanded_fail_descr("i0", rop.GUARD_VALUE)
def test_guard_class_oois(self):
ops = """
@@ -2435,113 +2447,6 @@
# ----------
- def _verify_fail_args(self, boxes, oparse, text):
- import re
- r = re.compile(r"\bwhere\s+(\w+)\s+is a\s+(\w+)")
- parts = list(r.finditer(text))
- ends = [match.start() for match in parts] + [len(text)]
- #
- virtuals = {}
- for match, end in zip(parts, ends[1:]):
- pvar = match.group(1)
- fieldstext = text[match.end():end]
- if match.group(2) == 'varray':
- arrayname, fieldstext = fieldstext.split(':', 1)
- tag = ('varray', self.namespace[arrayname.strip()])
- elif match.group(2) == 'vstruct':
- if ',' in fieldstext:
- structname, fieldstext = fieldstext.split(',', 1)
- else:
- structname, fieldstext = fieldstext, ''
- tag = ('vstruct', self.namespace[structname.strip()])
- else:
- tag = ('virtual', self.namespace[match.group(2)])
- virtuals[pvar] = (tag, None, fieldstext)
- #
- r2 = re.compile(r"([\w\d()]+)[.](\w+)\s*=\s*([\w\d()]+)")
- pendingfields = []
- for match in r2.finditer(text):
- pvar = match.group(1)
- pfieldname = match.group(2)
- pfieldvar = match.group(3)
- pendingfields.append((pvar, pfieldname, pfieldvar))
- #
- def _variables_equal(box, varname, strict):
- if varname not in virtuals:
- if strict:
- assert box == oparse.getvar(varname)
- else:
- assert box.value == oparse.getvar(varname).value
- else:
- tag, resolved, fieldstext = virtuals[varname]
- if tag[0] == 'virtual':
- assert self.get_class_of_box(box) == tag[1]
- elif tag[0] == 'varray':
- pass # xxx check arraydescr
- elif tag[0] == 'vstruct':
- pass # xxx check typedescr
- else:
- assert 0
- if resolved is not None:
- assert resolved.value == box.value
- else:
- virtuals[varname] = tag, box, fieldstext
- #
- basetext = text.splitlines()[0]
- varnames = [s.strip() for s in basetext.split(',')]
- if varnames == ['']:
- varnames = []
- assert len(boxes) == len(varnames)
- for box, varname in zip(boxes, varnames):
- _variables_equal(box, varname, strict=True)
- for pvar, pfieldname, pfieldvar in pendingfields:
- box = oparse.getvar(pvar)
- fielddescr = self.namespace[pfieldname.strip()]
- fieldbox = executor.execute(self.cpu, None,
- rop.GETFIELD_GC,
- fielddescr,
- box)
- _variables_equal(fieldbox, pfieldvar, strict=True)
- #
- for match in parts:
- pvar = match.group(1)
- tag, resolved, fieldstext = virtuals[pvar]
- assert resolved is not None
- index = 0
- for fieldtext in fieldstext.split(','):
- fieldtext = fieldtext.strip()
- if not fieldtext:
- continue
- if tag[0] in ('virtual', 'vstruct'):
- fieldname, fieldvalue = fieldtext.split('=')
- fielddescr = self.namespace[fieldname.strip()]
- fieldbox = executor.execute(self.cpu, None,
- rop.GETFIELD_GC,
- fielddescr,
- resolved)
- elif tag[0] == 'varray':
- fieldvalue = fieldtext
- fieldbox = executor.execute(self.cpu, None,
- rop.GETARRAYITEM_GC,
- tag[1],
- resolved, ConstInt(index))
- else:
- assert 0
- _variables_equal(fieldbox, fieldvalue.strip(), strict=False)
- index += 1
-
- def check_expanded_fail_descr(self, expectedtext, guard_opnum):
- from pypy.jit.metainterp.test.test_resume import ResumeDataFakeReader
- from pypy.jit.metainterp.test.test_resume import MyMetaInterp
- guard_op, = [op for op in self.loop.operations if op.is_guard()]
- fail_args = guard_op.getfailargs()
- fdescr = guard_op.getdescr()
- assert fdescr.guard_opnum == guard_opnum
- reader = ResumeDataFakeReader(fdescr, fail_args,
- MyMetaInterp(self.cpu))
- boxes = reader.consume_boxes()
- self._verify_fail_args(boxes, fdescr.oparse, expectedtext)
-
class TestLLtype(OptimizeOptTest, LLtypeMixin):
def test_residual_call_does_not_invalidate_caches(self):
@@ -2765,6 +2670,8 @@
'''
self.optimize_loop(ops, expected, preamble)
+ # ----------
+
def test_vref_nonvirtual_nonescape(self):
ops = """
[p1]
@@ -2777,7 +2684,7 @@
i0 = force_token()
jump(p1)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_loop(ops, expected, expected)
def test_vref_nonvirtual_escape(self):
ops = """
@@ -2800,7 +2707,7 @@
"""
# XXX we should optimize a bit more the case of a nonvirtual.
# in theory it is enough to just do 'p2 = p1'.
- self.optimize_loop(ops, expected)
+ self.optimize_loop(ops, expected, expected)
def test_vref_virtual_1(self):
ops = """
@@ -2840,7 +2747,7 @@
setfield_gc(p2, -3, descr=virtualtokendescr)
jump(p0, i1)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_loop(ops, expected, expected)
def test_vref_virtual_2(self):
ops = """
@@ -2869,7 +2776,7 @@
setfield_gc(p0, p2, descr=nextdescr)
#
call_may_force(i1, descr=mayforcevirtdescr)
- guard_not_forced(descr=fdescr) [p2, i1]
+ guard_not_forced(descr=fdescr2) [p2, i1]
#
setfield_gc(p0, NULL, descr=nextdescr)
p1 = new_with_vtable(ConstClass(node_vtable))
@@ -2883,10 +2790,10 @@
# the point of this test is that 'i1' should show up in the fail_args
# of 'guard_not_forced', because it was stored in the virtual 'p1b'.
self.optimize_loop(ops, expected)
- self.check_expanded_fail_descr('''p2, p1
- where p1 is a node_vtable, nextdescr=p1b
- where p1b is a node_vtable, valuedescr=i1
- ''', rop.GUARD_NOT_FORCED)
+ #self.check_expanded_fail_descr('''p2, p1
+ # where p1 is a node_vtable, nextdescr=p1b
+ # where p1b is a node_vtable, valuedescr=i1
+ # ''', rop.GUARD_NOT_FORCED)
def test_vref_virtual_and_lazy_setfield(self):
ops = """
@@ -2905,7 +2812,7 @@
setfield_gc(p0, NULL, descr=refdescr)
jump(p0, i1)
"""
- expected = """
+ preamble = """
[p0, i1]
i3 = force_token()
call(i1, descr=nonwritedescr)
@@ -2913,18 +2820,26 @@
setfield_gc(p0, NULL, descr=refdescr)
jump(p0, i1)
"""
- self.optimize_loop(ops, expected)
+ expected = """
+ [p0, i1]
+ i3 = force_token()
+ call(i1, descr=nonwritedescr)
+ guard_no_exception(descr=fdescr2) [i3, i1, p0]
+ setfield_gc(p0, NULL, descr=refdescr)
+ jump(p0, i1)
+ """
+ self.optimize_loop(ops, expected, preamble)
# the fail_args contain [i3, i1, p0]:
# - i3 is from the virtual expansion of p2
# - i1 is from the virtual expansion of p1
# - p0 is from the extra pendingfields
- self.loop.inputargs[0].value = self.nodeobjvalue
- self.check_expanded_fail_descr('''p2, p1
- p0.refdescr = p2
- where p2 is a jit_virtual_ref_vtable, virtualtokendescr=i3, virtualrefindexdescr=2
- where p1 is a node_vtable, nextdescr=p1b
- where p1b is a node_vtable, valuedescr=i1
- ''', rop.GUARD_NO_EXCEPTION)
+ #self.loop.inputargs[0].value = self.nodeobjvalue
+ #self.check_expanded_fail_descr('''p2, p1
+ # p0.refdescr = p2
+ # where p2 is a jit_virtual_ref_vtable, virtualtokendescr=i3, virtualrefindexdescr=2
+ # where p1 is a node_vtable, nextdescr=p1b
+ # where p1b is a node_vtable, valuedescr=i1
+ # ''', rop.GUARD_NO_EXCEPTION)
def test_vref_virtual_after_finish(self):
ops = """
@@ -2951,7 +2866,7 @@
guard_not_forced() []
jump(i1)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_loop(ops, expected, expected)
def test_vref_nonvirtual_and_lazy_setfield(self):
ops = """
@@ -2976,7 +2891,9 @@
guard_not_forced() [i1]
jump(i1, p1)
"""
- self.optimize_loop(ops, expected)
+ self.optimize_loop(ops, expected, expected)
+
+ # ----------
def test_arraycopy_1(self):
ops = '''
From hakanardo at codespeak.net Sun Oct 31 14:58:21 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sun, 31 Oct 2010 14:58:21 +0100 (CET)
Subject: [pypy-svn] r78609 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031135821.32131282B9E@codespeak.net>
Author: hakanardo
Date: Sun Oct 31 14:58:19 2010
New Revision: 78609
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop.py
Log:
fixed test
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop.py Sun Oct 31 14:58:19 2010
@@ -397,8 +397,12 @@
res = self.meta_interp(f, [100, 5], policy=StopAtXPolicy(externfn))
assert res == expected
- self.check_loop_count(2)
- self.check_tree_loop_count(2) # 1 loop, 1 bridge from interp
+ if self.optimizer == OPTIMIZER_FULL:
+ self.check_loop_count(2)
+ self.check_tree_loop_count(2) # 1 loop, 1 bridge from interp
+ else:
+ self.check_loop_count(2)
+ self.check_tree_loop_count(1) # 1 loop, 1 bridge from interp
def test_example(self):
myjitdriver = JitDriver(greens = ['i'],
From hakanardo at codespeak.net Sun Oct 31 15:00:10 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sun, 31 Oct 2010 15:00:10 +0100 (CET)
Subject: [pypy-svn] r78610 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031140010.81B9B36E0C1@codespeak.net>
Author: hakanardo
Date: Sun Oct 31 15:00:03 2010
New Revision: 78610
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop.py
Log:
fixed comment
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop.py Sun Oct 31 15:00:03 2010
@@ -402,7 +402,7 @@
self.check_tree_loop_count(2) # 1 loop, 1 bridge from interp
else:
self.check_loop_count(2)
- self.check_tree_loop_count(1) # 1 loop, 1 bridge from interp
+ self.check_tree_loop_count(1) # 1 loop, callable from the interp
def test_example(self):
myjitdriver = JitDriver(greens = ['i'],
From arigo at codespeak.net Sun Oct 31 15:00:46 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 15:00:46 +0100 (CET)
Subject: [pypy-svn] r78611 - pypy/branch/jit-unroll-loops/pypy/jit/metainterp
Message-ID: <20101031140046.666E6282B9E@codespeak.net>
Author: arigo
Date: Sun Oct 31 15:00:44 2010
New Revision: 78611
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py
Log:
Stick the name "Preample Loop" on the preamble loops.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py Sun Oct 31 15:00:44 2010
@@ -35,9 +35,9 @@
extraloops = [loop]
metainterp_sd.stats.view(errmsg=errmsg, extraloops=extraloops)
-def create_empty_loop(metainterp):
+def create_empty_loop(metainterp, name_prefix=''):
name = metainterp.staticdata.stats.name_for_new_loop()
- return TreeLoop(name)
+ return TreeLoop(name_prefix + name)
def make_loop_token(nb_args, jitdriver_sd):
loop_token = LoopToken()
@@ -65,7 +65,7 @@
loop.token = loop_token
loop.operations[-1].setdescr(loop_token) # patch the target of the JUMP
- loop.preamble = create_empty_loop(metainterp)
+ loop.preamble = create_empty_loop(metainterp, 'Preamble ')
loop.preamble.greenkey = greenkey
loop.preamble.inputargs = loop.inputargs
loop.preamble.token = make_loop_token(len(loop.inputargs), jitdriver_sd)
From cfbolz at codespeak.net Sun Oct 31 15:11:26 2010
From: cfbolz at codespeak.net (cfbolz at codespeak.net)
Date: Sun, 31 Oct 2010 15:11:26 +0100 (CET)
Subject: [pypy-svn] r78612 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031141126.036A636E0C3@codespeak.net>
Author: cfbolz
Date: Sun Oct 31 15:11:25 2010
New Revision: 78612
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py
Log:
revert these numbers to how they were on trunk
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py Sun Oct 31 15:11:25 2010
@@ -87,11 +87,11 @@
loop_tokens = []
loop_token = compile_new_loop(metainterp, loop_tokens, [], 0)
assert loop_tokens == [loop_token]
- assert loop_token.number == 2
- assert staticdata.globaldata.loopnumbering == 3
+ assert loop_token.number == 1
+ assert staticdata.globaldata.loopnumbering == 2
#
- assert len(cpu.seen) == 2
- assert cpu.seen[1][2] == loop_token
+ assert len(cpu.seen) == 1
+ assert cpu.seen[0][2] == loop_token
#
del cpu.seen[:]
metainterp = FakeMetaInterp()
@@ -105,7 +105,7 @@
assert loop_token_2 is loop_token
assert loop_tokens == [loop_token]
assert len(cpu.seen) == 0
- assert staticdata.globaldata.loopnumbering == 3
+ assert staticdata.globaldata.loopnumbering == 2
def test_resume_guard_counters():
From arigo at codespeak.net Sun Oct 31 15:14:07 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 15:14:07 +0100 (CET)
Subject: [pypy-svn] r78613 - pypy/extradoc/sprintinfo/ddorf2010
Message-ID: <20101031141407.DFCEB282BE3@codespeak.net>
Author: arigo
Date: Sun Oct 31 15:14:06 2010
New Revision: 78613
Modified:
pypy/extradoc/sprintinfo/ddorf2010/report.rst
Log:
Typos.
Modified: pypy/extradoc/sprintinfo/ddorf2010/report.rst
==============================================================================
--- pypy/extradoc/sprintinfo/ddorf2010/report.rst (original)
+++ pypy/extradoc/sprintinfo/ddorf2010/report.rst Sun Oct 31 15:14:06 2010
@@ -9,7 +9,7 @@
David spent the sprint working on the `arm-backend`_ branch, which is adding an
ARM backend to the JIT. With the help of Armin he added support for bridges in
-the JIT and generally implemented missing operations, mostly for handling integers.
+the JIT and generally implemented missing operations, mostly for handling integers so far.
.. _`arm-backend`: http://codespeak.net/svn/pypy/branch/arm-backend/
@@ -27,7 +27,7 @@
Carl Friedrich and Lukas started working in earnest on memory benchmarks to
understand the memory behaviour of Python code better. They have now
implemented a generic memory benchmark runner and a simple analysis that walks
-all objectsf and collects size information about them. They also added some
+all objects and collects size information about them. They also added some
benchmarks that were proposed in the comments of the recent `call for
benchmarks`_. As soon as some results from that work are there, we will post
about them.
From hakanardo at codespeak.net Sun Oct 31 15:14:11 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sun, 31 Oct 2010 15:14:11 +0100 (CET)
Subject: [pypy-svn] r78614 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031141411.38CBF282BF1@codespeak.net>
Author: hakanardo
Date: Sun Oct 31 15:14:09 2010
New Revision: 78614
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send_simple.py
Log:
fixed test
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py Sun Oct 31 15:14:09 2010
@@ -2,10 +2,11 @@
from pypy.rlib.jit import JitDriver, hint, purefunction
from pypy.jit.codewriter.policy import StopAtXPolicy
from pypy.jit.metainterp.test.test_basic import LLJitMixin, OOJitMixin
-
+from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_SIMPLE
class SendTests:
-
+ optimizer=OPTIMIZER_FULL
+
def test_green_send(self):
myjitdriver = JitDriver(greens = ['i'], reds = ['counter'])
lst = ["123", "45"]
@@ -164,8 +165,12 @@
for j in range(69, 75):
res = self.meta_interp(f, [j], policy=policy)
assert res == 42
- self.check_enter_count(5)
- self.check_loop_count(5)
+ if self.optimizer == OPTIMIZER_SIMPLE:
+ self.check_enter_count(3)
+ self.check_loop_count(3)
+ else:
+ self.check_enter_count(5)
+ self.check_loop_count(5)
def test_oosend_guard_failure(self):
myjitdriver = JitDriver(greens = [], reds = ['x', 'y', 'w'])
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send_simple.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send_simple.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send_simple.py Sun Oct 31 15:14:09 2010
@@ -6,8 +6,9 @@
from pypy.jit.metainterp.test.test_basic import LLJitMixin, OOJitMixin
class LoopDummyTest(test_send.SendTests):
+ optimizer=OPTIMIZER_SIMPLE
def meta_interp(self, func, args, **kwds):
- return ll_meta_interp(func, args, optimizer=OPTIMIZER_SIMPLE,
+ return ll_meta_interp(func, args, optimizer=self.optimizer,
CPUClass=self.CPUClass,
type_system=self.type_system,
**kwds)
From hakanardo at codespeak.net Sun Oct 31 15:20:44 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sun, 31 Oct 2010 15:20:44 +0100 (CET)
Subject: [pypy-svn] r78615 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031142044.1E022282BDA@codespeak.net>
Author: hakanardo
Date: Sun Oct 31 15:20:42 2010
New Revision: 78615
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py
Log:
fixed test
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py Sun Oct 31 15:20:42 2010
@@ -353,7 +353,10 @@
assert res == f(198)
# we get two TreeLoops: an initial one, and one entering from
# the interpreter
- self.check_tree_loop_count(2)
+ if self.optimizer == OPTIMIZER_SIMPLE:
+ self.check_tree_loop_count(1)
+ else:
+ self.check_tree_loop_count(2)
def test_two_behaviors(self):
py.test.skip("XXX fix me!!!!!!! problem in optimize.py")
From arigo at codespeak.net Sun Oct 31 15:28:03 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 15:28:03 +0100 (CET)
Subject: [pypy-svn] r78616 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031142803.230FC36E0CB@codespeak.net>
Author: arigo
Date: Sun Oct 31 15:28:01 2010
New Revision: 78616
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualizable.py
Log:
fix this test.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualizable.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualizable.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualizable.py Sun Oct 31 15:28:01 2010
@@ -197,7 +197,8 @@
return xy.inst_x
res = self.meta_interp(f, [20])
assert res == 134
- self.check_loops(getfield_gc=1, setfield_gc=1, everywhere=True)
+ self.check_loops(getfield_gc=0, setfield_gc=1)
+ self.check_loops(getfield_gc=1, setfield_gc=2, everywhere=True)
# ------------------------------
From arigo at codespeak.net Sun Oct 31 15:39:17 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 15:39:17 +0100 (CET)
Subject: [pypy-svn] r78617 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031143917.B7EF1282BAD@codespeak.net>
Author: arigo
Date: Sun Oct 31 15:39:16 2010
New Revision: 78617
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py
Log:
Raah.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py Sun Oct 31 15:39:16 2010
@@ -575,7 +575,7 @@
result += f('-c-----------l-', i+100)
self.meta_interp(g, [10], backendopt=True)
self.check_aborted_count(1)
- self.check_history(call_assembler=1, call=0)
+ self.check_loops(call_assembler=1, call=0)
self.check_tree_loop_count(3)
def test_directly_call_assembler(self):
From hakanardo at codespeak.net Sun Oct 31 15:39:53 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sun, 31 Oct 2010 15:39:53 +0100 (CET)
Subject: [pypy-svn] r78618 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031143953.5799036E0CD@codespeak.net>
Author: hakanardo
Date: Sun Oct 31 15:39:51 2010
New Revision: 78618
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py
Log:
fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py Sun Oct 31 15:39:51 2010
@@ -409,7 +409,10 @@
# we expect 1 loop, 1 entry bridge, and 1 bridge going from the
# loop back to the start of the entry bridge
self.check_loop_count(2) # 1 loop + 1 bridge
- self.check_tree_loop_count(2) # 1 loop + 1 entry bridge (argh)
+ if self.optimizer == OPTIMIZER_SIMPLE:
+ self.check_tree_loop_count(1) # 1 loop
+ else:
+ self.check_tree_loop_count(2) # 1 loop + 1 entry bridge (argh)
self.check_aborted_count(0)
def test_three_cases(self):
@@ -430,7 +433,10 @@
return node.x
res = self.meta_interp(f, [55])
assert res == f(55)
- self.check_tree_loop_count(2)
+ if self.optimizer == OPTIMIZER_SIMPLE:
+ self.check_tree_loop_count(1)
+ else:
+ self.check_tree_loop_count(2)
def test_three_classes(self):
class Base:
@@ -460,7 +466,10 @@
return n
res = self.meta_interp(f, [55], policy=StopAtXPolicy(extern))
assert res == f(55)
- self.check_tree_loop_count(2)
+ if self.optimizer == OPTIMIZER_SIMPLE:
+ self.check_tree_loop_count(1)
+ else:
+ self.check_tree_loop_count(2)
def test_bug1(self):
myjitdriver = JitDriver(greens = [], reds = ['n', 'node'])
From hakanardo at codespeak.net Sun Oct 31 15:43:50 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sun, 31 Oct 2010 15:43:50 +0100 (CET)
Subject: [pypy-svn] r78619 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031144350.EC0ED282BAD@codespeak.net>
Author: hakanardo
Date: Sun Oct 31 15:43:49 2010
New Revision: 78619
Added:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send_nounroll.py (props changed)
- copied unchanged from r78618, pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_nounroll.py
Removed:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_nounroll.py
Log:
rename
From hakanardo at codespeak.net Sun Oct 31 15:52:39 2010
From: hakanardo at codespeak.net (hakanardo at codespeak.net)
Date: Sun, 31 Oct 2010 15:52:39 +0100 (CET)
Subject: [pypy-svn] r78620 -
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test
Message-ID: <20101031145239.E18BD36E0CB@codespeak.net>
Author: hakanardo
Date: Sun Oct 31 15:52:38 2010
New Revision: 78620
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send_nounroll.py
Log:
fixed tests
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py Sun Oct 31 15:52:38 2010
@@ -165,7 +165,7 @@
for j in range(69, 75):
res = self.meta_interp(f, [j], policy=policy)
assert res == 42
- if self.optimizer == OPTIMIZER_SIMPLE:
+ if self.optimizer != OPTIMIZER_FULL:
self.check_enter_count(3)
self.check_loop_count(3)
else:
@@ -353,7 +353,7 @@
assert res == f(198)
# we get two TreeLoops: an initial one, and one entering from
# the interpreter
- if self.optimizer == OPTIMIZER_SIMPLE:
+ if self.optimizer != OPTIMIZER_FULL:
self.check_tree_loop_count(1)
else:
self.check_tree_loop_count(2)
@@ -409,7 +409,7 @@
# we expect 1 loop, 1 entry bridge, and 1 bridge going from the
# loop back to the start of the entry bridge
self.check_loop_count(2) # 1 loop + 1 bridge
- if self.optimizer == OPTIMIZER_SIMPLE:
+ if self.optimizer != OPTIMIZER_FULL:
self.check_tree_loop_count(1) # 1 loop
else:
self.check_tree_loop_count(2) # 1 loop + 1 entry bridge (argh)
@@ -433,7 +433,7 @@
return node.x
res = self.meta_interp(f, [55])
assert res == f(55)
- if self.optimizer == OPTIMIZER_SIMPLE:
+ if self.optimizer != OPTIMIZER_FULL:
self.check_tree_loop_count(1)
else:
self.check_tree_loop_count(2)
@@ -466,7 +466,7 @@
return n
res = self.meta_interp(f, [55], policy=StopAtXPolicy(extern))
assert res == f(55)
- if self.optimizer == OPTIMIZER_SIMPLE:
+ if self.optimizer != OPTIMIZER_FULL:
self.check_tree_loop_count(1)
else:
self.check_tree_loop_count(2)
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send_nounroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send_nounroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send_nounroll.py Sun Oct 31 15:52:38 2010
@@ -5,8 +5,10 @@
from pypy.jit.metainterp.test.test_basic import LLJitMixin, OOJitMixin
class LoopNoPSpecTest(test_send.SendTests):
+ optimizer=OPTIMIZER_NO_UNROLL
+
def meta_interp(self, func, args, **kwds):
- return ll_meta_interp(func, args, optimizer=OPTIMIZER_NO_UNROLL,
+ return ll_meta_interp(func, args, optimizer=self.optimizer,
CPUClass=self.CPUClass,
type_system=self.type_system,
**kwds)
@@ -22,9 +24,7 @@
class TestLLtype(LoopNoPSpecTest, LLJitMixin):
- def check_tree_loop_count(self, count):
- # we get one less entry bridge
- return LLJitMixin.check_tree_loop_count(self, count - 1)
+ pass
class TestOOtype(LoopNoPSpecTest, OOJitMixin):
pass
From arigo at codespeak.net Sun Oct 31 15:52:58 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 15:52:58 +0100 (CET)
Subject: [pypy-svn] r78621 -
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test
Message-ID: <20101031145258.94810282B90@codespeak.net>
Author: arigo
Date: Sun Oct 31 15:52:57 2010
New Revision: 78621
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_jump.py
Log:
(hakanardo, arigo) Kill this test. The problem was actually that
the dstvalues list contains duplicates.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_jump.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_jump.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_jump.py Sun Oct 31 15:52:57 2010
@@ -142,22 +142,3 @@
('push', s12),
('mov', ebx, s12),
('pop', ebx)]
-def test_crash():
- assembler = MockAssembler()
- dstvalues = [-56, -64, -72, -80, -88, 15, 14, 15, 13, 12, 15, -96,
- -104, -112, 10, 9, 8, 7, 6, 3, -120, 2, 1]
- srcvalues = [-56, -64, -72, -80, -88, -224, -216, -224, -248, -232,
- -224, -96, -104, -112, -184, -192, -264, 2, -312, -344,
- 10, 8, 14]
- print len(dstvalues), len(srcvalues)
- class MyLocation(AssemblerLocation):
- def __init__(self, value):
- self.value = value
- def location_code(self):
- return 'b'
-
- src = [MyLocation(i) for i in srcvalues]
- dst = [MyLocation(i) for i in dstvalues]
-
- remap_frame_layout(assembler, src, dst, '?')
- assert assembler.ops == []
From arigo at codespeak.net Sun Oct 31 15:56:43 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 15:56:43 +0100 (CET)
Subject: [pypy-svn] r78622 -
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86
Message-ID: <20101031145643.9B57236E35D@codespeak.net>
Author: arigo
Date: Sun Oct 31 15:56:42 2010
New Revision: 78622
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/jump.py
Log:
Add an assert that complains explicitly.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/jump.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/jump.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/jump.py Sun Oct 31 15:56:42 2010
@@ -7,7 +7,9 @@
srccount = {} # maps dst_locations to how many times the same
# location appears in src_locations
for dst in dst_locations:
- srccount[dst._getregkey()] = 0
+ key = dst._getregkey()
+ assert key not in srccount, "duplicate value in dst_locations!"
+ srccount[key] = 0
for i in range(len(dst_locations)):
src = src_locations[i]
if isinstance(src, ImmedLoc):
From arigo at codespeak.net Sun Oct 31 16:02:35 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 16:02:35 +0100 (CET)
Subject: [pypy-svn] r78623 - in pypy/branch/jit-unroll-loops/pypy/jit:
backend/x86/test metainterp/test
Message-ID: <20101031150235.2A463282BD6@codespeak.net>
Author: arigo
Date: Sun Oct 31 16:02:33 2010
New Revision: 78623
Added:
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_loop_unroll.py
- copied, changed from r78619, pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_loop_spec.py
Removed:
pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_loop_spec.py
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_unroll.py
Log:
Rename.
Copied: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_loop_unroll.py (from r78619, pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_loop_spec.py)
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_loop_spec.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_loop_unroll.py Sun Oct 31 16:02:33 2010
@@ -1,8 +1,8 @@
import py
from pypy.jit.backend.x86.test.test_basic import Jit386Mixin
-from pypy.jit.metainterp.test import test_loop_spec
+from pypy.jit.metainterp.test import test_loop_unroll
-class TestLoopSpec(Jit386Mixin, test_loop_spec.LoopSpecTest):
+class TestLoopSpec(Jit386Mixin, test_loop_unroll.LoopUnrollTest):
# for the individual tests see
# ====> ../../../metainterp/test/test_loop.py
pass
Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_unroll.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_unroll.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop_unroll.py Sun Oct 31 16:02:33 2010
@@ -3,7 +3,7 @@
from pypy.jit.metainterp.test import test_loop
from pypy.jit.metainterp.test.test_basic import LLJitMixin, OOJitMixin
-class LoopSpecTest(test_loop.LoopTest):
+class LoopUnrollTest(test_loop.LoopTest):
optimizer = OPTIMIZER_FULL
automatic_promotion_result = {
'int_add' : 3, 'int_gt' : 1, 'guard_false' : 1, 'jump' : 1,
@@ -11,8 +11,8 @@
# ====> test_loop.py
-class TestLLtype(LoopSpecTest, LLJitMixin):
+class TestLLtype(LoopUnrollTest, LLJitMixin):
pass
-class TestOOtype(LoopSpecTest, OOJitMixin):
+class TestOOtype(LoopUnrollTest, OOJitMixin):
pass
From arigo at codespeak.net Sun Oct 31 16:47:20 2010
From: arigo at codespeak.net (arigo at codespeak.net)
Date: Sun, 31 Oct 2010 16:47:20 +0100 (CET)
Subject: [pypy-svn] r78625 - pypy/branch/jit-unroll-loops/pypy/jit/tool/test
Message-ID: <20101031154720.60B56282BDA@codespeak.net>
Author: arigo
Date: Sun Oct 31 16:47:18 2010
New Revision: 78625
Modified:
pypy/branch/jit-unroll-loops/pypy/jit/tool/test/test_jitoutput.py
Log:
Fix.
Modified: pypy/branch/jit-unroll-loops/pypy/jit/tool/test/test_jitoutput.py
==============================================================================
--- pypy/branch/jit-unroll-loops/pypy/jit/tool/test/test_jitoutput.py (original)
+++ pypy/branch/jit-unroll-loops/pypy/jit/tool/test/test_jitoutput.py Sun Oct 31 16:47:18 2010
@@ -33,13 +33,13 @@
assert info.tracing_no == 1
assert info.asm_no == 1
assert info.blackhole_no == 1
- assert info.backend_no == 1
+ assert info.backend_no == 2
assert info.ops.total == 2
assert info.recorded_ops.total == 2
assert info.recorded_ops.calls == 0
assert info.guards == 1
- assert info.opt_ops == 6
- assert info.opt_guards == 1
+ assert info.opt_ops == 11
+ assert info.opt_guards == 2
assert info.forcings == 0
DATA = '''Tracing: 1 0.006992
From fijal at codespeak.net Sun Oct 31 16:55:54 2010
From: fijal at codespeak.net (fijal at codespeak.net)
Date: Sun, 31 Oct 2010 16:55:54 +0100 (CET)
Subject: [pypy-svn] r78626 - pypy/extradoc/talk/pycon2011
Message-ID: <20101031155554.C3F82282BDB@codespeak.net>
Author: fijal
Date: Sun Oct 31 16:55:53 2010
New Revision: 78626
Modified:
pypy/extradoc/talk/pycon2011/pypy-optimizations.txt
Log:
oops. I think this it the final version, was laying down in my wc
Modified: pypy/extradoc/talk/pycon2011/pypy-optimizations.txt
==============================================================================
--- pypy/extradoc/talk/pycon2011/pypy-optimizations.txt (original)
+++ pypy/extradoc/talk/pycon2011/pypy-optimizations.txt Sun Oct 31 16:55:53 2010
@@ -4,27 +4,35 @@
PyPy is a virtual machine for Python, featuring an advanced just in time
compiler, which can deliver exceptional performance. This talk is going to be
a deep dive into what exactly makes Python such a hard language to optimize,
-how PyPy is organized, and what optimizations our JIT can do for your code.
+how PyPy is organized, and what optimizations our JIT can do
+(and what it can't do) for your code.
+
+This talk will also contain a brief part about status of the PyPy project,
+it's goals and achievements.
Session type: Talk
Classification: Discuss in depth
Abstract:
-The talk will contain a list of features that are hard to optimize in Python,
-why and how we're fixing it in PyPy. This also might be used as a rough guide
-what features in Python are optimizable right now and what are not.
+The talk will explain a bit in details how a python interpreter
+internally and why some operations are costly. We'll go through several
+python features, how the work, why they're slow in CPython and how we're
+fixing it.
The list of mentioned features is not exhaustive, however we will try to
focus at least on the following:
* Dynamic language - In Python code we have no known types, like a statically typed
language. Even operations like "a + b" can do anything, unless we know more
- about the code.
+ about the code, especially associated types.
* Frame introspection - Frame objects need to be allocated for every function
call, and all local variables are stored on the frame, and must be accessible
from further down the call stack.
+ PyPy uses a novel approach called "virtualizables" which makes it possible
+ to not allocate frames at all in most common cases.
+
* Object model - All user defined Python objects have a dictionary which stores
their attributes, as does every type. When Python does an attribute lookup
this requires at least two dictionary lookups.
@@ -36,8 +44,8 @@
decided to go via ctypes, this part will explain how are we going to handle
ctypes calls to be fast.
-* `array` module - users of the CPython array module probably know it can save
- them quiet a bit of memory, however it's also slower than using a list, due
+* `array` module - users of the CPython's array module probably know it can save
+ them quite a bit of memory, however it's also slower than using a list, due
to the overhead of boxing and unboxing on every operations. Here we will tie
everything together and describe how the ``array`` module is much faster with
PyPy's JIT, combining our optimizations to: unbox values, remove the
From agaynor at codespeak.net Sun Oct 31 17:04:44 2010
From: agaynor at codespeak.net (agaynor at codespeak.net)
Date: Sun, 31 Oct 2010 17:04:44 +0100 (CET)
Subject: [pypy-svn] r78627 - pypy/extradoc/talk/pycon2011
Message-ID: <20101031160444.94ACB282BE3@codespeak.net>
Author: agaynor
Date: Sun Oct 31 17:04:42 2010
New Revision: 78627
Modified:
pypy/extradoc/talk/pycon2011/pypy-optimizations.txt
Log:
Random grammar and typo fixes.
Modified: pypy/extradoc/talk/pycon2011/pypy-optimizations.txt
==============================================================================
--- pypy/extradoc/talk/pycon2011/pypy-optimizations.txt (original)
+++ pypy/extradoc/talk/pycon2011/pypy-optimizations.txt Sun Oct 31 17:04:42 2010
@@ -14,17 +14,16 @@
Classification: Discuss in depth
Abstract:
-The talk will explain a bit in details how a python interpreter
-internally and why some operations are costly. We'll go through several
-python features, how the work, why they're slow in CPython and how we're
-fixing it.
+The talk will explain a bit in details how a python interpreter works
+internally and why some operations are costly. We'll go through several python
+features, how they work, why they're slow in CPython and how we're fixing it.
The list of mentioned features is not exhaustive, however we will try to
focus at least on the following:
-* Dynamic language - In Python code we have no known types, like a statically typed
- language. Even operations like "a + b" can do anything, unless we know more
- about the code, especially associated types.
+* Dynamic language - In Python code we have no known types, like a statically
+ typed language. Even operations like "a + b" can do anything, unless we know
+ more about the code, especially associated types.
* Frame introspection - Frame objects need to be allocated for every function
call, and all local variables are stored on the frame, and must be accessible
@@ -41,10 +40,10 @@
(except more PyPy specific) called map dictionaries. XXX type versions?
* FFI calls - calling C from Python is costly and hard to optimize. In PyPy we
- decided to go via ctypes, this part will explain how are we going to handle
- ctypes calls to be fast.
+ expse C APIs to Python code via ctypes, this part will explain how are we
+ going to handle ctypes calls to be fast.
-* `array` module - users of the CPython's array module probably know it can save
+* `array` module - users of CPython's array module probably know it can save
them quite a bit of memory, however it's also slower than using a list, due
to the overhead of boxing and unboxing on every operations. Here we will tie
everything together and describe how the ``array`` module is much faster with